Pandas just got 150 timer Faster

Mar 15 / Sujal Dhungana

Pandas undoubtedly is one of the most loved and most used Python packages. Specially for Data analyst and Data Scientists pandas is an everyday utility. Despite being so popular there always had been one limitation that had frustrated pandas users forever and it was its inability to scale for large datasets. Not anymore. 

With the recent release of RAPIDS cuDF from Nvidia you can use the power of your Nvidia GPU to accelerate your pandas. RAPIDS cuDF is a GPU DataFrame library that provides a pandas-like API for data loading and manipulation. With RAPIDs cuDF v23.10 you can now use the GPU accelerated pandas seamlessly on your program with no code changed. 

This feature was built for data scientists who want to continue using pandas as data sizes grow into the gigabytes and pandas performance slows. In cuDF’s pandas accelerator mode, operations execute on the GPU where possible and on the CPU (using pandas) otherwise, synchronizing under the hood as needed. This enables a unified CPU/GPU experience that brings best-in-class performance to your pandas workflows. 
RAPIDs Website - Nvidia

How can I use it today?

1. I have Nvidia GPU on my Computer

If you have got a cuda enabled GPU on your computer, then you need to follow the following steps to supercharge your pandas.

1.1. Install RAPIDs on your Computer

You only need to do this once.
  • Make sure you have got the proper cuda version on your machine.
  • Install Rapids from here Installation Guide - RAPIDS Docs 

1.2. Accelerate Pandas on Jupyter Notebook

Simple add a line: %load_ext cudf.pandas infront of your pandas import.

1.3. Accelerate Existing Python Scripts

While running the script provide a parameter cudf.pandas as module.
Or update the script to enforce it to use cuda enabled pandas.

2. I don't have Nvidia GPU on my Computer

If you don't have a Nvidia GPU on your computer, you can still make use of accelerated pandas in Google Colab. Simply use this link to create a Google Colab in your account and experience the magic of Pandas in 150x speed.

References

Check out our other blog posts.

Created with