Polars GPU engine powered by cuDF [Open Beta]
BRINGING ACCELERATED COMPUTING TO POLARS
Zero Code Change
Pass engine="gpu"
to the Polars collect
API to add GPU acceleration to your data processing tasks like data aggregation, merges, window functions, and much more.
Ecosystem Compatible
Polars' growing ecosystem of data visualization, IO, & ML libraries can be accelerated with the GPU engine.
Seamless CPU Fallback
For queries where one or more operations are not supported in cuDF, the entire query execution will gracefully fallback to the default CPU engine.
Query Optimizer
The GPU engine utilizes Polars' query optimizer to ensure efficient execution and minimal memory usage.
Process Hundreds of Millions of Rows in Seconds
Performance improves as data size grows
Accelerate Polars Workflows up to 13x
PDS-H benchmark | GPU: NVIDIA H100 | CPU: Intel Xeon W9-3495X (Sapphire Rapids) | Storage: Local NVMe
Note: PDS-H is derived from TPC-H but these results are not comparable to TPC-H results. Learn More
Get Started
Install Polars with pip using the the GPU engine "extra" and the NVIDIA PyPI index:
pip install polars[gpu] --extra-index-url=https://pypi.nvidia.com
Default GPU Engine
Materialize the Polars LazyFrame into a DataFrame with the default GPU engine configuration by calling .collect(engine="gpu")
:
import polars as pl
ldf = pl.LazyFrame({"a": [1.242, 1.535]})
print(
ldf.select(
pl.col("a").round(1)
).collect(engine="gpu")
)
Custom GPU Engine
For finer control on the engine, you can pass a GPUEngine
object with additional configurations like device details and verbosity to the engine=
parameter:
import polars as pl
ldf = pl.LazyFrame({"a": [1.242, 1.535]})
gpu_engine = pl.GPUEngine(
device=0, # This is the default
raise_on_fail=True, # Fail loudly if can't execute on the GPU
)
print(
ldf.select(
pl.col("a").round(1)
).collect(engine=gpu_engine)
)
Frequently Asked Questions
The following command will install Polars with the required RAPIDS packages:pip install polars[gpu] --extra-index-url=https://pypi.nvidia.com
Yes, you can use the Polars GPU engine on a Windows machine by installing WSL2 (requires Windows 11 or higher). For download and troubleshooting instructions, please visit the RAPIDS installation page.
Currently, the GPU engine only supports Polars' Python API.
The GPU engine currently supports most of the core expressions and data types, but not all. For a detailed list of supported expressions visit the GPU Support page on the Polars website.
See the Polars GPUEngine API documentation.
Raise an issue on the Polars GitHub and the project contributors will respond as soon as possible.
Accelerate Polars with RAPIDS
Try Now
Try the Polars GPU engine on Google Colab
Learn More
Find out more on the Polars GPU Support Page
Share Feedback
Share feedback on the Polars GitHub repository