dagster-polars

library0.27.12pypypi✓ verified 84d ago

Dagster integration library for Polars, providing IO managers, types, and utilities to read/write Polars DataFrames from/to various file formats (Parquet, CSV, Delta Lake, etc.). Current version: 0.27.9. Active development, follows Dagster releases.

workflow data database serialization

pip install dagster-polars

github ↗package ↗

INSTALL

IMPORT

SIG · DAGSTER-POLARS

dagster-polars

workflowpythonv0.27.12

Install

16.1s avg

Import

—

Disk

499MB

Pass rate

9/ 10

Env Coverage9 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.27.12 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✓ —

✓ 17.9s

py 3.11

✓ —

✓ 15.9s

py 3.12

✓ —

✓ 13.7s

py 3.13

✓ —

✓ 14.1s

py 3.9

✕ build_error

✓ 19.1s

499MB installed

● package 499MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

BasePolarsUPathIOManager

✓ from dagster_polars import BasePolarsUPathIOManager

✗ from dagster_polars import PolarsIOManager

PolarsParquetIOManager

✓ from dagster_polars import PolarsParquetIOManager

✗ from dagster_polars import PolarsIOManager

DataFramePartitions

✓ from dagster_polars import DataFramePartitions

Define an asset that returns a Polars DataFrame and use PolarsIOManager to persist it.

from dagster import Definitions, asset from dagster_polars import PolarsIOManager import polars as pl @asset def my_asset() -> pl.DataFrame: return pl.DataFrame({"x": [1, 2, 3]}) defs = Definitions( assets=[my_asset], resources={"io_manager": PolarsIOManager(base_dir="./data")} )

Debug

Known issues

breakingVersion 0.19.0 changed the import path for PolarsIOManager from `dagster_polars.io_manager` to `dagster_polars`. Old imports will break.

fix

Update imports to `from dagster_polars import PolarsIOManager`.

affects: >=0.19.0

gotchaPolarsIOManager by default uses the file extension to determine format. If your asset output doesn't have an explicit extension, it may fail or save as Parquet unexpectedly.

fix

Specify `extension` in the IO manager config or use `PolarsParquetIOManager` / `PolarsCSVIOManager` for explicit format.

affects: all

gotchaWhen using with Dagster's `@asset` decorator, the asset function must return a Polars DataFrame. Returning None or other types may cause errors.

fix

Ensure assets return a `pl.DataFrame` or use `PolarsBaseIOManager` if you need custom behavior.

affects: all

Upgrade

Version history

0.27.12latest on PyPI · released May 22, 2026

Audit

Dependencies

dagsterrequiredCore Dagster framework

polarsrequiredData manipulation library

pyarrowoptionalParquet Arrow support

Agent activity

25 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/dagster-io/community-integrations ↗

packagehttps://pypi.org/project/dagster-polars/ ↗