Install & Compatibility
Where this runs
tested against v0.2.61 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
✓ 83.2s
py 3.11
✕ build_error
✓ 73.7s
py 3.12
✕ build_error
✓ 63s
py 3.13
✕ build_error
✓ 60.6s
py 3.9
✕ build_error
✕ timeout
4915MB installed
● package 4915MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
StreamingDataset
✓ from litdata import StreamingDataset
✗ from lightning.data import StreamingDataset
Old import path from earlier Lightning packages has been removed.
StreamingDataLoader
✓ from litdata import StreamingDataLoader
✗ from litdata.streaming import StreamingDataLoader
StreamingDataLoader is a top-level import.
optimize
✓ from litdata import optimize
✗ from litdata.processing import optimize
optimize moved to top-level.
LightningDataset
✓ from litdata import LightningDataset
Example of using StreamingDataset with a dummy input directory. To use real data, replace the input_dir with a valid URI.
from litdata import StreamingDataset, StreamingDataLoader
# Create a simple streaming dataset
class MyDataset(StreamingDataset):
def __init__(self):
super().__init__(input_dir="s3://my-bucket/data", shuffle=True)
dataset = MyDataset()
dataloader = StreamingDataLoader(dataset, batch_size=32)
for batch in dataloader:
print(batch)
break
Errors
Common errors & fixes
FileNotFoundError: No such file or directory
Input directory does not contain properly formatted chunk files or the path is incorrect.
fixPreprocess your data using `from litdata import optimize; optimize(...)` to create chunks. Ensure the input_dir points to a directory with .bin and .mtx files.
ModuleNotFoundError: No module named 'lightning'
Attempting to import from the old package name 'lightning' instead of 'litdata'.
fixUse `from litdata import StreamingDataset` instead of `from lightning.data import StreamingDataset`.
Upgrade
Version history
0.2.61latest on PyPI · released Feb 20, 2026
Audit
Dependencies
numpyrequiredCore dependency for data handling