Registry / data / conda-package-streaming

conda-package-streaming

library0.13.0pypypi✓ verified 90d ago

An efficient library to read from new and old format .conda and .tar.bz2 conda packages. It enables downloading conda metadata from packages without transferring the entire file and getting metadata from local .tar.bz2 packages without reading entire files. The library, currently at version 0.12.0, uses enhanced pip lazy_wheel for `.conda` files and `tarfile.open` for `.tar.bz2` to stream data efficiently. It maintains a regular release cadence, with major updates roughly every few months.

data serialization

pip install conda-package-streaming

github ↗package ↗docs ↗

INSTALL

IMPORT

SIG · CONDA-PACKAGE-STRE

conda-package-streaming

datapythonv0.13.0

Install

2.4s avg

Import

576ms

Disk

38MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.13.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.920 runs

installs and imports cleanly · install 0.0s · import 0.604s · 36MB

glibc

py 3.10–3.920 runs

installs and imports cleanly · install 2.4s · import 0.547s · 45MB

38MB installed

● package 38MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

stream_conda_info

✓ from conda_package_streaming.url import stream_conda_info

✗ from conda_package_streaming import stream_conda_info

The primary streaming functions like `stream_conda_info` are nested within submodules like `url`, `s3`, or directly under `package_streaming` for local files, not directly under the top-level package.

stream_conda_info (for S3)

✓ from conda_package_streaming.s3 import stream_conda_info

Use the `s3` submodule for streaming directly from S3 buckets, requiring `boto3` to be installed.

stream_conda_info (for local files)

✓ from conda_package_streaming import package_streaming # then use package_streaming.stream_conda_info(...)

For local file paths, `stream_conda_info` is accessed via the `package_streaming` submodule.

conda_reader_for_url

✓ from conda_package_streaming.url import conda_reader_for_url

Provides a file-like object from a URL, used for more granular control over streaming.

This quickstart demonstrates how to stream package metadata (specifically `info/index.json`) from a remote `.conda` or `.tar.bz2` package URL without downloading the entire file. It iterates through the package's members and extracts the `index.json` content.

import json from conda_package_streaming.url import stream_conda_info # Replace with a valid .conda or .tar.bz2 URL # For example: url = 'https://repo.anaconda.com/pkgs/main/linux-64/python-3.9.7-h62f7035_1.conda' url = 'https://repo.anaconda.com/pkgs/main/linux-64/zlib-1.2.13-h5eee18b_0.conda' print(f"Streaming info from: {url}") try: found_index_json = False for tar, member in stream_conda_info(url): if member.name == 'info/index.json': index_json = json.load(tar.extractfile(member)) print("\n--- Found info/index.json ---") print(json.dumps(index_json, indent=2)) found_index_json = True break # Stop once index.json is found if not found_index_json: print("\n--- info/index.json not found in package ---") except Exception as e: print(f"An error occurred: {e}")

Debug

Known issues

gotchaThe behavior of `stream_conda_info` differs between `.conda` and `.tar.bz2` formats. For `.tar.bz2`, it yields all members, while for `.conda`, it yields members from the requested inner archive, allowing early termination.

fix

Be mindful of the package format you are processing. For `.conda` files, you can `break` early once the desired metadata is found. For `.tar.bz2`, be prepared to iterate through more members or implement specific filtering.

affects: All versions

gotchaWhen using `conda_reader_for_url` for `.conda` files, the returned file-like object *must* be seekable. For `.tar.bz2` files, it only needs to be readable.

fix

Ensure your file-like objects for `.conda` are seekable if using `conda_reader_for_url`. If dealing with non-seekable streams and `.conda` packages, consider alternative streaming approaches or pre-downloading if seeking is unavoidable.

affects: All versions

gotchaThis library is optimized for *streaming metadata* and individual components without full package download. Using its URL-based APIs to extract an entire package is inefficient and against its design; prefer pre-downloading the package for full extraction.

fix

If your use case requires the entire package content, download the `.conda` or `.tar.bz2` file first using a standard HTTP client, then use the file-based APIs provided by `conda-package-streaming` or `conda-package-handling`.

affects: All versions

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'conda_package_streaming.package_streaming'

Attempting to import `package_streaming` or its functions directly from the top-level `conda_package_streaming` module, instead of as a submodule.

fix

Correct the import path to `from conda_package_streaming import package_streaming` and then access functions like `package_streaming.stream_conda_info`.

json.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Attempting to `json.load()` a `tarfile.TarInfo` object directly, or an extracted file-like object that is empty or does not contain valid JSON, often due to an incorrect `member.name` check or reading the wrong file.

fix

Ensure `member.name` correctly identifies a JSON file (e.g., `'info/index.json'`) and that `tar.extractfile(member)` returns a valid, non-empty file-like object containing well-formed JSON before attempting to load it.

Upgrade

Version history

0.13.0latest on PyPI · released Jun 8, 2026

Audit

Dependencies

pythonrequiredRequired runtime environment.

requestsrequiredRequired for HTTP requests to fetch remote packages.

zstandardrequiredRequired for handling Zstandard compressed streams (used in .conda packages).

boto3optionalOptional dependency for S3 streaming functionality.

Agent activity

20 hits · last 30 days

node

Google (search)

OpenAI (training)

Resources

githubhttps://github.com/conda/conda-package-streaming ↗

packagehttps://pypi.org/project/conda-package-streaming/ ↗

docshttps://conda.github.io/conda-package-streaming/ ↗