Deep Lake is a Python library for building, managing, and querying multi-modal datasets for AI. It enables storing and streaming data (images, videos, audio, text, embeddings) directly from cloud storage to machine learning models, supporting various operations like version control, indexing, and complex queries. As of version 4.5.10, it features a C++ core for enhanced performance and offers robust data management for AI workflows. The project is actively developed with frequent minor releases.
pip install deeplakeVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to create a new Deep Lake dataset, define its schema with tensors for images and labels, append synthetic data, and then load and query the dataset. It shows how to handle authentication via environment variables (recommended for non-interactive use) and provides options for local or cloud storage paths.
Old: `ds = deeplake.dataset.create_empty(path)`. New: `ds = deeplake.empty(path)`. Tensor access often changed from `ds.tensors.foo.append()` to `ds.foo.append()`.
Migrate `activeloop.login()` calls to `deeplake.login()` or set the `DEEPLAKE_TOKEN` environment variable for non-interactive authentication.
Iterate over samples or use slicing to process data in chunks. E.g., `for sample in ds.tensor_name: ...` or `ds.tensor_name[start:end].numpy()` for smaller slices.
Ensure the path format is correct, including the `hub://` prefix for cloud datasets. Verify `DEEPLAKE_TOKEN` grants access to the specified organization.
Use `ds.create_tensor()` to define `htype` (e.g., 'image', 'text') and `sample_compression`. Ensure appended NumPy arrays or other data types match expectations (e.g., RGB images should be (H, W, 3)).
Run `deeplake.login()` interactively, or set the `DEEPLAKE_TOKEN` environment variable to your Activeloop token before running your script.
Verify the dataset path is correct. For Hub datasets, ensure the token has access to the specified path. For local datasets, check file system permissions.
To get the number of samples in a dataset, use `len(ds)`. To get the length of a specific tensor (number of samples it contains), use `len(ds.tensor_name)`.
Ensure the `htype` of the tensor matches the type of data you are appending. If you need to store different types, create separate tensors or define a more generic `htype` if applicable, or define a flexible schema.
No dependency data recorded yet.