Registry / data / dvc-gs

dvc-gs

JSON →
library3.1.0pypypi✓ verified 89d ago

dvc-gs is the official Google Cloud Storage plugin for DVC (Data Version Control). It enables DVC to store and retrieve data artifacts from Google Cloud Storage buckets, allowing users to version large files and models in the cloud. The current version is 3.0.2, and it typically follows DVC's release cadence, with frequent updates to align with core DVC features and bug fixes.

pip install dvc-gs
INSTALL
IMPORT
SIG · DVC-GS
D
dvc-gs
datapythonv3.1.0
Install
23.5s avg
Import
1109ms
Disk
202MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.9–3.13
musl
3.9–3.13
Install & Compatibility
Where this runs
tested against v3.1.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.10–3.940 runs
installs and imports cleanly · install 0.0s · import 1.150s · 196.1MB
glibc
py 3.10–3.940 runs
installs and imports cleanly · install 23.5s · import 1.068s · 198MB
202MB installed
● package 202MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

dvc.api
✓ import dvc.api # dvc-gs functionality is implicitly used by dvc.api calls # when a Google Cloud Storage remote is configured.
Users typically interact with dvc-gs functionality indirectly through DVC's CLI or dvc.api, rather than direct Python imports from the dvc_gs package itself. dvc-gs provides the backend filesystem implementation.

This quickstart demonstrates how to initialize a DVC repository, configure a Google Cloud Storage remote using `dvc-gs`, add a data file, and push it to the GCS bucket. It assumes DVC is installed and you have appropriate GCS credentials configured (e.g., `GOOGLE_APPLICATION_CREDENTIALS` environment variable or `gcloud` authenticated). Replace `your-gcs-bucket` with your actual bucket name.

import os from dvc.repo import Repo # Initialize DVC in a new directory os.makedirs('my_project', exist_ok=True) os.chdir('my_project') repo = Repo.init() # Configure a Google Cloud Storage remote # Replace 'your-gcs-bucket' with your actual bucket name # Ensure GOOGLE_APPLICATION_CREDENTIALS points to a service account key or use gcloud auth if not os.environ.get('GOOGLE_APPLICATION_CREDENTIALS'): print("Warning: GOOGLE_APPLICATION_CREDENTIALS not set. Ensure gcloud is authenticated or anonymous access is allowed for the bucket.") repo.remote.add( name='my_gs_remote', url=f'gs://{os.environ.get("GCS_BUCKET_NAME", "your-gcs-bucket")}' ) # Create a dummy data file with open('data.txt', 'w') as f: f.write('hello dvc-gs') # Add the file to DVC and push to GCS repo.add('data.txt') repo.push('data.txt') print("data.txt added and pushed to GCS.") # To verify, you can pull the data into another location or check your GCS bucket # For example, in a new directory: # os.chdir('..') # os.makedirs('another_project', exist_ok=True) # os.chdir('another_project') # Repo.clone('path/to/my_project', 'another_project') # new_repo = Repo('.') # new_repo.pull('data.txt')
dvc --version
Debug
Known issues
breakingDVC 3.0.0 (and consequently dvc-gs 3.0.0) introduced significant changes to the DVC API, particularly affecting programmatic usage with `dvc.api.get_url()`, `dvc.api.open()`, and `dvc.api.read()`. Code written for DVC 2.x will likely break with DVC 3.x.
fix
Consult the DVC 3.0 migration guide for updated API usage. For example, `dvc.api.get_url()` now requires `url` and `path` arguments.
affects: 3.0.0+
gotchaDVC-GS relies on Google Cloud credentials. Without proper authentication, operations will fail with permission errors. Common methods include setting `GOOGLE_APPLICATION_CREDENTIALS` or using `gcloud auth application-default login`.
fix
Ensure your environment is correctly authenticated with Google Cloud. For programmatic access, `GOOGLE_APPLICATION_CREDENTIALS` pointing to a service account key JSON file is recommended. For CLI, `gcloud auth login` and `gcloud auth application-default login` are typical.
affects: All
gotchaWhile `dvc-gs` version 3.0.2 introduced support for anonymous GCS login, previous versions or specific bucket configurations might still require explicit authentication even for public data, leading to unexpected `AccessDenied` errors.
fix
Check your `dvc-gs` version. If older than 3.0.2, upgrade. Always verify bucket permissions and access policies on Google Cloud Platform, even for publicly accessible buckets, to ensure DVC has the necessary read/write permissions.
affects: <3.0.2, some 3.0.2+ configurations
Errors
Common errors & fixes
ERROR: Plugin 'gs' is not found. Check if the plugin is installed.
The dvc-gs plugin is not installed or DVC cannot find it. This usually happens if you install `dvc` but forget `dvc-gs` or `dvc[gs]`.
fix
Install the plugin: `pip install dvc-gs` or `pip install dvc[gs]`.
gcsfs.exceptions.NoCredentialsError: NoCredentialsError: Could not determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS, provide explicit credentials, or load them from gcloud config.
DVC-GS (via gcsfs) could not find any Google Cloud credentials in the environment.
fix
Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of your service account key file, or run `gcloud auth application-default login` to set up user credentials.
dvc.api.DvcException: 'gs://your-gcs-bucket' is not a valid DVC remote or cannot be accessed.
This generic error often indicates issues with the remote configuration (e.g., typo in URL, bucket doesn't exist, or permissions issues even if credentials are found).
fix
Double-check the GCS bucket URL for typos. Verify that the bucket exists and that the configured credentials have appropriate read/write access to it.
Upgrade
Version history
3.1.0latest on PyPI · released Jun 9, 2026
Audit
Dependencies
dvcrequiredThis is a plugin for DVC, providing the Google Cloud Storage backend.
gcsfsrequiredProvides the underlying filesystem interface for Google Cloud Storage. Automatically installed with `dvc-gs` or `dvc[gs]`.
Agent activity
40 hits · last 30 days
node
32
OpenAI (training)
2
Resources
dvc-gs — pip install dvc-gs · libregistry