yq

library3.4.3pypypiunverified

yq is a Python-based command-line processor for YAML, XML, and TOML documents, acting as a lightweight wrapper that transparently converts these formats to JSON and pipes them to the `jq` command-line tool for querying and manipulation. It's currently at version 3.4.3 and maintains an active release cadence with regular updates and fixes.

data devops

pip install yq

github ↗package ↗

INSTALL

IMPORT

SIG · YQ

datapythonv3.4.3

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

The `yq` library primarily provides command-line utilities (`yq`, `xq`, `tomlq`). To use it programmatically in Python, you typically invoke these commands via `subprocess`. This example demonstrates how to create a YAML file, extract a value, and perform a transformation using `yq`.

import subprocess import os # Create a sample YAML file yaml_content = """ name: Jane Doe age: 28 city: San Francisco details: occupation: Developer hobbies: [coding, gaming] """ with open("sample.yaml", "w") as f: f.write(yaml_content) print("--- Original YAML ---") print(yaml_content) # Process the YAML file using yq to extract the name try: # Example 1: Get the name result_name = subprocess.run( ["yq", ".name", "sample.yaml"], capture_output=True, text=True, check=True ) print(f"\n--- Extracted Name --- {result_name.stdout.strip()}") # Example 2: Update the age and city using a jq-like expression result_updated = subprocess.run( ["yq", '.age = 29 | .city = "Seattle"', "sample.yaml"], capture_output=True, text=True, check=True ) print("\n--- Updated YAML (age 29, city Seattle) ---") print(result_updated.stdout.strip()) except FileNotFoundError: print("\nError: 'yq' command not found. Please ensure yq (this Python package and the external 'jq' binary) is installed and in your system PATH.") except subprocess.CalledProcessError as e: print(f"\nError processing YAML with yq: {e}") print(f"Stderr: {e.stderr.strip()}") finally: # Clean up the sample file if os.path.exists("sample.yaml"): os.remove("sample.yaml")

yq --version

Debug

Known issues

gotchaThe `yq` Python library depends on the external `jq` command-line tool. You must have `jq` installed and available in your system's PATH for `yq` to function correctly. This is not a Python dependency.

fix

Install `jq` via your system's package manager (e.g., `brew install jq` on macOS, `sudo apt-get install jq` on Debian/Ubuntu).

affects: All versions

breakingVersion 3.4.0 changed the behavior of `yq -y` to induce quoting for string scalars that start with '08' or '09' to prevent them from being interpreted as octal numbers by some YAML parsers. This might change the output format for certain string values.

fix

Review outputs for YAML documents containing strings starting with '08' or '09'. If consistent non-quoted output is required, consider post-processing or adjusting input data if possible.

affects: >=3.4.0

gotchaVersions 3.3.0 and 3.3.1 had conflicting behaviors regarding the interpretation of characters that cannot be parsed in octal as integers. Version 3.3.0 attempted to prevent this, but 3.3.1 reverted that change. This could lead to inconsistent data interpretation if documents contain strings resembling octal numbers.

fix

Be cautious when handling YAML/XML/TOML data that might contain strings like '08' or '09'. Ensure your version of `yq` (and `PyYAML` indirectly) handles these as strings or explicitly quote them in your source data.

affects: 3.3.0 - 3.3.1

deprecatedEarlier versions used the `toml` library for TOML processing. Version 3.2.0 switched to `tomlkit` for better round-trip preservation and more robust handling. While this was an internal change, it's good to be aware of the underlying parser for TOML files.

fix

Ensure you are using `yq` version 3.2.0 or newer for improved TOML parsing capabilities, especially if round-trip fidelity is important.

affects: <3.2.0

Errors

Common errors & fixes

yq: command not found

The yq executable is not in your system's PATH, typically due to an incomplete installation or the pip binary directory not being included in PATH.

fix

Install yq using `pip install yq` and ensure your system's PATH environment variable includes the directory where pip installs executables (e.g., `~/.local/bin` on Linux/macOS or `C:\PythonXX\Scripts` on Windows).

Error: `jq` command not found

The `kislyuk/yq` tool relies on the `jq` command-line utility for processing JSON, which is not installed or not accessible in your system's PATH.

fix

Install `jq` separately for your operating system (e.g., `brew install jq` on macOS, `sudo apt-get install jq` on Debian/Ubuntu, `choco install jq` on Windows) and verify it's in your PATH.

yq: error: argument 'query': expected one argument

You have provided multiple positional arguments where `yq` expects a single `jq` query string, followed by optional input file paths.

fix

Wrap your `jq` query in a single set of quotes (e.g., `yq '.foo | .bar' my_file.yaml`) to ensure it is parsed as one argument.

yq: error: unrecognized arguments: -r

You are attempting to pass `jq`-specific flags (like `-r` for raw output, or `-c` for compact output) directly to `yq`, which only recognizes its own limited set of flags.

fix

Pass `jq` flags within the `jq` query string (e.g., `yq -y '.foo | @text' my_file.yaml` for raw text output), or pipe `yq`'s output to `jq` directly (e.g., `yq .foo my_file.yaml | jq -r .`).

Upgrade

Version history

3.4.3latest on PyPI · released Apr 27, 2024

Audit

Dependencies

PyYAMLrequiredRequired for YAML parsing and serialization.

lxmlrequiredRequired for XML parsing and serialization.

tomlkitrequiredRequired for TOML parsing and serialization.

Agent activity

37 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/kislyuk/yq ↗

packagehttps://pypi.org/project/yq/ ↗