Install & Compatibility
Where this runs
tested against v0.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 17.8MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 1.6s · import 0.000s · 18MB
16MB installed
● package 16MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
current_spark_context
✓ from databricks.connect_extras.context import current_spark_context
A utility to get the active SparkSession, typically from Databricks Connect or a Databricks notebook environment.
get_current_notebook_path
✓ from databricks.connect_extras.notebook import get_current_notebook_path
A utility to retrieve the path of the current Databricks notebook.
This quickstart demonstrates how to use `current_spark_context` from `databricks.connect_extras` to obtain a SparkSession. This utility is particularly useful when working with Databricks Connect v2 or directly within a Databricks notebook environment. Outside these contexts, it requires `databricks-connect` to be installed and configured.
import os
from databricks.connect_extras.context import current_spark_context
# This example demonstrates retrieving a Spark session using databricks.connect_extras.
# To run this successfully outside a Databricks Notebook, you must:
# 1. Install Databricks Connect: pip install "databricks-connect[databricks-connect-dependencies]"
# 2. Configure Databricks Connect using `databricks-connect configure`
# or by setting environment variables (DATABRICKS_HOST, DATABRICKS_TOKEN, DATABRICKS_CLUSTER_ID, etc.).
print("Attempting to get Spark session via databricks.connect_extras...")
try:
spark = current_spark_context()
if spark:
print(f"Successfully retrieved SparkSession (Spark version: {spark.version})")
# Example usage: create a simple DataFrame
data = [("Alice", 1), ("Bob", 2), ("Charlie", 3)]
df = spark.createDataFrame(data, ["Name", "Value"])
print("\nExample DataFrame created and shown:")
df.show()
else:
print("Spark session could not be retrieved. Ensure Databricks Connect is properly configured or run in a Databricks Notebook.")
except Exception as e:
print(f"An error occurred: {e}")
print("Please ensure `databricks-connect` is installed and configured, and your environment is set up for Databricks Connect v2.")
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'databricks.connect_extras.context'
The `databricks-pypi-extras` library or a specific submodule was not installed, or the import path is incorrect.
fixEnsure you have installed the library using `pip install databricks-pypi-extras`. Verify the import path against the official GitHub repository's `src` directory, as the library is modular and may introduce new sub-packages.
Spark session could not be retrieved. Ensure Databricks Connect is properly configured or run in a Databricks Notebook.
Attempting to use `current_spark_context()` or similar utilities from `databricks.connect_extras` outside of a properly configured Databricks Connect environment or a Databricks Notebook.
fixInstall `databricks-connect` (`pip install "databricks-connect[databricks-connect-dependencies]"`) and configure it using `databricks-connect configure`, or run your code directly within a Databricks notebook.
Upgrade
Version history
0.1latest on PyPI · released Feb 14, 2019
Audit
Dependencies
databricks-connectoptionalRequired for utilities in `databricks.connect_extras` module to function, particularly for Databricks Connect v2 features.
Resources
No resource links recorded.