Registry / ai-ml / h2o-pysparkling-3-1

h2o-pysparkling-3-1

library3.46.0.6.post1pypypiunverified

Sparkling Water integrates H2O's Fast Scalable Machine Learning with Apache Spark, enabling scalable ML workflows. Current version: 3.46.0.6.post1. Release cadence follows H2O-3 major/minor releases.

ai-ml data devops

pip install h2o-pysparkling-3.1

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · H2O-PYSPARKLING-3-

h2o-pysparkling-3-1

ai-mlpythonv3.46.0.6.post1

Install

17.1s avg

Import

—

Disk

258MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v3.46.0.6.post1 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.000s · 259.1MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 17.1s · import 0.000s · 260MB

258MB installed

● package 258MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

H2OContext

✓ from pysparkling import H2OContext

✗ from h2o import H2OContext

H2OContext is in pysparkling, not h2o

✓ from pysparkling import HC

✗ from h2o import HC

HC is a shortcut in pysparkling

Initialize Spark and H2OContext. Must be run in a Spark environment (pyspark shell or submitted job).

from pyspark.sql import SparkSession from pysparkling import H2OContext spark = SparkSession.builder.appName('app').getOrCreate() sc = spark.sparkContext # Initialize H2OContext h2o_context = H2OContext.getOrCreate(sc) # Start H2O services h2o_context.start() print(f'H2O cluster status: {h2o_context.cluster().status()}')

Debug

Known issues

breakingPySparkling 3.2+ requires Spark 3.2.x; PySparkling 3.1 requires Spark 3.1.x. Using wrong Spark version causes runtime errors.

fix

Match the major.minor version of h2o-pysparkling with your Spark version. For Spark 3.1.x, use h2o-pysparkling-3.1.

affects: all

deprecatedThe H2OContext API has changed. Older code using H2OContext(sc) directly may fail; use H2OContext.getOrCreate(sc) or H2OContext(sc).

fix

Use H2OContext.getOrCreate(spark.sparkContext) or H2OContext(sc) depending on version. Check Sparkling Water changelog for exact changes.

affects: >=3.36

gotchaPySparkling requires Java 8 or 11. Java 17+ is not supported and will cause cryptic errors.

fix

Set JAVA_HOME to Java 8 or 11 before starting Spark.

affects: all

gotchaH2OContext must be initialized inside a Spark context (e.g., in a PySpark shell or Spark job). Running outside Spark (plain Python) fails with 'No SparkContext found'.

fix

Run code via spark-submit or pyspark shell.

affects: all

Upgrade

Version history

3.46.0.6.post1latest on PyPI · released Nov 19, 2024

Audit

Dependencies

No dependency data recorded yet.

Agent activity

20 hits · last 30 days

node

Anthropic

OpenAI (training)

Resources

githubhttps://github.com/h2oai/sparkling-water ↗

packagehttps://pypi.org/project/h2o-pysparkling-3-1/ ↗

homepagehttps://www.h2o.ai/products/h2o-sparkling-water/ ↗