Install & Compatibility
Where this runs
tested against v0.4.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 19.3MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 2.6s · import 0.000s · 20MB
17MB installed
● package 17MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
ceja
✓ import ceja
All functions are exposed directly under the 'ceja' module namespace after a simple import.
This quickstart demonstrates how to initialize a SparkSession, create a DataFrame, and apply a ceja string matching function (damerau_levenshtein_distance) to columns within the DataFrame. Ensure findspark is installed and initialized if you're not running directly on a Spark cluster.
import findspark
findspark.init() # Initialize findspark if not running in a native Spark environment
from pyspark.sql import SparkSession
import pyspark.sql.functions as sf
import ceja
# Create a SparkSession
spark = SparkSession.builder.appName("CejaQuickstart").getOrCreate()
# Sample data
data = [ ("jellyfish", "smellyfish"), ("li", "lee"), ("luisa", "bruna"), (None, None) ]
df = spark.createDataFrame(data, ["word1", "word2"])
# Apply a ceja function (e.g., damerau_levenshtein_distance)
result_df = df.withColumn(
"damerau_levenshtein_distance",
ceja.damerau_levenshtein_distance(sf.col("word1"), sf.col("word2"))
)
result_df.show()
# Stop SparkSession
spark.stop()
Debug
Known issues
gotchaceja functions are designed for PySpark DataFrames. Attempting to use them directly on native Python strings or non-Spark data structures will result in runtime errors like 'AttributeError' or 'TypeError'.fixAlways pass Spark DataFrame columns (e.g., `sf.col("column_name")`) to ceja functions after importing `pyspark.sql.functions as sf`. affects: All
gotchaThe library's last update was in February 2023. While generally stable, this slow release cadence might lead to compatibility issues with very recent versions of Python or PySpark, or a slower response to new feature requests/bug fixes.fixTest thoroughly with your specific PySpark and Python environment. Consider pinning versions if stability is critical.
affects: All current versions (0.4.0)
gotchaThe project lacks extensive official documentation beyond the GitHub README and has an empty project description on PyPI, which can make advanced usage or troubleshooting more challenging.fixRefer to the GitHub repository's README for available functions and basic usage patterns. Inspect the source code if deeper understanding is required.
affects: All
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'pyspark'
The PySpark library is not installed or not correctly configured in your Python environment. ceja relies entirely on PySpark for its functionality.
fixInstall PySpark using `pip install pyspark` and ensure your environment variables (like `SPARK_HOME`) are correctly set if running locally.
AttributeError: 'str' object has no attribute '_jc_op_eq'
You are attempting to pass a literal Python string to a ceja function which expects a Spark Column object.
fixWrap your string literals with `sf.lit()` or ensure you are passing `sf.col("your_column")` to ceja functions. TypeError: Column is not iterable
A ceja function received an incorrect type, likely a Spark Column object where a Python iterable or a different Column operation was expected, or vice-versa.
fixVerify the expected input types for the specific ceja function you are using. Ensure you are applying the function to the correct Spark DataFrame columns.
Upgrade
Version history
0.4.0latest on PyPI · released Feb 23, 2023
Audit
Dependencies
pysparkrequiredceja functions are designed to operate on PySpark DataFrames and require a running Spark environment.
Resources
No resource links recorded.