semhash

library0.4.1pypypi✓ verified 88d ago

A fast Python library for multimodal semantic deduplication and filtering. Currently at version 0.4.1, with active development and an irregular release cadence.

ai-ml search

pip install semhash

github ↗homepage ↗

INSTALL

IMPORT

SIG · SEMHASH

semhash

ai-mlpythonv0.4.1

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

SemHash

✓ from semhash import SemHash

deduplicate

✓ from semhash import deduplicate

Basic example showing how to compute semantic hashes and deduplicate text.

from semhash import SemHash # Create a SemHash instance (default uses a lightweight embedding model) hasher = SemHash() texts = [ "The quick brown fox jumps over the lazy dog.", "A quick brown fox jumps over a lazy dog.", "Lazy dog is sleeping." ] # Compute hashes hashes = hasher.encode(texts) # Find duplicates within the list duplicates = hasher.find_duplicates(texts, threshold=0.7) print(duplicates) # Or deduplicate them directly unique_texts = deduplicate(texts, threshold=0.7) print(unique_texts)

Debug

Known issues

gotchaSemHash uses a default embedding model (typically a lightweight transformer) which might not capture all nuances. For domain-specific data, consider providing a custom model.

fix

Pass a custom model via SemHash(model_name='path/to/model').

affects: all

breakingIn version 0.4.0, the API for finding duplicates changed: find_duplicates now returns a list of tuples instead of a dict. The deduplicate function was also added.

fix

Update code to expect list of tuples (e.g., [(index, duplicate_index), ...]). Before 0.4.0, it returned a dict mapping indices to lists of indices.

affects: >=0.4.0

deprecatedThe 'model' parameter in SemHash constructor is deprecated as of 0.4.0, use 'model_name' instead.

fix

Use model_name='...' instead of model='...'.

affects: >=0.4.0

Errors

Common errors & fixes

AttributeError: module 'semhash' has no attribute 'deduplicate'

The deduplicate function was added in version 0.4.0.

fix

Upgrade semhash to 0.4.0 or later: pip install --upgrade semhash

ImportError: cannot import name 'SemHash' from 'semhash'

The package might not be installed correctly, or an older version lacks this class.

fix

Ensure semhash is installed: pip install semhash. If that fails, check for conflicting packages.

OSError: Model 'all-MiniLM-L6-v2' not found or not supported

The default model relies on the sentence-transformers library, which may not be installed or the model cannot be downloaded.

fix

Install sentence-transformers: pip install sentence-transformers. Or specify a different model via model_name parameter.

Upgrade

Version history

0.4.1latest on PyPI · released Jan 20, 2026

Audit

Dependencies

torchoptionalRequired for model inference and embeddings.

transformersoptionalUsed if you load transformer-based models for hashing.

Agent activity

72 hits · last 30 days

node

Amazon

OpenAI (training)

Resources

githubhttps://github.com/MinishLab/semhash ↗

homepagehttps://github.com/MinishLab/semhash ↗