Registry / serialization / unicategories

unicategories

JSON →
library0.1.2pypypi✓ verified 89d ago

unicategories is a Python library that provides a Unicode category database, generated and cached on setup. It exposes a dictionary of `RangeGroup` instances, containing all Unicode category character ranges detected on your system. This module offers an efficient way to work with Unicode character classifications, such as 'Letter, uppercase' (Lu) or 'Number, decimal digit' (Nd), by storing ranges rather than individual characters for memory efficiency. The current version is 0.1.2, released on April 2, 2023.

pip install unicategories
INSTALL
IMPORT
SIG · UNICATEGORIES
U
unicategories
serializationpythonv0.1.2
Install
2.7s avg
Import
66ms
Disk
17MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.9–3.13
musl
3.9–3.13
Install & Compatibility
Where this runs
tested against v0.1.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.068s · 19.4MB
glibc
py 3.10–3.920 runs
installs and imports cleanly · install 2.7s · import 0.063s · 20MB
17MB installed
● package 17MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

categories
✓ from unicategories import categories

This quickstart demonstrates how to import the `categories` dictionary, access `RangeGroup` instances by category code (e.g., 'Lu' for uppercase letters), retrieve characters or code points using `characters()` and `codes()` iterators, and check for character inclusion with `has()`.

from unicategories import categories # Get an iterator for all Unicode uppercase characters upper_characters_iterator = categories['Lu'].characters() print(f"First 10 uppercase characters: {''.join(list(upper_characters_iterator)[:10])}") # Check if a character belongs to a specific category is_digit = categories['Nd'].has('7') print(f"Is '7' a decimal digit? {is_digit}") is_lowercase_a = categories['Ll'].has('a') print(f"Is 'a' a lowercase letter? {is_lowercase_a}") # Get an iterator for all Unicode code points in a category code_points_iterator = categories['Zs'].codes() # Zs: Space Separator print(f"First 5 space separator code points: {list(code_points_iterator)[:5]}")
Debug
Known issues
gotchaThe library primarily uses iterators (e.g., `characters()`, `codes()`) for memory efficiency. If you need a complete list, remember to explicitly convert the iterator to a list or another collection, which will consume more memory.
fix
Use `list(categories['Lu'].characters())` if a full list is required, but be mindful of memory usage for very large categories.
affects: All versions
gotchaWhile `unicategories` supports Python 2.7, Python 2 is end-of-life. It is strongly recommended to use this library with Python 3.5+ to ensure security, maintainability, and compatibility with the latest Unicode standards and Python features.
fix
Ensure your project is running on Python 3.5 or newer. Python 3 handles Unicode natively, reducing potential encoding issues.
affects: All versions supporting Python 2.7
gotchaThis library provides access to Unicode *categories*. For other Unicode character properties (like name, numeric value, bidirectional class), use Python's built-in `unicodedata` module.
fix
Combine `unicategories` for category-based filtering/lookup with `unicodedata` for individual character properties. Example: `import unicodedata; char_name = unicodedata.name('A')`.
affects: All versions
Errors
Common errors & fixes
UnicodeDecodeError: 'charmap' codec can't decode byte X in position Y: character maps to <undefined>
Attempting to read or process text data that was encoded in one format (e.g., UTF-8) but is being decoded using a different, incompatible codec (e.g., Windows-1252 or a default system encoding like 'charmap') by Python. This is a common issue when dealing with files or external data sources not explicitly specified as UTF-8.
fix
Always specify the correct encoding, preferably UTF-8, when opening files or decoding byte strings. For file operations: `with open('filename.txt', 'r', encoding='utf-8') as f: ...`. For byte strings: `my_bytes.decode('utf-8')`.
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position X-Y: truncated \uXXXX escape
This error often occurs on Windows when a backslash (`\`) in a string literal, especially in file paths, is misinterpreted as the start of a Unicode escape sequence (`\u` or `\U`) and is not followed by valid hexadecimal digits.
fix
Use raw strings by prefixing with `r` (e.g., `r'C:\Users\...'`) or double the backslashes (`'C:\\Users\\...'`). Alternatively, use `pathlib.Path` for platform-agnostic path handling: `from pathlib import Path; path = Path('C:/Users/...')`.
UnicodeEncodeError: 'ascii' codec can't encode character '\uXXXX' in position Y: ordinal not in range(128)
Attempting to convert a Unicode string containing non-ASCII characters to an ASCII byte string without specifying an appropriate encoding, or when the target encoding (like 'ascii') cannot represent the characters present.
fix
Explicitly encode the Unicode string into a suitable byte encoding, such as UTF-8, using the `.encode()` method: `my_unicode_string.encode('utf-8')`.
Upgrade
Version history
0.1.2latest on PyPI · released Apr 2, 2023
Audit
Dependencies

No dependency data recorded yet.

Agent activity
19 hits · last 30 days
node
14
Amazon
1
OpenAI (training)
1
Resources
unicategories — pip install unicategories · libregistry