Install & Compatibility
Where this runs
tested against v1.0.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.000s · 17.9MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 1.6s · import 0.000s · 18MB
16MB installed
● package 16MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
make_tokenizer
✓ from funcparserlib.lexer import make_tokenizer
TokenSpec
✓ from funcparserlib.lexer import TokenSpec
Token
✓ from funcparserlib.lexer import Token
tok
✓ from funcparserlib.parser import tok
Parser
✓ from funcparserlib.parser import Parser
many
✓ from funcparserlib.parser import many
forward_decl
✓ from funcparserlib.parser import forward_decl
finished
✓ from funcparserlib.parser import finished
This quickstart demonstrates how to define a tokenizer using `make_tokenizer` and then build a simple parser for 'number operator number' expressions using `funcparserlib`'s combinators like `tok`, `+`, and `>>`. It covers tokenizing input, defining terminal parsers, and combining them into a full grammar, finally parsing a sample string.
from typing import List
from funcparserlib.lexer import make_tokenizer, TokenSpec, Token
from funcparserlib.parser import tok, Parser, many, finished
def tokenize(s: str) -> List[Token]:
specs = [
TokenSpec('whitespace', r'\s+'),
TokenSpec('number', r'\d+'),
TokenSpec('op', r'[+-*/()]'),
]
tokenizer = make_tokenizer(specs)
return [t for t in tokenizer(s) if t.type != 'whitespace']
# Define parsers
number_parser: Parser[Token, int] = tok('number') >> (lambda t: int(t.value))
op_parser: Parser[Token, str] = tok('op') >> (lambda t: t.value)
# A simple grammar for 'number op number'
expr_parser: Parser[Token, tuple] = number_parser + op_parser + number_parser
# Parse a string
input_str = "123 + 45"
tokens = tokenize(input_str)
result = (expr_parser + -finished).parse(tokens)
print(f"Input: {input_str}")
print(f"Tokens: {tokens}")
print(f"Parsed result: {result}")
# Expected: (123, '+', 45)
Debug
Known issues
breakingFuture versions (e.g., 2.0.0 and beyond) will drop support for Python 2.7. If you rely on Python 2.7, ensure you pin your `funcparserlib` version to `<2.0.0` (e.g., `~=1.0`).fixUpgrade your Python environment to 3.8+ or pin `funcparserlib` to a 1.x version if Python 2.7 compatibility is required.
affects: >=2.0.0 (upcoming)
breakingVersion 1.0.0 dropped support for Python versions 3.4, 3.5, and 3.6. Using `funcparserlib` 1.0.0+ with these Python versions will lead to incompatibilities.fixUpgrade your Python environment to 3.7+ (or 3.8+ for full current support) to use `funcparserlib` 1.0.0 or newer.
affects: 1.0.0+
gotchaThe `Parser.__init__()` constructor is considered internal and may change in future versions. Directly instantiating `Parser` is discouraged.fixAlways use primitive parsers like `tok()`, `a()`, `some()`, `forward_decl()`, `finished`, and parsing combinators (`+`, `|`, `>>`, `many()`, etc.) to construct new parser objects.
affects: All versions
gotchaWhen combining parsers with the `+` operator (e.g., `p1 + p2`), the result can be a tuple of parsed values. If any component parser is skipped (e.g., `-p` or `skip(p)`), the tuple structure might be affected, potentially leading to unexpected results if not explicitly handled.fixExplicitly transform the parsing result into the desired structure using the `>>` operator with a lambda function (e.g., `(p1 + p2) >> (lambda x: MyNode(x[0], x[1]))`) or use `-p` / `skip(p)` to prevent unwanted elements from appearing in the result tuple.
affects: All versions
Errors
Common errors & fixes
SyntaxError: multiple exception types must be parenthesized
Using an older version of funcparserlib (e.g., 0.3.6) with modern Python versions (e.g., Python 3.13) which no longer support Python 2-style exception syntax (e.g., `except NoParseError, e:`).
fixUpgrade `funcparserlib` to version 1.0.0 or newer, which has modernized its codebase for Python 3 compatibility. `pip install --upgrade funcparserlib`.
funcparserlib.parser.NoParseError: got unexpected token: X
In older versions of funcparserlib, parse error messages were less informative, only showing the unexpected token without indicating what was expected.
fixUpgrade `funcparserlib` to version 1.0.0 or newer. Version 1.0.0 improved parse exceptions to include expected tokens and grammar rules at the stopped position, making debugging easier.
TypeError: 'str' object is not callable (when using `>>` with an unexpected argument)
The `>>` operator expects a function as its right-hand operand to transform the parser's result. If a non-callable object (like a string) is provided, this error occurs.
fixEnsure the right-hand side of `>>` is a callable (e.g., a function, a lambda, or a type constructor) that accepts the output of the left-hand parser and returns the desired transformed value.
Upgrade
Version history
1.0.1latest on PyPI · released Nov 3, 2022
Audit
Dependencies
No dependency data recorded yet.