faster-fifo

library1.5.2pypypi✓ verified 88d ago

A drop-in replacement for Python's multiprocessing.Queue that uses a shared-memory ring buffer and a POSIX semaphore for lower latency and higher throughput. Version 1.5.2 requires Python >=3.9. Actively maintained.

database workflow

pip install faster-fifo

github ↗package ↗

INSTALL

IMPORT

SIG · FASTER-FIFO

faster-fifo

databasepythonv1.5.2

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

Queue

✓ from faster_fifo import Queue

✗ from faster_fifo import faster_fifo_queue

The main class is named `Queue`, not `faster_fifo_queue`.

Queue

✓ from faster_fifo import Queue

✗ from multiprocessing import Queue

Use the faster_fifo Queue to get speed improvements; multiprocessing.Queue is the fallback.

Create a Queue, put a message, get it back. Works like multiprocessing.Queue but faster.

from faster_fifo import Queue import os q = Queue() q.put('hello') msg = q.get() print(msg) # hello

Debug

Known issues

gotchaThe underlying implementation uses shared memory; do not pass large objects (e.g., >1GB) without testing, as they may cause memory pressure.

fix

For very large data, consider serializing to disk and sending a path.

affects: all

gotchafaster-fifo Queue is not a drop-in replacement for multiprocessing.Queue in all cases; e.g., it does not support the `timeout` parameter in `get()` (it blocks indefinitely by default).

fix

Use `q.get(timeout=0.5)` is not supported; implement your own timeout with a separate thread/process.

affects: <=1.5.2

gotchaObjects must be picklable (default serializer is pickle). If you use custom objects that are not picklable, the queue will fail.

fix

Ensure all queued objects are picklable or switch to a different serializer (unsupported).

affects: all

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'faster_fifo'

The library is not installed, or installed in a different environment.

fix

Run `pip install faster-fifo --upgrade` in the correct environment.

OSError: [Errno 28] No space left on device

Shared memory filesystem (e.g., /dev/shm) is full. Faster-fifo uses POSIX shared memory.

fix

Increase /dev/shm size (e.g., mount -o remount,size=2G /dev/shm) or reduce queue/message size.

AttributeError: module 'faster_fifo' has no attribute 'Queue'

Importing the module instead of the class: `import faster_fifo` then using `faster_fifo.Queue` is correct, but some users mistakenly do `from faster_fifo import faster_fifo_queue`.

fix

Use `from faster_fifo import Queue` or `from faster_fifo import Queue as FQueue`.

Upgrade

Version history

1.5.2latest on PyPI · released Dec 28, 2024

Audit

Dependencies

No dependency data recorded yet.

Agent activity

31 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/alex-petrenko/faster-fifo ↗

packagehttps://pypi.org/project/faster-fifo/ ↗