Registry / devops / scraper.js-queue-bull

scraper.js-queue-bull

library5.0.5jsnpmunverified

A queue plugin for Scraper.js (v5.0.5) that uses Bull as the backend, backed by Redis. It provides a simple interface to add, process, count, and close queue items for web scraping workflows. The package is in maintenance mode with infrequent updates; the last release was in 2017. Environment variables configure Redis (REDIS_PORT, REDIS_HOST). Note: the package is not well-documented or tested, and has breaking changes in major versions (e.g., v2.0.0 removed options, v3.0.0 added options to process()). Use with caution.

devops testing

npm install scraper.js-queue-bull

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · SCRAPER.JS-QUEUE-B

scraper.js-queue-bull

devopsjavascriptv5.0.5

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

default

✓ import Queue from 'scraperjs-queue-bull'

✗ const Queue = require('scraperjs-queue-bull')

Uses CommonJS by default, but ESM may work in TS bundlers. Check Node version.

Queue

✓ import { Queue } from 'scraperjs-queue-bull'

Named export 'Queue' matches the default export.

Shows instantiation from environment variables, adding items, and processing them sequentially.

import Queue from 'scraperjs-queue-bull'; // Create a queue instance const queue = new Queue({ name: 'scraper', redis: { port: process.env.REDIS_PORT || 6379, host: process.env.REDIS_HOST || 'localhost' } }); // Process items (sequential) queue.process(async (item) => { console.log('Processing item:', item); // Your scraping logic here }); // Add items queue.add({ url: 'https://example.com' }); queue.add({ url: 'https://example2.com' }); // Optionally wait and close setTimeout(async () => { const count = await queue.count(); console.log('Remaining:', count); queue.close(); }, 5000);

Debug

Known issues

breakingv2.0.0 removed options parameter from add() and process()

fix

If upgrading from v1, remove all options from add() and process(). Options are now extracted from queueItem.

affects: >=2.0.0

breakingv3.0.0 added options to process() as a second parameter

fix

When calling process(handler, options), if you pass options, they are now the second argument.

affects: >=3.0.0

gotchaUses environment variables (PRIORITY_QUEUE, REDIS_PORT, REDIS_HOST) to configure Redis, not constructor options

fix

Set env vars before runtime, or modify the package source code. Options are not passed to constructor.

affects: >=0.0.0

deprecatedPackage is unmaintained with no updates since 2017

fix

Consider migrating to 'bull' directly or using 'scraperjs-queue-bull' as-is.

affects: >=0.0.0

Errors

Common errors & fixes

Error: Redis connection to 127.0.0.1:6379 failed

Redis is not running or config mismatch

fix

Start Redis server or set REDIS_PORT and REDIS_HOST environment variables.

TypeError: queue.process is not a function

Incorrect import or module version mismatch

fix

Use 'import Queue from "scraperjs-queue-bull"' and instantiate with 'new Queue()'.

Error: Cannot find module 'bull'

Missing peer dependency

fix

Run 'npm install bull'.

Upgrade

Version history

5.0.5latest on npm

Audit

Dependencies

bullrequiredCore queueing library backed by Redis

redisoptionalRedis client for Bull

Agent activity

12 hits · last 30 days

node

Resources

githubhttps://github.com/leahcimic/scraperjs-queue-bull ↗

packagescraper.js-queue-bull ↗

homepagehttps://github.com/leahcimic/scraperjs-queue-bull#readme ↗