-
Notifications
You must be signed in to change notification settings - Fork 0
feat(stem): close BullMQ parity gaps across events, control, and stores #17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
ef9bea8
docs(site): add canonical stem vs bullmq comparison
kingwill101 6321d54
feat(stem): close bullmq parity gaps across events, control, and stores
kingwill101 ecb4f22
fix(brokers): allow broadcast-only subscriptions for queue events
kingwill101 b7684eb
fix(stem): address retry timing, batch idempotency, and flaky tests
kingwill101 5559a24
fix(canvas): preserve stored batch task id order
kingwill101 937f980
feat(clock): add shared stem clock abstraction
kingwill101 ab50691
refactor(stem): use shared clock across runtime modules
kingwill101 599c438
refactor(adapters): route backend time reads through stem clock
kingwill101 8e8dc15
refactor(cli,dashboard): align timestamps with stem clock
kingwill101 90c9257
docs: address review nits across parity and event docs
kingwill101 da08861
feat(core): tighten event semantics and UTC clock behavior
kingwill101 af15c63
test(adapters): harden revoke and queue-event contract coverage
kingwill101 4f4caf3
fix(worker): improve control semantics and timing robustness
kingwill101 e2f75b0
feat(canvas): tighten batch inspection and submission idempotency
kingwill101 477ed1a
fix(redis): preserve broadcast consumer groups and cleanup disposal
kingwill101 0f44afd
fix(signals): freeze control command timestamps
kingwill101 3de91d1
fix(redis): remove unnecessary non-null assertions in consume loop
kingwill101 39078ca
fix(redis): retain broadcast fan-out for queue subscriptions
kingwill101 603e805
fix(redis): repair broadcast NOGROUP recovery path
kingwill101 1ebe7e9
fix(redis): guard delivery emission after stream cancellation
kingwill101 23139ab
fix(redis): harden claim timer delivery on closed streams
kingwill101 99c3dc3
test(adapters): avoid auto-cancel race in broadcast fanout contract
kingwill101 1e0d73a
test(adapters): pre-arm broadcast listeners before fanout publish
kingwill101 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| --- | ||
| title: Stem vs BullMQ | ||
| sidebar_label: Stem vs BullMQ | ||
| sidebar_position: 1 | ||
| slug: /comparisons/stem-vs-bullmq | ||
| --- | ||
|
|
||
| This page is the canonical Stem comparison matrix for BullMQ-style features. | ||
| It focuses on capability parity, not API-level compatibility. | ||
|
|
||
| **As of:** February 24, 2026 | ||
|
|
||
| ## Status semantics | ||
|
|
||
| | Status | Meaning | | ||
| | --- | --- | | ||
| | `✓` | Functionally equivalent built-in capability exists in Stem. | | ||
| | `~` | Partial or non-isomorphic capability exists, but semantics differ from BullMQ/BullMQ-Pro. | | ||
| | `✗` | No built-in capability in Stem today. | | ||
|
|
||
| ## Feature matrix | ||
|
|
||
| | BullMQ row | Stem | Rationale (with evidence) | | ||
| | --- | --- | --- | | ||
| | Backend | `✓` | Stem supports multiple backends/adapters (Redis, Postgres, SQLite, in-memory). See [Broker Overview](../brokers/overview.md) and [Developer Environment](../getting-started/developer-environment.md). | | ||
| | Observables | `✓` | Stem has built-in metrics, tracing, and lifecycle signals. See [Observability](../core-concepts/observability.md) and [Signals](../core-concepts/signals.md). | | ||
| | Group Rate Limit | `✓` | Stem supports group-scoped rate limiting via `TaskOptions.groupRateLimit`, `groupRateKey`, and `groupRateKeyHeader`. See [Rate Limiting](../core-concepts/rate-limiting.md). | | ||
| | Group Support | `✓` | Stem provides `Canvas.group` and `Canvas.chord` primitives. See [Canvas Patterns](../core-concepts/canvas.md). | | ||
| | Batches Support | `✓` | Stem exposes first-class batch APIs (`submitBatch`, `inspectBatch`) with durable batch lifecycle status. See [Canvas Patterns](../core-concepts/canvas.md). | | ||
| | Parent/Child Dependencies | `✓` | Stem supports dependency composition through chains, groups/chords, and workflow steps. See [Canvas Patterns](../core-concepts/canvas.md) and [Workflows](../core-concepts/workflows.md). | | ||
| | Deduplication (Debouncing) | `~` | `TaskOptions.unique` prevents duplicate enqueue claims, but semantics are lock/TTL-based rather than BullMQ-native dedupe APIs. See [Uniqueness](../core-concepts/uniqueness.md). | | ||
| | Deduplication (Throttling) | `~` | `uniqueFor` and lock TTL windows approximate throttling behavior, but are not a direct BullMQ equivalent. See [Uniqueness](../core-concepts/uniqueness.md). | | ||
| | Priorities | `✓` | Stem supports task priority and queue priority ranges. See [Tasks](../core-concepts/tasks.md) and [Routing](../core-concepts/routing.md). | | ||
| | Concurrency | `✓` | Workers support configurable concurrency and isolate pools. See [Workers](../workers/index.md) and [Worker Control](../workers/worker-control.md). | | ||
| | Delayed jobs | `✓` | Delayed execution is supported via enqueue options and broker scheduling. See [Quick Start](../getting-started/quick-start.md) and [Broker Overview](../brokers/overview.md). | | ||
| | Global events | `✓` | Stem exposes global lifecycle events through `StemSignals`, plus queue-scoped custom events through `QueueEvents`. See [Signals](../core-concepts/signals.md) and [Queue Events](../core-concepts/queue-events.md). | | ||
| | Rate Limiter | `✓` | Stem supports per-task rate limits with pluggable limiter backends. See [Rate Limiting](../core-concepts/rate-limiting.md). | | ||
| | Pause/Resume | `✓` | Stem provides queue pause/resume commands (`stem worker pause`, `stem worker resume`) and persistent pause state when a revoke store is configured. See [Worker Control](../workers/worker-control.md). | | ||
| | Sandboxed worker | `~` | Stem supports isolate-based execution boundaries, but this is not equivalent to BullMQ's Node child-process sandbox model. See [Worker Control](../workers/worker-control.md). | | ||
| | Repeatable jobs | `✓` | Stem Beat supports interval, cron, solar, and clocked schedules. See [Scheduler](../scheduler/index.md) and [Beat Scheduler Guide](../scheduler/beat-guide.md). | | ||
| | Atomic ops | `~` | Stem includes atomic behavior in specific stores/flows, but end-to-end transactional guarantees (for all enqueue/ack/result paths) are not universally built-in. See [Tasks idempotency guidance](../core-concepts/tasks.md#idempotency-checklist) and [Best Practices](../getting-started/best-practices.md). | | ||
| | Persistence | `✓` | Stem persists task/workflow/schedule state through pluggable backends/stores. See [Persistence & Stores](../core-concepts/persistence.md). | | ||
| | UI | `~` | Stem includes an experimental dashboard, not a fully mature operator UI parity target yet. See [Dashboard](../core-concepts/dashboard.md). | | ||
| | Optimized for | `~` | Stem is optimized for jobs/messages plus durable workflow orchestration, not only queue semantics. See [Core Concepts](../core-concepts/index.md) and [Workflows](../core-concepts/workflows.md). | | ||
|
|
||
| ## Update policy | ||
|
|
||
| When this matrix changes: | ||
|
|
||
| 1. Update the **As of** date. | ||
| 2. Keep row names aligned with BullMQ terminology. | ||
| 3. Update rationale links so every status remains auditable. | ||
|
|
||
| ## BullMQ events parity notes | ||
|
|
||
| Stem supports the two common BullMQ event-listening styles: | ||
|
|
||
| | BullMQ concept | Stem equivalent | | ||
| | --- | --- | | ||
| | `QueueEvents` listeners | `QueueEvents` + `QueueEventsProducer` (queue-scoped custom events) | | ||
| | Custom queue events | `producer.emit(queue, eventName, payload, headers, meta)` | | ||
| | Worker-specific event listeners | `StemSignals` convenience APIs with `workerId` filters (`onWorkerReady`, `onWorkerInit`, `onTaskFailure`, `onControlCommandCompleted`, etc.) | | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| --- | ||
| title: Queue Events | ||
| sidebar_label: Queue Events | ||
| sidebar_position: 9 | ||
| slug: /core-concepts/queue-events | ||
| --- | ||
|
|
||
| Stem supports queue-scoped custom events similar to BullMQ `QueueEvents` and | ||
| "custom events" patterns. | ||
|
|
||
| Use this when you need lightweight event streams for domain notifications | ||
| (`order.created`, `invoice.settled`) without creating task handlers. | ||
|
|
||
| ## API Surface | ||
|
|
||
| - `QueueEventsProducer.emit(queue, eventName, payload, headers, meta)` | ||
| - `QueueEvents.start()` / `QueueEvents.close()` | ||
| - `QueueEvents.events` stream (all events for that queue) | ||
| - `QueueEvents.on(eventName)` stream (filtered by name) | ||
|
|
||
| All events are delivered as `QueueCustomEvent`, which implements `StemEvent`. | ||
|
|
||
| ## Producer + Listener | ||
|
|
||
| ```dart title="lib/queue_events.dart" file=<rootDir>/../packages/stem/example/docs_snippets/lib/queue_events.dart#queue-events-producer-listener | ||
|
|
||
| ``` | ||
|
|
||
| ## Fan-out to Multiple Listeners | ||
|
|
||
| Multiple listeners on the same queue receive each emitted event. | ||
|
|
||
| ```dart title="lib/queue_events.dart" file=<rootDir>/../packages/stem/example/docs_snippets/lib/queue_events.dart#queue-events-fanout | ||
|
|
||
| ``` | ||
|
|
||
| ## Semantics | ||
|
|
||
| - Events are queue-scoped: listeners receive only events for their configured | ||
| queue. | ||
| - `on(eventName)` matches exact event names. | ||
| - `headers` and `meta` round-trip to listeners. | ||
| - Event names and queue names must be non-empty. | ||
| - Delivery follows the underlying broker's broadcast behavior for active | ||
| listeners (no historical replay API is built into `QueueEvents`). | ||
|
|
||
| ## When to Use Queue Events vs Signals | ||
|
|
||
| - Use [Signals](./signals.md) for runtime lifecycle hooks (task/worker/scheduler/control). | ||
| - Use Queue Events for application-domain events you publish and consume. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.