Skip to content

Comments

Fix MongoDB webhook workflow e2e test#15

Draft
dustintownsend wants to merge 1 commit intomainfrom
claude/fix-mongodb-e2e-test-01MKcsmv3jztEdJeSYeysQQa
Draft

Fix MongoDB webhook workflow e2e test#15
dustintownsend wants to merge 1 commit intomainfrom
claude/fix-mongodb-e2e-test-01MKcsmv3jztEdJeSYeysQQa

Conversation

@dustintownsend
Copy link
Contributor

  • Make hook creation atomic by using the unique index on token instead of a check-then-insert pattern, which had a TOCTOU race condition
  • Properly catch MongoDB duplicate key errors (code 11000) and convert to 409
  • Add debug logging to hooks create, get, and getByToken operations
  • Add comprehensive webhook flow tests to the testing package:
    • Test creating and retrieving hooks with random tokens (like e2e test)
    • Test concurrent hook creation and lookup
    • Test that getByToken returns 404 for non-existent tokens
    • Test immediate visibility of hooks after creation

These changes help diagnose and potentially fix the MongoDB e2e test failure where webhookWorkflow gets 404 instead of 202.

- Make hook creation atomic by using the unique index on token instead of
  a check-then-insert pattern, which had a TOCTOU race condition
- Properly catch MongoDB duplicate key errors (code 11000) and convert to 409
- Add debug logging to hooks create, get, and getByToken operations
- Add comprehensive webhook flow tests to the testing package:
  - Test creating and retrieving hooks with random tokens (like e2e test)
  - Test concurrent hook creation and lookup
  - Test that getByToken returns 404 for non-existent tokens
  - Test immediate visibility of hooks after creation

These changes help diagnose and potentially fix the MongoDB e2e test failure
where webhookWorkflow gets 404 instead of 202.
@github-actions
Copy link
Contributor

github-actions bot commented Dec 6, 2025

Benchmark Comparison

Cross-comparison of workflow performance across World implementations.

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

World Workflow Time Wall Time Overhead vs Fastest
🥇 💾 Starter (in-memory) 0.048s (+4.4%) 1.009s (~) 0.961s 1.00x
🔴 Redis 0.053s (+28.4% 🔺) 1.009s (~) 0.956s 1.10x
🍃 MongoDB 0.081s (-24.1% 🟢) 1.009s (~) 0.928s 1.69x
💻 Local 0.086s (-5.2% 🟢) 1.009s (~) 0.923s 1.80x
🗄️ Turso 0.096s (+10.9% 🔺) 1.010s (-0.6%) 0.915s 2.00x

workflow with 1 step

World Workflow Time Wall Time Overhead vs Fastest
🥇 💾 Starter (in-memory) 0.138s (+1.3%) 1.007s (~) 0.870s 1.00x
🔴 Redis 0.154s (+29.7% 🔺) 1.007s (~) 0.853s 1.12x
💻 Local 0.259s (-6.0% 🟢) 1.008s (~) 0.749s 1.88x
🍃 MongoDB 0.289s (-3.9%) 1.007s (~) 0.718s 2.10x
🗄️ Turso 0.333s (+8.0% 🔺) 1.026s (+1.7%) 0.693s 2.42x

workflow with 10 sequential steps

World Workflow Time Wall Time Overhead vs Fastest
🥇 💾 Starter (in-memory) 1.016s (+1.3%) 1.607s (+14.2% 🔺) 0.590s 1.00x
🔴 Redis 1.120s (+32.7% 🔺) 2.007s (+99.5% 🔺) 0.887s 1.10x
💻 Local 1.903s (-3.8%) 2.007s (-9.2% 🟢) 0.104s 1.87x
🍃 MongoDB 2.145s (-0.8%) 3.011s (~) 0.866s 2.11x
🗄️ Turso 2.152s (~) 3.011s (~) 0.859s 2.12x

workflow with 10 parallel steps

World Workflow Time Wall Time Overhead vs Fastest
🥇 💾 Starter (in-memory) 0.334s (+1.5%) 1.007s (~) 0.673s 1.00x
🔴 Redis 0.405s (+44.2% 🔺) 1.006s (~) 0.601s 1.21x
💻 Local 0.640s (-5.9% 🟢) 1.007s (~) 0.367s 1.92x
🍃 MongoDB 1.577s (-3.3%) 2.007s (~) 0.429s 4.73x
🗄️ Turso 1.582s (~) 2.010s (~) 0.428s 4.74x

Stream Benchmarks

Stream benchmarks include Time to First Byte (TTFB) metrics.

workflow with stream

World Workflow Time TTFB Wall Time Overhead vs Fastest
🥇 💾 Starter (in-memory) 0.230s (+2.1%) 1.006s (~) 1.008s (~) 0.778s 1.00x
🔴 Redis 0.253s (+31.2% 🔺) 1.003s (~) 1.007s (~) 0.754s 1.10x
🗄️ Turso 0.491s (+1.2%) 0.974s (~) 1.009s (~) 0.518s 2.13x
🍃 MongoDB 0.529s (+2.4%) 0.929s (-2.0%) 1.008s (~) 0.478s 2.30x
💻 Local ⚠️ missing - - - -

Summary: Average Performance by World

World Avg Workflow Time Benchmarks
💾 Starter (in-memory) 0.353s 5
🔴 Redis 0.397s 5
💻 Local 0.722s 4
🍃 MongoDB 0.924s 5
🗄️ Turso 0.931s 5
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • vs Fastest: How much slower compared to the fastest World for this benchmark

Worlds:

  • 💻 Local: Built-in workflow world
  • 🐘 Postgres: PostgreSQL database world
  • ▲ Vercel: Vercel production world
  • 💾 Starter: In-memory reference implementation
  • 🍃 MongoDB: MongoDB database world
  • 🔴 Redis: Redis/BullMQ backend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants