Product Drop Backend System

A high-reliability product drop system with durable workflows, real-time updates, and multi-layered bot mitigation. Includes a modern Next.js + Tailwind frontend.

Features

Durable Execution: Restate virtual objects + timers for reliable drop lifecycle and recovery
Provable Lottery: Commit/reveal seed + Merkle participant snapshot for independent verification
Scalable Winner Selection: Weighted selection via Fenwick tree (no O(totalTickets) expansion)
Backup Winners: Automatic waitlist promotion if winners don’t purchase in time
Geo-fencing: Optional location restrictions + bonus multipliers
Bot Resistance: Proof-of-work + behavioral signals + optional FingerprintJS integration
Real-time UX: SSE for live drop lists, state updates, and queue status
Purchase Protection: Single-use purchase tokens, expiration, and double-purchase prevention

Tech Stack

Layer	Technology
Runtime	Node.js + TypeScript (ESM)
Backend Framework	Hono (lightweight, fast)
Durable Execution	Restate SDK
Messaging	NATS v3 modular client
Frontend	Next.js 15 + Tailwind CSS
Load Testing	k6, Playwright
Package Manager	pnpm

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Next.js Frontend                          :3005            │
│  - Drop listing homepage + drop detail pages                │
│  - Queue + PoW client + SSE hooks                            │
│  - Proxies /api/* and /events/* to backend services         │
└─────────────────┬───────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────┐
│  API Server (Hono)                         :3003            │
│  - Bot validation middleware                                │
│  - Routes: /api/drop/*, /api/drops/*, /api/pow/challenge    │
└─────────────────┬───────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────┐
│  Restate Runtime (Docker)                                   │
│  - Ingress API: :8080                                       │
│  - Admin API:   :9070                                       │
│  - Drop & Participant virtual objects                       │
│  - Durable timers for phase transitions                     │
└─────────────────┬───────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────┐
│  SSE Server (Hono)                         :3004            │
│  - /events/:dropId/:userId endpoint                         │
│  - /events/drops stream for live active drop listings        │
└─────────────────────────────────────────────────────────────┘

Design Deep Dive

1. Durable Execution with Restate

The core state machine uses Restate virtual objects:

Drop: Manages the lifecycle (registration → lottery → purchase → completed)
Participant: Tracks individual user state per drop
UserRollover: Global rollover balance across drops

This provides automatic crash recovery, exactly-once semantics, and durable timers for scheduled phase transitions. When you call:

ctx.objectSendClient(dropObject, dropId, { delay: delayMs }).runLottery({});

That timer survives server restarts — Restate persists it and fires it at the right time.

2. Real-time Event System (NATS + SSE)

Restate handlers → publish to NATS → SSE server subscribes → pushes to browser

NATS v3 modular client for pub/sub messaging between services
Server-Sent Events for browser push (simpler than WebSockets for one-way updates)
Clock synchronization with exponential moving average to handle client/server drift:

// Smooth out network jitter for accurate countdowns
const alpha = 0.3;
return Math.round(prevOffset * (1 - alpha) + newOffset * alpha);

3. Deterministic Weighted Lottery

// Memory-efficient weighted selection without replacement (no ticket expansion)
// - Build Fenwick tree over per-user weights
// - Repeatedly sample a weight-indexed random point using a deterministic seed
// - Remove winner weight and continue
const winners = selectWinnersWithMultipliers(entries, seed, winnerCount);

Provable seed via commit/reveal, combined with the participant Merkle root
Deterministic RNG ensures reproducibility for auditors
Weighted selection without replacement where more tickets = higher win chance, but any user wins at most once
Scales to large drops because memory is (O(N)) participants, not (O(\text{totalTickets}))

4. Multi-Layer Bot Mitigation

Layer	Weight	Technique
Fingerprinting	40%	Browser fingerprint confidence score
Timing Analysis	30%	Human-like interaction patterns (1-5s optimal)
Proof-of-Work	30%	SHA-256 challenge solving

The PoW challenge requires finding a nonce where SHA256(challenge + nonce) starts with N zeros — forces compute cost on the client before registration.

5. Rollover System

Clever engagement mechanic that reduces the sting of losing:

Paid entries that lose → convert to rollover credits
Rollover credits auto-apply to next drop registration (before free entry)
Stacks up to a maximum to prevent infinite accumulation

// Entry breakdown: rollover first, then free, then paid
const rolloverToUse = Math.min(rolloverBalance, desiredTickets);
const remainingAfterRollover = desiredTickets - rolloverToUse;
const freeEntry = remainingAfterRollover > 0 ? 1 : 0;
const paidEntries = Math.max(0, remainingAfterRollover - freeEntry);

6. Quadratic Ticket Pricing

Additional tickets cost progressively more, preventing whales from dominating:

// Cost = 1² + 2² + ... + (n-1)² = n(n-1)(2n-1)/6
return ((n * (n + 1) * (2 * n + 1)) / 6) * priceUnit;

Tickets	Total Cost
1	Free
2	$1
3	$5
5	$30
10	$285

7. State Machine UX

The frontend gracefully handles every edge case with empathetic messaging:

Winner who didn't purchase in time → "Time Expired" with encouragement
Missed registration entirely → context-aware messaging based on current phase
Server clock sync → smooth countdowns that don't jump around

Quick Start (with Makefile)

The easiest way to run the project is using the Makefile:

# Install everything
make install

# Start Restate + backend + initialize drop
make dev

# In another terminal, start the frontend
make frontend

# Open http://localhost:3005

Other useful commands

make status       # View drop state
make lottery      # Trigger lottery manually
make reset        # Quick reset (restart + re-init)
make reset-full   # Full reset (clear all state)
make logs         # View Restate logs
make help         # Show all available commands

Manual Setup

Prerequisites

Node.js 20+ (or pnpm)
Docker & Docker Compose

1. Install Dependencies

# Install backend dependencies
pnpm install

# Install frontend dependencies
cd web && pnpm install && cd ..

2. Start Restate Runtime

# Start Restate Docker container
docker-compose up -d

# Verify it's running
docker-compose ps

3. Start Backend Services

# Start API server, SSE server, and Restate worker
pnpm dev

You should see:

╔══════════════════════════════════════════════════════════╗
║           Product Drop Backend Started                   ║
╠══════════════════════════════════════════════════════════╣
║  API Server:      http://localhost:3003                  ║
║  SSE Server:      http://localhost:3004                  ║
║  Restate Worker:  http://localhost:9080                  ║
╚══════════════════════════════════════════════════════════╝

4. Register Worker with Restate

# Register the Restate worker (port 9080 is the SDK default)
curl localhost:9070/deployments \
  -H 'content-type: application/json' \
  -d '{"uri":"http://host.docker.internal:9080"}'

5. Initialize a Demo Drop

# Create a drop with 5-minute registration window
NOW=$(date +%s)000
END=$((NOW + 300000))
curl localhost:8080/Drop/demo-drop-1/initialize \
  -H 'content-type: application/json' \
  -d "{\"dropId\":\"demo-drop-1\",\"inventory\":10,\"registrationStart\":$((NOW - 1000)),\"registrationEnd\":$END,\"purchaseWindow\":300}"

6. Start Frontend (Optional)

cd web && pnpm dev

Open http://localhost:3005

Resetting the Project

Full Reset (Clear All State)

# 1. Stop everything
docker-compose down

# 2. Remove Restate data volume (clears all state)
docker volume rm waitingroom_restate-data 2>/dev/null || true

# 3. Restart Restate
docker-compose up -d

# 4. Re-register worker (after starting backend with pnpm dev)
curl localhost:9070/deployments \
  -H 'content-type: application/json' \
  -d '{"uri":"http://host.docker.internal:9080"}'

# 5. Initialize a new drop
NOW=$(date +%s)000
END=$((NOW + 300000))
curl localhost:8080/Drop/demo-drop-1/initialize \
  -H 'content-type: application/json' \
  -d "{\"dropId\":\"demo-drop-1\",\"inventory\":10,\"registrationStart\":$((NOW - 1000)),\"registrationEnd\":$END,\"purchaseWindow\":300}"

Quick Reset (Keep Restate Running)

# Restart just the Restate container (clears in-memory state)
docker-compose restart

# Re-register worker
curl localhost:9070/deployments \
  -H 'content-type: application/json' \
  -d '{"uri":"http://host.docker.internal:9080"}'

# Re-initialize drop
NOW=$(date +%s)000 && END=$((NOW + 300000)) && \
curl localhost:8080/Drop/demo-drop-1/initialize \
  -H 'content-type: application/json' \
  -d "{\"dropId\":\"demo-drop-1\",\"inventory\":10,\"registrationStart\":$((NOW - 1000)),\"registrationEnd\":$END,\"purchaseWindow\":300}"

Restate Commands Reference

Deployment Management

# List all deployments
curl localhost:9070/deployments

# Register a new deployment
curl localhost:9070/deployments \
  -H 'content-type: application/json' \
  -d '{"uri":"http://host.docker.internal:9080"}'

# Force re-register (if services changed)
curl localhost:9070/deployments \
  -H 'content-type: application/json' \
  -d '{"uri":"http://host.docker.internal:9080","force":true}'

Invoking Services Directly

# Get drop state
curl localhost:8080/Drop/demo-drop-1/getState \
  -H 'content-type: application/json' \
  -d '{}'

# Trigger lottery manually
curl localhost:8080/Drop/demo-drop-1/runLottery \
  -H 'content-type: application/json' \
  -d '{}'

# Get participant state
curl localhost:8080/Participant/demo-drop-1:user-123/getState \
  -H 'content-type: application/json' \
  -d '{}'

Viewing Invocations

# List all invocations (admin API)
curl localhost:9070/invocations

# Get specific invocation
curl localhost:9070/invocations/{invocation_id}

Port Reference

Service	Port	Purpose
Next.js Frontend	3005	User interface
API Server	3003	REST API endpoints
SSE Server	3004	Real-time event streaming
Restate Ingress	8080	Service invocation API
Restate Admin	9070	Deployment & management API
Restate Worker	9080	SDK handler endpoint

API Endpoints

Drop Management

Method	Path	Description
POST	`/api/drop/:id/register`	Register for a drop (requires queue token + bot checks)
GET	`/api/drop/:id/status`	Get current drop status
POST	`/api/drop/:id/lottery`	Trigger lottery manually
POST	`/api/drop/:id/purchase/start`	Start purchase (get token)
POST	`/api/drop/:id/purchase`	Complete purchase
GET	`/api/drop/:id/inclusion-proof/:userId`	Fetch Merkle inclusion proof for independent verification

Drops Listing

Method	Path	Description
GET	`/api/drops/active`	List active drops (used by homepage/SSE snapshot)

Proof of Work

Method	Path	Description
GET	`/api/pow/challenge`	Get PoW challenge

SSE (Direct Connection)

Method	Path	Description
GET	`http://localhost:3004/events/:dropId/:userId`	SSE connection for real-time updates
GET	`http://localhost:3004/events/drops`	SSE stream of active drops for the homepage

In production on Fly.io, the frontend typically connects via same-origin /events/* (proxied by Next.js route handlers).

Drop Lifecycle

REGISTRATION → LOTTERY → PURCHASE → COMPLETED

Registration Phase: Users register with bot validation
Lottery Phase: Fair random selection of winners
Purchase Phase: Winners can complete purchases within time window
Completed: All inventory sold or phase expires

Bot Mitigation

The system uses a multi-layered approach with weighted scoring:

Layer	Weight	Description
FingerprintJS	40%	Browser fingerprinting confidence
Timing Analysis	30%	Human-like interaction patterns (1-5s optimal)
Proof-of-Work	30%	SHA-256 challenge solving

Trust Score Threshold: 50/100 minimum to pass validation

Environment Variables

For local development, copy the templates:

cp env.example .env
cp web/env.example web/.env.local

RESTATE_INGRESS_URL=http://localhost:8080
FINGERPRINT_API_KEY=your_fpjs_secret_key  # Optional for dev
POW_DIFFICULTY=2                          # Leading zero bytes required
API_PORT=3003
SSE_PORT=3004

Project Structure

.
├── src/
│   ├── api/              # Hono API server
│   │   ├── routes/       # API route handlers
│   │   └── middleware/   # Bot guard, rate limiting
│   ├── sse/              # SSE server
│   ├── restate/          # Restate virtual objects
│   │   ├── drop.ts       # Drop state machine
│   │   └── participant.ts # Participant state
│   ├── lib/              # Shared utilities
│   └── scripts/          # CLI scripts
├── web/                  # Next.js frontend
│   ├── app/              # App router pages
│   ├── components/       # React components
│   ├── hooks/            # Custom hooks (SSE, countdown)
│   └── lib/              # API client, types
└── docker-compose.yml    # Restate runtime

Troubleshooting

"Connection refused" on SSE

The backend servers aren't running. Start them with pnpm dev.

"Deployment not found" or services not registered

Re-register the worker:

curl localhost:9070/deployments \
  -H 'content-type: application/json' \
  -d '{"uri":"http://host.docker.internal:9080"}'

Stuck invocations / infinite retries

Restart Restate to clear stuck state:

docker-compose restart

CORS errors in browser

Make sure you're accessing the frontend at http://localhost:3005 and the SSE server is running on port 3004.

Hydration errors in Next.js

This was fixed - don't use Date.now() at module level. Use useEffect for client-only calculations.

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 11 Commits
.kiro/specs/ml-bot-detection		.kiro/specs/ml-bot-detection
fly		fly
shared		shared
src		src
tests		tests
web		web
.cursorrules		.cursorrules
.dockerignore		.dockerignore
.gitignore		.gitignore
Makefile		Makefile
README.md		README.md
docker-compose.yml		docker-compose.yml
env.example		env.example
package-lock.json		package-lock.json
package.json		package.json
playwright.config.ts		playwright.config.ts
pnpm-lock.yaml		pnpm-lock.yaml
tsconfig.json		tsconfig.json
tsup.config.ts		tsup.config.ts

10thfloor/dropslop

Folders and files

Latest commit

History

Repository files navigation

Product Drop Backend System

Features

Tech Stack

Architecture

Design Deep Dive

1. Durable Execution with Restate

2. Real-time Event System (NATS + SSE)

3. Deterministic Weighted Lottery

4. Multi-Layer Bot Mitigation

5. Rollover System

6. Quadratic Ticket Pricing

7. State Machine UX

Quick Start (with Makefile)

Other useful commands

Manual Setup

Prerequisites

1. Install Dependencies

2. Start Restate Runtime

3. Start Backend Services

4. Register Worker with Restate

5. Initialize a Demo Drop

6. Start Frontend (Optional)

Resetting the Project

Full Reset (Clear All State)

Quick Reset (Keep Restate Running)

Restate Commands Reference

Deployment Management

Invoking Services Directly

Viewing Invocations

Port Reference

API Endpoints

Drop Management

Drops Listing

Proof of Work

SSE (Direct Connection)

Drop Lifecycle

Bot Mitigation

Environment Variables

Project Structure

Troubleshooting

"Connection refused" on SSE

"Deployment not found" or services not registered

Stuck invocations / infinite retries

CORS errors in browser

Hydration errors in Next.js

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages