Dynamic Worker Platform

A system for deploying and executing JavaScript workers at runtime using workerd's WorkerLoader API.

What it does

This platform lets you upload JavaScript code via HTTP API and execute it immediately. Each worker runs in its own isolated environment and can have custom environment variables and resource limits.

Basic usage

# Start the server
npm install workerd
npx workerd serve workerd.capnp --experimental

# Deploy a worker
curl -X POST http://localhost:8080/deploy/my-worker \
  -H "Content-Type: application/json" \
  -d '{
    "compatibilityDate": "2025-01-01",
    "mainModule": "index.js",
    "modules": {
      "index.js": {
        "js": "export default { async fetch(request) { return new Response(\"Hello world\"); } }"
      }
    }
  }'

# Execute the worker
curl http://localhost:8080/worker/my-worker/

Features

• Dynamic deployment: Upload code via REST API, runs immediately
• Worker isolation: Each worker has its own environment and bindings
• Persistent storage: Workers and metrics stored in KV, survive restarts
• Resource limits: Configurable timeouts, code size limits, module limits
• Monitoring: Tracks execution time, success rates, request counts
• Multiple modules: Workers can import from multiple JavaScript files

API endpoints

• POST /deploy/{name} - Deploy a new worker
• GET /worker/{name}/* - Execute a worker (passes remaining path to worker)
• GET /list - List all deployed workers with metrics
• GET /metrics - Platform-wide metrics
• GET /metrics/{name} - Specific worker metrics
• DELETE /delete/{name} - Delete a worker

Worker format

Workers are deployed as JSON with this structure:

{
  "compatibilityDate": "2025-01-01",
  "mainModule": "index.js",
  "modules": {
    "index.js": {
      "js": "export default { async fetch(request, env, ctx) { return new Response('hello'); } }"
    }
  },
  "env": {
    "API_KEY": "secret"
  },
  "limits": {
    "timeout": 5000,
    "maxCodeSize": 1048576,
    "maxModules": 50
  }
}

Architecture

The main worker handles HTTP requests and routes them:

• Deployment requests store worker code in KV storage
• Execution requests load worker code from KV and create instances using WorkerLoader
• Metrics are tracked and stored for each execution

Built with:

• workerd (Cloudflare's Workers runtime)
• WorkerLoader API (for dynamic worker creation)
• KV storage (file-based, for persistence)

Running

Requirements: Node.js 18+

git clone [repo]
cd dynamic-worker-platform
npm install workerd
npx workerd serve workerd.capnp --experimental

The server runs on http://localhost:8080 by default.

Files

• main-worker.js - Main orchestrator that handles deployment and routing
• workerd.capnp - workerd configuration file
• example-workers/ - Example worker definitions
• kv-data/ - Local storage for workers and metrics