Vana Runtime Task - Demo

A production-ready template for building tasks that run in Vana Runtime with TEE-safe operation submission.

Status: Ready for customization
Runtime Version: 1.0.0+
Difficulty: Beginner-friendly

---

🎯 What This Template Provides

• ✅ Complete FastAPI server with all required endpoints
• ✅ Operations manifest for runtime validation
• ✅ Docker deployment ready
• ✅ Structured TODOs for easy customization
• ✅ LLM-friendly code with clear guidance
• ✅ Security guidelines built-in
• ✅ Local testing with docker-compose

---

📋 Quick Start

1. Test the Template (As-Is)

# Build and run
docker-compose up --build

# In another terminal, test endpoints
curl http://localhost:8000/health
curl http://localhost:8000/manifest
curl -X POST http://localhost:8000/process \
  -H "Content-Type: application/json" \
  -d '{"batch_id": "test-001"}'

2. Customize for Your Use Case

Search for TODO-CUSTOMIZE comments in main.py and follow the inline guidance.

3. Deploy to Vana Runtime

# Build for production
docker build -t myorg/my-task:v1 .

# Push to registry
docker push myorg/my-task:v1

# Register with Vana Runtime (onchain)
# See "Deployment" section below

---

🏗️ Project Structure

vana-task-template/
├── main.py                 # FastAPI application (CUSTOMIZE THIS)
│                           # - Core task logic and endpoints
│                           # - Search for TODO-CUSTOMIZE to find customization points
│
├── operations.json         # Operations manifest (CUSTOMIZE THIS)
│                           # - Declares all operations your task supports
│                           # - Runtime validates all requests against this
│
├── requirements.txt        # Python dependencies (ADD YOURS)
│                           # - Add any libraries your task needs
│
├── .env.example           # Environment variables template
│                           # - Documents all available env vars
│                           # - Copy to .env for local development
│
├── Dockerfile             # Docker build configuration
│                           # - Production container definition
│
├── docker-compose.yml     # Local testing setup
│                           # - Simulates runtime environment
│
├── test-task.sh           # Quick test script
│                           # - Validates all endpoints
│
├── data/                  # Input data (READ-ONLY)
│                           # - Mounted by runtime with encrypted/decrypted data
│                           # - Your task reads from here
│                           # - Never write to this directory
│
├── work/                  # Working directory (READ-WRITE)
│                           # - Use for temporary/intermediate files
│                           # - Cleared between task runs
│                           # - Not persisted
│
├── output/                # Output artifacts (WRITE-ONLY)
│                           # - Write final results here
│                           # - Runtime collects files from this directory
│                           # - Available via /task/{id}/artifacts API
│
└── README.md              # This file

Data Flow:

Runtime → [data/] → Your Task → [work/] → Processing → [output/] → Runtime → Client
          (input)              (temporary)           (artifacts)

---

🔧 Customization Guide

Before You Start:

• 📖 Review .env.example for available environment variables
• 📖 Review "Environment Variables" section below for detailed guidance
• 📖 Review TASK_API_CONTRACT.md in vana-runtime for API requirements
• 🔍 Search for TODO-CUSTOMIZE in main.py for customization points

---

Step 1: Define Your Operations

Edit operations.json:

{
  "version": "1.0",
  "operations": [
    {
      "name": "your_operation_name",
      "path": "/your/path",
      "method": "POST",
      "description": "What this operation does",
      "is_async": false
    }
  ]
}

Guidelines:

• name: Used in Runtime API → /task/{id}/operation/{name}
• path: Internal endpoint in your task
• method: HTTP method (GET, POST, PUT, DELETE)
• is_async: Set true if operation returns operation_id

Step 2: Implement Operation Logic

Search for TODO-CUSTOMIZE in main.py. Key areas:

Synchronous Operation:

@app.post("/process")
async def process_batch(request: ProcessBatchRequest):
    # TODO-CUSTOMIZE: Your processing logic here
    
    # 1. Read data from DATA_PATH
    data = read_data_files(DATA_PATH)
    
    # 2. Process data
    results = your_processing_logic(data, request.filters)
    
    # 3. Write outputs to OUTPUT_PATH
    write_artifact(OUTPUT_PATH, "results.json", results)
    
    # 4. Return aggregate stats (NOT raw data!)
    return ProcessBatchResponse(
        result="processed",
        count=len(results),
        debug_info={
            "processing_time_ms": 1500,
            "files_processed": len(data)
        }
    )

Async Operation:

@app.post("/analyze")
async def analyze(request: AnalyzeRequest):
    # TODO-CUSTOMIZE: Start background processing
    
    # 1. Create operation
    operation_id = create_async_operation("analyze", params)
    
    # 2. Start background task
    asyncio.create_task(run_analysis(operation_id, request))
    
    # 3. Return operation_id (Runtime detects async)
    return AnalyzeResponse(
        operation_id=operation_id,
        status="processing"
    )

Step 3: Add Your Dependencies

Edit requirements.txt:

# Base requirements (required)
fastapi==0.104.1
uvicorn[standard]==0.24.0
pydantic==2.5.0

# Your custom dependencies
pandas==2.1.3
numpy==1.26.2
scikit-learn==1.3.2

Note: Pin versions for reproducible builds

Step 4: Implement Custom Logic

Data Processing:

• Read from /app/data (read-only)
• Use /app/work for temporary files
• Write outputs to /app/output

Example:

from pathlib import Path
import json

# Read data
data_dir = Path(DATA_PATH)
for data_file in data_dir.rglob("*.json"):
    with open(data_file) as f:
        data = json.load(f)
        # Process data...

# Write output
output_file = Path(OUTPUT_PATH) / "results.json"
with open(output_file, 'w') as f:
    json.dump(results, f)

---

🔧 Environment Variables

Runtime-Provided Variables

These are automatically set by Vana Runtime when your task starts:

Variable	Type	Description	Example
TASK_ID	Integer (string)	Unique task identifier from onchain registry	"123"
DATASET_ID	Integer (string)	Dataset being processed	"100"
RUN_ID	String	Unique execution identifier	"123-abc123de"
DLP_ID	Integer (string)	Data Liquidity Pool ID (optional)	"1"
DATA_PATH	Directory path	Read-only input data directory	/app/data
WORK_PATH	Directory path	Temporary working directory	/app/work
OUTPUT_PATH	Directory path	Output artifacts directory	/app/output

Usage Example:

import os
from pathlib import Path

# Read runtime-provided variables
TASK_ID = os.getenv("TASK_ID")
DATASET_ID = os.getenv("DATASET_ID")
DATA_PATH = os.getenv("DATA_PATH", "/app/data")
OUTPUT_PATH = os.getenv("OUTPUT_PATH", "/app/output")

# Read data
data_files = list(Path(DATA_PATH).rglob("*.json"))

# Write output
output_file = Path(OUTPUT_PATH) / "results.json"

Custom Variables

Add your own environment variables for task configuration:

Best Practices:

1. Prefix with your task name to avoid collisions
2. Always provide defaults in your code
3. Document in .env.example
4. Don't store secrets (use secure secret management)

Example:

# Custom configuration
MODEL_PATH = os.getenv("MYTASK_MODEL_PATH", "/app/models/default.pt")
BATCH_SIZE = int(os.getenv("BATCH_SIZE", "32"))
MAX_WORKERS = int(os.getenv("MAX_WORKERS", "4"))
ENABLE_CACHING = os.getenv("ENABLE_CACHING", "true").lower() == "true"

Local Testing

Set variables in docker-compose.yml:

environment:
  - TASK_ID=1
  - DATASET_ID=100
  - RUN_ID=test-run-123
  - LOG_LEVEL=DEBUG
  - BATCH_SIZE=16

Or with manual docker run:

docker run \
  -e TASK_ID=1 \
  -e DATASET_ID=100 \
  -e RUN_ID=test \
  -v $(pwd)/data:/app/data:ro \
  -v $(pwd)/output:/app/output \
  myorg/my-task:v1

See .env.example for complete reference.

---

🔐 Security Guidelines

What to Return in Responses

✅ SAFE (Aggregate Statistics):

{
  "result": "processed",
  "count": 1000,
  "debug_info": {
    "processing_time_ms": 45000,
    "memory_peak_mb": 512,
    "files_processed": 1000,
    "files_skipped": 10,
    "error_count": 2,
    "average_processing_time_per_file_ms": 45,
    "unique_categories_found": 15
  }
}

❌ UNSAFE (Raw Data/PII):

{
  "files": [
    {"user_id": "123", "email": "user@example.com", "data": "..."},  // ❌ PII
    {"content": "raw decrypted data..."}                              // ❌ Sensitive
  ],
  "logs": "Processing file user_123_private_data.txt..."             // ❌ Reveals data
}

Rule: Return statistics and aggregates, NEVER raw data or PII.

---

🧪 Local Testing

1. Create Test Data

# Create sample data file
mkdir -p data
echo '{"test": "data"}' > data/sample.json

2. Run Locally

# Start task
docker-compose up --build

# Test health
curl http://localhost:8000/health

# Test manifest
curl http://localhost:8000/manifest

# Test operation
curl -X POST http://localhost:8000/process \
  -H "Content-Type: application/json" \
  -d '{"batch_id": "test-001", "filters": {"min_quality": 0.8}}'

# Check outputs
ls -la output/

3. Test Async Operations

# Start async operation
curl -X POST http://localhost:8000/analyze \
  -H "Content-Type: application/json" \
  -d '{"dataset": "full"}'
# Returns: {"operation_id": "task-op-abc123"}

# Check status
curl http://localhost:8000/operation/task-op-abc123/status
# Returns: {"status": "processing", "progress": 0.5}

# Cancel operation
curl -X POST http://localhost:8000/operation/task-op-abc123/cancel
# Returns: {"status": "cancelled"}

---

📖 Required Endpoints Reference

Your task MUST implement these endpoints for Vana Runtime integration:

1. Health Check (REQUIRED)

@app.get("/health")
async def health():
    return {"status": "healthy"}

Called by: Runtime every 60 seconds
Purpose: Verify task is alive
Failure: 5 consecutive failures → task marked as failed

2. Operations Manifest (REQUIRED)

Option A: Static file at /app/operations.json (recommended)

Option B: Dynamic endpoint

@app.get("/manifest")
async def get_manifest():
    return {"version": "1.0", "operations": [...]}

Purpose: Runtime validates all operation submissions against this

3. Declared Operations (REQUIRED)

Implement all operations declared in your manifest:

@app.post("/your-operation-path")
async def your_operation(request: YourRequest):
    # Your logic here
    return {"result": "success"}

4. Async Operation Status (REQUIRED if async)

@app.get("/operation/{operation_id}/status")
async def get_operation_status(operation_id: str):
    return {
        "operation_id": operation_id,
        "status": "completed",  # or "processing", "failed"
        "result": {...}
    }

5. Async Operation Cancel (REQUIRED if async)

@app.post("/operation/{operation_id}/cancel")
async def cancel_operation(operation_id: str):
    # Cancel background processing
    return {"operation_id": operation_id, "status": "cancelled"}

6. Graceful Shutdown (OPTIONAL but recommended)

@app.post("/shutdown")
async def shutdown():
    # Cleanup resources
    return {"acknowledged": True}

---

🚀 Deployment

Deployment Architecture

Your task runs inside Vana Runtime's TEE environment:

┌─────────────────────────────────────────┐
│         Vana Runtime (TEE)              │
│  ┌───────────────────────────────────┐  │
│  │   vana-network (internet access)  │  │
│  │   - Runtime API                   │  │
│  │   - Database                      │  │
│  │   - Onchain queries               │  │
│  └───────────────────────────────────┘  │
│                  ↕                       │
│  ┌───────────────────────────────────┐  │
│  │ vana-tasks-network (ISOLATED)     │  │
│  │   ┌─────────────────────────┐     │  │
│  │   │   Your Task Container   │     │  │
│  │   │   - No internet access  │     │  │
│  │   │   - Data mounted at     │     │  │
│  │   │     /app/data (ro)      │     │  │
│  │   │   - Output at           │     │  │
│  │   │     /app/output (rw)    │     │  │
│  │   └─────────────────────────┘     │  │
│  └───────────────────────────────────┘  │
└─────────────────────────────────────────┘

Key Points:

• 🔒 Network Isolation: Your task has NO internet access (TEE security)
• 📁 Volume Mounts: Runtime mounts data/output volumes automatically
• 🔍 Health Monitoring: Runtime polls /health every 60 seconds
• 📊 Operation Tracking: All operations logged by runtime for audit trail
• ⚡ Communication: All client↔task communication flows through runtime

---

1. Build Your Task

# Build Docker image
docker build -t myorg/my-task:v1 .

# Test locally first
docker run -p 8000:8000 \
  -e TASK_ID=1 \
  -e DATASET_ID=100 \
  -e RUN_ID=test \
  -v $(pwd)/data:/app/data:ro \
  -v $(pwd)/output:/app/output \
  myorg/my-task:v1

Requirements:

• ✅ Must expose port 8000
• ✅ Must include /app/operations.json
• ✅ Must implement /health endpoint
• ✅ AMD64 architecture (for TEE compatibility)

---

2. Push to Registry

# Push to Docker Hub (or your registry)
docker push myorg/my-task:v1

# Note image hash (needed for onchain registration)
docker inspect myorg/my-task:v1 --format='{{.Id}}'

Registry Options:

• Docker Hub (public/private)
• GitHub Container Registry
• AWS ECR
• Google Container Registry

Security Note: Runtime pulls images over HTTPS and verifies image hash

---

3. Register with Vana (Onchain)

Step A: Register Task

// Register task in onchain registry
registerTask(
    taskId=123,
    imageUrl="myorg/my-task:v1",
    imageHash="sha256:abc123...",  // From docker inspect
    datasetId=100
)

Step B: Get Dataset Owner Approval

// Dataset owner must approve your task
approveTaskForDataset(
    taskId=123, 
    datasetId=100
)

Approval Process:

1. Task developer registers task with image URL and hash
2. Dataset owner reviews task code and manifest
3. Dataset owner approves task for their dataset
4. Task becomes available for execution

Security: Only approved tasks can access dataset data

---

4. Execute Task in Runtime

Start Task:

# Runtime pulls image, starts container, mounts volumes
curl -X POST "http://runtime:8000/task/123?dataset_id=100"

# Response:
# {
#   "task_id": 123,
#   "run_id": "123-abc123de",
#   "status": "pending"
# }

Monitor Task:

# Check task status
curl http://runtime:8000/task/123/status

# Response when ready:
# {
#   "status": "running",
#   "container_name": "task-123-abc-...",
#   "has_manifest": true,
#   "operations_count": 3
# }

Submit Operations:

# Submit operation (runtime validates against manifest)
curl -X POST http://runtime:8000/task/123/operation/process_batch \
  -H "Content-Type: application/json" \
  -d '{"batch_id": "batch-001"}'

# Response (if sync):
# {"result": "processed", "count": 42}

# Response (if async):
# {"operation_id": "123-abc-op1", "status": "processing"}

Download Artifacts:

# List artifacts
curl http://runtime:8000/task/123/artifacts

# Download artifact
curl -O http://runtime:8000/task/123/artifact/artifact-123

---

5. Monitoring & Debugging

Runtime Health Checks:

• Runtime calls GET /health every 60 seconds
• 5 consecutive failures → Task marked as failed
• Failed tasks are stopped and cleaned up

Operation Tracking:

• All operations logged in runtime database
• Query operation history: GET /task/123/operations
• Get operation details: GET /task/123/operation/{op_id}

Logs:

# View task container logs (via runtime host)
docker logs task-123-abc123de-myimage-a1b2

# Runtime logs (includes all runtime↔task communication)
docker logs vana-runtime

Debug Info:

• Operations return debug_info with stats
• Use /stats endpoint for task-level metrics
• No log streaming (security - prevents data leakage)

---

6. Lifecycle Management

Stop Task:

# Graceful shutdown (calls /shutdown, then stops container)
curl -X POST http://runtime:8000/task/123/stop

Cancel Operation:

# Cancel async operation
curl -X POST http://runtime:8000/task/123/operation/{op_id}/cancel

Cleanup:

• Runtime automatically cleans up stopped tasks
• Artifacts persist until manually deleted or expired
• Container and volumes removed on task stop

---

🔍 Debugging

View Logs

# Local testing
docker-compose logs -f

# Check outputs
ls -la output/

# Inspect container
docker exec -it vana-task-dev /bin/bash

Common Issues

"Operation not found in manifest"

• Cause: Mismatch between operations.json and runtime request
• Fix: Ensure operation name in operations.json matches exactly
• Check: curl http://localhost:8000/manifest and verify operation is listed

"Task has no operations manifest"

• Cause: Manifest file missing or not accessible
• Fix: Ensure operations.json is copied in Dockerfile: COPY operations.json .
• Alternative: Implement /manifest endpoint as fallback

"Health check failed"

• Cause: Task crashed or /health endpoint not responding
• Fix: Check container logs: docker logs vana-task-dev
• Check: Ensure /health returns status code 200-299

"No data files found"

• Cause: Data directory not mounted or empty
• Fix: Verify docker-compose.yml mounts ./data:/app/data
• Test: Create sample data: echo '{"test":"data"}' > data/sample.json

"Container exits immediately"

• Cause: Application error on startup
• Fix: Check logs for Python errors
• Debug: Run interactively: docker run -it myorg/my-task:v1 /bin/bash

"Operation times out"

• Cause: Operation takes longer than 600s default timeout
• Fix: For long operations, use async pattern (return operation_id)
• Alternative: Request runtime timeout increase (PROXY_TIMEOUT env var)

"Async operation stuck in processing"

• Cause: Status endpoint not returning "completed" status
• Fix: Ensure /operation/{id}/status returns status: "completed" when done
• Check: Valid completion statuses: "completed", "success", "done", "finished"

"Runtime can't connect to task"

• Cause: Task not exposing port 8000 or not listening on 0.0.0.0
• Fix: Ensure Dockerfile: EXPOSE 8000 and uvicorn: --host 0.0.0.0 --port 8000

Debug Checklist

When troubleshooting issues:

• Check container is running: docker ps
• Verify health endpoint: curl http://localhost:8000/health
• Verify manifest loaded: curl http://localhost:8000/manifest
• Check logs for errors: docker logs vana-task-dev
• Verify data directory has files: ls -la data/
• Check output directory permissions: ls -ld output/
• Test operations with curl (see test-task.sh)
• Verify operations.json syntax: jq . operations.json

Getting Help

Template Issues:

• Review inline TODO-CUSTOMIZE comments
• Check this README's examples
• Verify operations.json format

Runtime Integration:

• See Vana Runtime TASK_API_CONTRACT.md
• Verify all required endpoints implemented
• Check health check response format

Production Issues:

• Check runtime logs: docker logs vana-runtime
• Query operation history: GET /task/{id}/operations
• Review operation details: GET /task/{id}/operation/{op_id}

---

📝 Customization Examples

Example 1: Simple Data Processing

@app.post("/process")
async def process_batch(request: ProcessBatchRequest):
    import pandas as pd
    from pathlib import Path
    
    # Read CSV files from data directory
    data_files = Path(DATA_PATH).rglob("*.csv")
    dfs = [pd.read_csv(f) for f in data_files]
    combined = pd.concat(dfs)
    
    # Apply filters
    if request.filters:
        if "min_quality" in request.filters:
            combined = combined[combined['quality'] >= request.filters['min_quality']]
    
    # Process and aggregate
    summary = {
        "total_rows": len(combined),
        "unique_ids": combined['id'].nunique(),
        "average_score": combined['score'].mean()
    }
    
    # Write output
    output_file = Path(OUTPUT_PATH) / f"{request.batch_id}_summary.json"
    with open(output_file, 'w') as f:
        json.dump(summary, f)
    
    return ProcessBatchResponse(
        result="processed",
        count=len(combined),
        debug_info={
            "processing_time_ms": 2000,
            "unique_ids": summary["unique_ids"],
            "average_score": float(summary["average_score"])
        }
    )

Example 2: ML Model Inference

# Add to requirements.txt:
# torch==2.1.0
# transformers==4.35.2

from transformers import pipeline

# Load model on startup
classifier = None

@app.on_event("startup")
async def startup():
    global classifier
    classifier = pipeline("sentiment-analysis")
    logger.info("Model loaded")

@app.post("/analyze")
async def analyze(request: AnalyzeRequest):
    # Start async processing
    operation_id = create_async_operation("analyze", {})
    
    # In background: run inference and track progress
    asyncio.create_task(run_inference(operation_id))
    
    return AnalyzeResponse(operation_id=operation_id)

---

🎓 Learning Resources

Vana Runtime Documentation:

• TASK_API_CONTRACT.md - Complete API specification
• RUNTIME_TASK_INTEGRATION.md - Integration guide
• SECURITY_REFACTOR.md - Security model

FastAPI Documentation:

• https://fastapi.tiangolo.com/

Docker Documentation:

• https://docs.docker.com/

---

✅ Pre-Deployment Checklist

Before deploying to Vana Runtime:

• All TODO-CUSTOMIZE items addressed
• Operations manifest matches actual endpoints
• Tested locally with docker-compose
• Debug info returns aggregates only (no raw data/PII)
• Error handling implemented
• Health check always returns 2xx when task is working
• Async operations implement status and cancel endpoints
• Shutdown endpoint cleans up resources
• Docker image builds successfully
• Tested in AMD64 environment (for TEE compatibility)

---

🔍 TODO Tag Reference

Use these to find customization points:

# Find all customization points
grep -r "TODO-CUSTOMIZE" main.py

# Find specific customization types
grep "TODO-CUSTOMIZE: Define" main.py    # Data models
grep "TODO-CUSTOMIZE: Implement" main.py  # Operation logic
grep "TODO-CUSTOMIZE: Add" main.py        # Additional features

---

🆘 Support

For template issues:

• Check this README
• Review inline TODO-CUSTOMIZE comments
• Check Vana Runtime documentation

For runtime integration:

• See TASK_API_CONTRACT.md in vana-runtime repo
• Check /health endpoint requirements
• Verify operations manifest format

---

📄 License

[Your license here]

---

🎉 You're Ready!

This template provides everything needed to build a production-ready task for Vana Runtime. Follow the TODO-CUSTOMIZE comments, test locally, and deploy with confidence.

Happy building! 🚀