Serverless Task Scheduler

A serverless application built on AWS that allows scheduling tasks to run at specific times using AWS Lambda, DynamoDB, and EventBridge.

Architecture Overview

graph TD
    Client[Client] -->|POST /schedule-task| API[API Gateway]
    API -->|Invokes| ScheduleLambda[Schedule Task Lambda]
    ScheduleLambda -->|Stores task| DynamoDB[(DynamoDB)]
    ScheduleLambda -->|Creates rule| EventBridge[EventBridge]
    EventBridge -->|Triggers at scheduled time| ExecuteLambda[Execute Task Lambda]
    ExecuteLambda -->|Retrieves task| DynamoDB
    ExecuteLambda -->|Executes action<br/>e.g., HTTP webhook| ExternalSystem[External System]

Loading

AWS Services Used

1. AWS Lambda

   • Schedule Task Function: Handles API requests to create new tasks
   • Execute Task Function: Runs at scheduled times to execute the tasks

2. Amazon DynamoDB

   • Stores task definitions, status, and metadata
   • Schema includes task ID, action type, payload, status, timestamps

3. Amazon EventBridge (CloudWatch Events)

   • Creates scheduled rules based on the requested execution time
   • Targets the Execute Task Lambda when the scheduled time arrives

4. AWS API Gateway

   • Exposes the REST API endpoint for scheduling tasks

5. AWS SAM/CloudFormation

   • Defines the infrastructure as code
   • Manages deployment and resources

Technical Details

Data Flow

1. Task Creation:

   • Client sends POST request to /schedule-task endpoint
   • Schedule Task Lambda validates the request
   • Generates a unique task ID
   • Stores task in DynamoDB
   • Creates EventBridge rule to trigger at specified time

2. Task Execution:

   • EventBridge rule triggers Execute Task Lambda at scheduled time
   • Lambda retrieves task details from DynamoDB
   • Updates task status to "running"
   • Executes the requested action (e.g., HTTP webhook)
   • Updates task status to "completed" or "failed"

Task States

• scheduled: Initial state when task is created
• running: Task is currently being executed
• completed: Task executed successfully
• failed: Task execution failed

Trade-offs and Considerations

Advantages

1. Serverless Architecture

   • No server management required
   • Pay-per-use pricing model
   • Automatic scaling

2. Decoupled Components

   • Separation of task scheduling and execution
   • Each function has a single responsibility

3. Persistence and Durability

   • Tasks are stored in DynamoDB for durability
   • Task state is tracked and can be queried

4. Event-Driven

   • Leverages EventBridge for time-based events
   • No need for polling or continuous running processes

Limitations

1. EventBridge Limitations

   • Maximum of 300 rules per event bus
   • Each rule can have maximum 5 targets
   • Creates a rule per task which may not scale well for thousands of scheduled tasks

2. Scheduling Granularity

   • Current implementation uses cron expressions with minute-level granularity
   • No sub-minute scheduling possible with this design

3. Error Handling

   • Limited retry functionality (would need to be implemented separately)
   • No dead letter queue for failed executions

4. Cost Considerations

   • Each scheduled task creates an EventBridge rule (pricing implications)
   • DynamoDB provisioned throughput costs for read/write capacity

5. Rule Cleanup

   • The current implementation doesn't automatically delete EventBridge rules after execution
   • Could lead to rule proliferation over time

Deployment Instructions

Prerequisites

• AWS CLI installed and configured
• AWS SAM CLI installed
• Node.js (v22.x or compatible version)

Deployment Steps

1. Clone the repository

git clone https://github.com/yourusername/serverless-task-scheduler.git
cd serverless-task-scheduler

2. Install dependencies

npm install

3. Build the application

sam build

4. Deploy to AWS

sam deploy --guided

Follow the prompts to complete the deployment:

• Stack Name: task-scheduler
• AWS Region: your preferred region
• Confirm changes before deployment: Y
• Allow SAM CLI to create IAM roles: Y
• Save arguments to samconfig.toml: Y

5. Note the API endpoint

After deployment completes, the API Gateway endpoint will be displayed in the outputs section. Save this URL for testing.

Local Testing

1. Start local API

sam local start-api

2. Test scheduling a task

curl -X POST http://localhost:3000/schedule-task \
  -H "Content-Type: application/json" \
  -d '{
    "action": "webhook",
    "payload": {
      "url": "https://webhook.site/your-webhook-id",
      "data": {"message": "Task executed!"}
    },
    "run_at": "2025-05-17T15:00:00Z"
  }'

3. Test task execution locally

sam local invoke executeTaskFunction \
  --event events/task-execution-event.json

Create a file events/task-execution-event.json with content:

{
  "source": "custom.scheduler",
  "detail": {
    "taskId": "your-task-id-from-previous-response"
  }
}

Monitoring and Debugging

• View Lambda logs in CloudWatch Logs
• Monitor task status in DynamoDB
• Track EventBridge rules in the EventBridge console

Project Details

Scheduling Implementation

• Default Configuration: The current implementation uses a recurring daily cron expression that runs at the specified hour and minute from the run_at parameter
• One-Time Execution: To make tasks execute only once, modify the code to use the at(yyyy-mm-ddThh:mm:ss) schedule expression format instead of cron, as mentioned in the code comments:

  // Instead of:
  const cronExpression = `cron(${minute} ${hour} * * ? *)`;
  
  // Use for one-time execution:
  const atExpression = `at(${scheduledDate.toISOString().slice(0, 19)})`;

Supported Actions

• Current Actions:

  • webhook: The only action type currently implemented
  • Sends an HTTP POST request to the URL specified in the payload

• Testing Webhooks:

  • webhook.site is recommended for testing webhook execution
  • Generate a unique webhook URL from webhook.site
  • Specify this URL in your task payload
  • Monitor incoming requests on webhook.site to verify task execution

• Sample Webhook Payload:

  {
    "action": "webhook",
    "payload": {
      "url": "https://webhook.site/your-unique-webhook-id",
      "data": {
        "message": "Task executed successfully",
        "timestamp": "2025-05-16T12:00:00Z"
      }
    },
    "run_at": "2025-05-16T12:00:00Z"
  }