Ducto DSL

---

About

dsl-transformer is a lightweight, embeddable data transformation engine designed for structured data (JSON, maps, structs). Transformations are defined using a DSL (JSON or HCL-based), making it suitable for use cases like:

• Event stream manipulation
• API response mutation
• ETL pipelines
• Testing tools and CLI automation

It is:

• ⚡ Minimal
• 🟣 Composable
• 🟩 Fully testable

dsl-transformer is a part of the larger Ducto project, combining many interesting practices together including Feature Flagging.

DSL Specification

See doc/specs.md for a specification of the DSL.

---

Features (WIP)

• CLI for local testing
• Serverless runtime compatibility
• Declarative set and copy operations
• Support for basics map, delete, merge
• Linter included for instruction validation
• Conditionals
• Aggregations and Filtering
• Embeddable Go SDK
• Input can be JSON or YAML
• HCL-powered syntax option
• OpenTelemetry instrumentation

Also, see our OSS Release Checklist.

---

Example

examples/01-simplest.json

Purpose: Show the simplest possible working example of a Ducto 'program'.

{
  "version": 1,
  "on_error": "fail",
  "instructions": [
    {"op": "set", "key": "greeting", "value": "hello world"}
  ]
}

examples/02-enrich-log.json

Purpose: Enrich incoming telemetry events with environment defaults, severity mapping, and drop test/debug data.

Input:

{
  "message": "Disk space low",
  "level": "warn"
}

The goal here is to:

• Default missing env to "default-env".
• Convert "level" into a "severity" field using a mapping.
• Remove debug_info.
• Set a "processed": true flag.

Program:

{
  "version": 1,
  "instructions": [
    
    // Defaults
    { "op": "coalesce", "key": "env", "value": "default-env" },
    { "op": "coalesce", "key": "level", "value": "low" },

    // Alter severity by known log levels 
    { "op": "if", "condition": { "equals": { "key": "level", "value": "warn" } }, "then": [
      { "op": "set", "key": "severity", "value": "medium" }
    ]},
    {
      "op": "if",
      "condition": {
        "or": [
          { "equals": { "key": "level", "value": "error" } },
          { "equals": { "key": "level", "value": "critical" } }
        ]
      },
      "then": [
        { "op": "set", "key": "severity", "value": "high" }
      ]
    },

    // Remove unnecessary keys
    { "op": "delete", "key": "debug_info" },

    // Not needed, but by set this value last we show we've finished processing this program
    { "op": "set", "key": "processed", "value": true }
  ]
}

Output:

{
  "message": "Disk space low",
  "level": "warn",
  "env": "default-env",
  "severity": "medium",
  "processed": true
}

Install

go install github.com/tommed/ducto-dsl/cmd/ducto-dsl@latest

# Run (From a file)
ducto-dsl program.json < input.json

# Run (From piped text)
echo '{"foo": "bar"}' | ducto-dsl program.json

# Lint
ducto-dsl lint program.json

From Source

Lint

go run ./cmd/ducto-dsl lint examples/01-simplest.json

Run

# Simple Example
echo '{"foo":"bar"}' | go run ./cmd/ducto-dsl examples/01-simplest.json

# Telemetry Example
go run ./cmd/ducto-dsl examples/02-enrich-log.json < test/data/input-telemetry-log.json

Development

Please make sure you read our Code of Conduct before engaging with this project.

Testing

make test         # Short tests
make test-full    # Full tests
make coverage     # Coverage report (HTML)
make lint-install # Install lint prerequisites
make lint         # Run static analysis
make clean        # Remove binaries and generated artifacts

CLI

There are Makefile targets for a macOS binary and Windows binary. Or simply build all:

# Build all binaries
make build-all

# Build (macOS)
make ducto-dsl-macos

# Build (Microsoft Windows)
make ducto-dsl-windows

Status

See status.md for up-to-date CI, coverage, and project health. Our OSS Release Checklist also provides an overview of where we are with this project.

License

• Code is all licensed under MIT
• The Ducto name, logos and robot illustrations (and likeness) are (C) Copyright 2025 Tom Medhurst, all rights reserved.