Skip to content

Add Visual Diagrams to KSML Documentation #407

@KasparMetsa

Description

@KasparMetsa

Description

Add visual diagrams to documentation to illustrate data flow and stream processing concepts.

Image Image Image Image

Scope

Create diagrams showing:

  • Data flow between Kafka broker, Kafka Streams, and KSML
  • Topic input/output relationships
  • Stream operations (filter, map, groupBy, aggregate)
  • Windowed aggregations with time progression
  • State stores and changelog topics
  • Multi-pipeline interactions

Suggested Tools

  • Mermaid (GitHub native support)
    • Mermaid diagrams work out of the box with our mkdocs-material in Github Pages, just add to mkdocs.yaml:
extra_javascript:
  - https://unpkg.com/mermaid@10/dist/mermaid.min.js

extra:
  mermaid2: true

Target Locations

  • Front page
  • Quick Start, "Understanding KSML"
  • Tutorial pages (intermediate/aggregations.md, basics-tutorial.md)
  • Architecture/concepts sections
  • Examples with complex pipelines

Acceptance Criteria

  • At least 5-10 diagrams added to key documentation pages
  • Diagrams are clear and help users understand data flow
  • Source files included for future editing
  • Consistent style across all diagrams
  • Review the docs with a focus on making them clear for “a data engineer, analyst, or Python developer” who’s comfortable with “SQL, Python, and scripting language”.

DEMO_mermaid_diagram.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions