-
Notifications
You must be signed in to change notification settings - Fork 14
Open
Description
Description
Add visual diagrams to documentation to illustrate data flow and stream processing concepts.
Scope
Create diagrams showing:
- Data flow between Kafka broker, Kafka Streams, and KSML
- Topic input/output relationships
- Stream operations (filter, map, groupBy, aggregate)
- Windowed aggregations with time progression
- State stores and changelog topics
- Multi-pipeline interactions
Suggested Tools
- Mermaid (GitHub native support)
- Mermaid diagrams work out of the box with our mkdocs-material in Github Pages, just add to mkdocs.yaml:
extra_javascript:
- https://unpkg.com/mermaid@10/dist/mermaid.min.js
extra:
mermaid2: true
Target Locations
- Front page
- Quick Start, "Understanding KSML"
- Tutorial pages (intermediate/aggregations.md, basics-tutorial.md)
- Architecture/concepts sections
- Examples with complex pipelines
Acceptance Criteria
- At least 5-10 diagrams added to key documentation pages
- Diagrams are clear and help users understand data flow
- Source files included for future editing
- Consistent style across all diagrams
- Review the docs with a focus on making them clear for “a data engineer, analyst, or Python developer” who’s comfortable with “SQL, Python, and scripting language”.
Metadata
Metadata
Assignees
Labels
No labels