[Feature] Phase 3: Enhanced Metrics & Job Tracking (DORA)

[GrammaTonic]

📊 Phase 3: Enhanced Metrics & Job Tracking (DORA)

Timeline: Week 2-3 (2025-11-26 to 2025-12-03)
Status: ⏳ Blocked by Phase 2
Goal: Add job duration tracking, cache hit rates, and queue time metrics for DORA calculations

🎯 Objectives

• Implement job duration histogram with buckets
• Track queue time (job assignment to start)
• Measure cache hit rates (BuildKit, apt, npm)
• Enable DORA metrics calculations

✅ Tasks (10 Total)

• TASK-027: Extend /tmp/jobs.log format to include: timestamp,job_id,status,duration_seconds,queue_time_seconds (CSV format)
• TASK-028: Implement job start/end time tracking by hooking into GitHub Actions runner job lifecycle (via log parsing)
• TASK-029: Update metrics collector to calculate job duration histogram buckets
• TASK-030: Add queue time metric: github_runner_queue_time_seconds
• TASK-031: Implement cache hit rate tracking by parsing Docker BuildKit cache logs
• TASK-032: Add cache metrics: github_runner_cache_hit_rate{cache_type="buildkit|apt|npm"}
• TASK-033: Update metrics collector script to read cache logs from /var/log/buildkit.log
• TASK-034: Test job duration tracking by running actual GitHub Actions workflows
• TASK-035: Validate cache metrics with controlled builds (force cache miss vs cache hit scenarios)
• TASK-036: Document job log format in docs/features/PROMETHEUS_IMPROVEMENTS.md

📋 Acceptance Criteria

• ✅ Job duration histogram captures p50, p95, p99 durations
• ✅ Queue time accurately reflects time between job assignment and start
• ✅ Cache hit rate metrics track BuildKit, apt, and npm cache performance
• ✅ DORA metrics can be calculated from collected data

🔗 Dependencies

• BLOCKED BY: Phase 2 (requires metrics infrastructure)

---

Part of: Prometheus Monitoring Implementation (v2.3.0)