Phase 1053: Workflow-Session Unification

.specflow/orchestration-state.json

@@ -5,7 +5,7 @@
     "name": "specflow",
     "path": "/Users/ppatterson/dev/specflow"
   },
-  "last_updated": "2026-01-19T07:30:21.594Z",
+  "last_updated": "2026-01-20T04:25:45.064Z",
   "orchestration": {
     "phase": {
       "number": null,
@@ -14,8 +14,8 @@
       "status": "not_started"
     },
     "next_phase": {
-      "number": "1053",
-      "name": "Workflow-Session Unification"
+      "number": "1055",
+      "name": "Smart Batching & Orchestration"
     },
     "step": {
       "current": "design",
@@ -272,6 +272,15 @@
         "completed_at": "2026-01-19T07:30:21.594Z",
         "tasks_completed": 0,
         "tasks_total": 0
+      },
+      {
+        "type": "phase_completed",
+        "phase_number": "1053",
+        "phase_name": "Workflow-Session Unification",
+        "branch": "1053-workflow-session-unification",
+        "completed_at": "2026-01-20T04:25:45.063Z",
+        "tasks_completed": 0,
+        "tasks_total": 0
       }
     ]
   }

.specify/archive/1053-workflow-session-unification/checklists/implementation.md

@@ -0,0 +1,56 @@
+# Implementation Checklist: Workflow-Session Unification
+
+**Purpose**: Verify implementation quality during development
+**Created**: 2026-01-19
+**Feature**: [spec.md](../spec.md)
+
+## Core Session ID Fix
+
+- [ ] I-001 `findNewSession()` function is completely removed from workflow-service.ts
+- [ ] I-002 No polling of `sessions-index.json` anywhere in codebase
+- [ ] I-003 Session ID is obtained exclusively from CLI JSON output `result.session_id`
+- [ ] I-004 `WorkflowExecutionSchema` includes required `sessionId` field
+
+## Storage Architecture
+
+- [ ] I-005 Workflow metadata stored at `{project}/.specflow/workflows/{sessionId}/metadata.json`
+- [ ] I-006 Index file exists at `.specflow/workflows/index.json`
+- [ ] I-007 Index file contains all required fields: sessionId, skill, status, startedAt, costUsd
+- [ ] I-008 No duplication of Claude JSONL files - linking only
+- [ ] I-009 `.specflow/workflows/` added to `.gitignore` automatically
+- [ ] I-010 Old global workflows in `~/.specflow/workflows/` are cleaned up
+
+## API Implementation
+
+- [ ] I-011 `/api/workflow/start` creates pending workflow correctly
+- [ ] I-012 `/api/workflow/start` migrates to session-keyed storage after session ID received
+- [ ] I-013 `/api/workflow/status` reads from project-local path
+- [ ] I-014 `/api/workflow/list` reads from project-local index.json
+- [ ] I-015 `/api/session/history` returns sessions sorted by startedAt descending
+
+## UI Components
+
+- [ ] I-016 `SessionPendingState` component shows loading state correctly
+- [ ] I-017 `SessionViewerDrawer` accepts explicit sessionId prop
+- [ ] I-018 `SessionViewerDrawer` shows pending state when sessionId is null
+- [ ] I-019 `SessionHistoryList` displays sessions in table format
+- [ ] I-020 Active sessions have green indicator in history list
+- [ ] I-021 Clicking session row opens Session Viewer with that session
+
+## Resume Capability
+
+- [ ] I-022 `useWorkflowExecution` supports starting with `resumeSessionId`
+- [ ] I-023 `/api/workflow/start` accepts `resumeSessionId` parameter
+- [ ] I-024 Follow-up input on historical sessions creates new workflow with resume flag
+
+## Error Handling
+
+- [ ] I-025 Missing session files handled gracefully (show error, don't crash)
+- [ ] I-026 CLI failures before session ID captured marked as failed workflow
+- [ ] I-027 Orphaned pending workflows cleaned up appropriately
+
+## Notes
+
+- Check items off as completed: `[x]`
+- Implementation items verify code is written correctly
+- Run `specflow check --gate implement` to verify completion

.specify/archive/1053-workflow-session-unification/checklists/verification.md

@@ -0,0 +1,56 @@
+# Verification Checklist: Workflow-Session Unification
+
+**Purpose**: Verify feature works correctly post-implementation
+**Created**: 2026-01-19
+**Feature**: [spec.md](../spec.md)
+
+## User Story 1: Start Workflow and See Session (P1)
+
+- [ ] V-001 Start workflow from dashboard, session ID appears in workflow state after first CLI response
+- [ ] V-002 Session ID available within 2 seconds of first CLI response completing
+- [ ] V-003 Session Viewer drawer shows correct session content (not stale/different session)
+- [ ] V-004 When session ID not yet available, UI shows "Waiting for session..." gracefully
+- [ ] V-005 No race conditions when opening Session Viewer immediately after starting workflow
+
+## User Story 2: View Session History (P2)
+
+- [ ] V-006 Project detail shows "Sessions" section with list of past sessions
+- [ ] V-007 Sessions list includes: sessionId, skill name, status, timestamp, cost
+- [ ] V-008 Sessions sorted by timestamp (most recent first)
+- [ ] V-009 Clicking session row opens Session Viewer with that session's messages
+- [ ] V-010 Active session has green indicator distinguishing it from historical sessions
+- [ ] V-011 Up to 50 sessions displayed in history
+
+## User Story 3: Resume Any Past Session (P3)
+
+- [ ] V-012 Can type follow-up message when viewing any past session
+- [ ] V-013 Sending follow-up creates new workflow using `--resume {sessionId}`
+- [ ] V-014 Resumed session continues with correct context
+
+## Edge Cases
+
+- [ ] V-015 CLI crash before session ID: Workflow marked as failed, user can retry
+- [ ] V-016 Multiple rapid workflow starts: Each gets unique session, no conflicts
+- [ ] V-017 Session files missing from Claude storage: Graceful error message shown
+
+## Phase USER GATE Criteria (from phase file)
+
+- [ ] V-018 Start workflow → Session ID available within 2 seconds
+- [ ] V-019 Session Viewer shows correct session immediately
+- [ ] V-020 Can view history of past workflow sessions
+- [ ] V-021 No race conditions when starting multiple workflows sequentially
+
+## UI Design Verification
+
+- [ ] V-UI1 UI implementation matches ui-design.md mockups
+- [ ] V-UI2 SessionHistoryList shows sessions in table format per mockup
+- [ ] V-UI3 SessionPendingState shows appropriate placeholder per mockup
+- [ ] V-UI4 Active session indicator (green dot) visible in history list
+- [ ] V-UI5 Dark mode styling consistent with rest of dashboard
+
+## Notes
+
+- Check items off as completed: `[x]`
+- Verification items must be tested with real workflows
+- Run `specflow check --gate verify` to verify completion
+- Items V-018 through V-021 are the USER GATE - must pass for phase completion

.specify/archive/1053-workflow-session-unification/discovery.md

@@ -0,0 +1,185 @@
+# Discovery: Workflow-Session Unification
+
+**Phase**: `1053-workflow-session-unification`
+**Created**: 2026-01-19
+**Status**: Complete
+
+## Phase Context
+
+**Source**: ROADMAP phase, PDR: workflow-dashboard-orchestration.md
+**Goal**: Unify workflows and Claude sessions as the same concept, fixing session detection on workflow start.
+
+---
+
+## Codebase Examination
+
+### Related Implementations
+
+| Location | Description | Relevance |
+|----------|-------------|-----------|
+| `packages/dashboard/src/lib/services/workflow-service.ts` | Core workflow execution service | Primary target - stores workflows at `~/.specflow/workflows/{execution_id}.json` |
+| `packages/dashboard/src/lib/session-parser.ts` | Parses Claude JSONL session files | Extracts messages from `~/.claude/projects/{hash}/{session_id}.jsonl` |
+| `packages/dashboard/src/hooks/use-workflow-execution.ts` | React hook for workflow state | Polls workflow status, triggers notifications |
+| `packages/dashboard/src/hooks/use-session-messages.ts` | React hook for session messages | Uses `findActiveSession()` to discover sessions via polling |
+| `packages/dashboard/src/components/projects/session-viewer-drawer.tsx` | Session viewer UI | Displays session messages, needs sessionId to work |
+| `packages/dashboard/src/app/api/session/active/route.ts` | API to find active session | Polls `sessions-index.json` with timestamp heuristics |
+| `packages/dashboard/src/lib/project-hash.ts` | Calculates Claude project hash | Used for session file path resolution |
+
+### Existing Patterns & Conventions
+
+- **UUID for Execution IDs**: Workflow executions use `randomUUID()` for unique IDs (`workflow-service.ts:491`)
+- **File-based Persistence**: Workflow state saved to JSON files, not database
+- **Polling Pattern**: 3-second intervals for status updates (`use-workflow-execution.ts:21`)
+- **Session Detection**: Current approach uses `findNewSession()` which polls `sessions-index.json` after initial delay (`workflow-service.ts:432-470`)
+
+### Integration Points
+
+- **CLI Invocation**: Workflows spawn Claude CLI with `--output-format json` which returns `session_id` in response
+- **Session Files**: Claude stores sessions at `~/.claude/projects/{hash}/{session_id}.jsonl`
+- **Session Index**: Claude writes `sessions-index.json` immediately when CLI starts
+
+### Key Discovery: Session ID is Already Available
+
+**Current Behavior** (workflow-service.ts:806-812):
+```typescript
+const result = JSON.parse(stdout) as ClaudeCliResult;
+exec.sessionId = result.session_id;  // Session ID is in JSON output!
+```
+
+The session ID is immediately available in the CLI's JSON output. We don't need to poll `sessions-index.json` at all - the JSON output already contains it. The current implementation:
+1. Starts polling `sessions-index.json` (unreliable, race conditions)
+2. ALSO parses session_id from stdout (reliable, immediate)
+
+The fix is to **rely solely on the JSON output** and remove the polling approach.
+
+### Constraints Discovered
+
+- **Session ID Timing**: Session ID available AFTER first CLI response (not at spawn time)
+- **Workflow Directory**: Current location `~/.specflow/workflows/` is global, not project-scoped
+- **No Migration Needed**: User confirmed we can delete old workflows and start fresh
+- **Constitution VIII**: Operational state (`.specflow/`) vs repo knowledge (`.specify/`) separation
+
+---
+
+## Requirements Sources
+
+### From ROADMAP/Phase File
+
+1. **Architectural Unification**: Workflow = Session = Claude conversation (same concept)
+2. **Single Source of Truth**: Store workflow metadata in project: `.specflow/workflows/{session_id}/`
+3. **Immediate Session Detection**: Capture session ID when CLI returns (not via polling)
+4. **Workflow/Session History**: List all workflow/sessions for a project
+5. **Session Viewer Integration**: Update to use unified model
+
+### From Previous Phase (1052)
+
+Phase 1052 implemented Session Viewer UI that provides the viewing infrastructure. This phase extends it with proper session linking.
+
+### From Memory Documents
+
+- **Constitution VIII**: `.specflow/` for operational state, `.specify/` for repo knowledge
+- **Constitution III**: CLI over direct edits - but workflow state is dashboard internal
+- **Tech Stack**: Next.js, TypeScript, file-based persistence
+
+---
+
+## Scope Clarification
+
+### Questions Asked
+
+#### Question 1: Migration Strategy
+
+**Context**: Existing workflows stored at `~/.specflow/workflows/{execution_id}.json`
+
+**Question**: Should we migrate existing workflow data?
+
+**Options Presented**:
+- A (Recommended): Migrate existing workflows to session-ID-keyed directories
+- B: Start fresh, old workflows become orphaned
+
+**User Answer**: Delete old workflows and start fresh
+
+**Research Done**: Confirmed this simplifies implementation - no migration logic needed
+
+---
+
+#### Question 2: Session Detection Method
+
+**Context**: Phase file mentions several options including `firstPrompt` matching
+
+**Question**: How should we identify our sessions?
+
+**Options Presented**:
+- A (Recommended): Execution ID prefix in first prompt
+- B: Skill signature match
+- C: Timestamp + project (current approach)
+
+**User Answer**: User asked to research vibe-kanban for alternative solutions
+
+**Research Done**:
+- Searched vibe-kanban GitHub repo - no specific session detection code found
+- Found [Claude Agent SDK documentation](https://platform.claude.com/docs/en/agent-sdk/sessions) showing session ID is returned in initial `system` message with `subtype === 'init'`
+- Found [GitHub issue #1335](https://github.com/anthropics/claude-code/issues/1335) confirming JSON output contains `session_id` immediately
+- **Key Finding**: Our current code ALREADY parses `session_id` from CLI output - we just need to remove the unreliable polling fallback
+
+**Conclusion**: No workaround needed. The CLI's `--output-format json` response includes `session_id` directly. Remove the `findNewSession()` polling and rely solely on parsing the JSON output.
+
+---
+
+### Confirmed Understanding
+
+**What the user wants to achieve**:
+- Unify workflow and session concepts so they're treated as the same thing
+- Fix race conditions in session detection
+- Enable viewing any session's history from the dashboard
+- Store workflow metadata per-project in `.specflow/workflows/{session_id}/`
+
+**How it relates to existing code**:
+- Refactor `workflow-service.ts` to remove `findNewSession()` polling
+- Session ID comes directly from CLI JSON output (already being parsed)
+- Move storage from global `~/.specflow/workflows/` to project-local `.specflow/workflows/`
+- Update Session Viewer to properly link to clicked session
+
+**Key constraints and requirements**:
+- Delete old workflows, no migration
+- Session ID available only after first CLI response completes
+- Maintain Constitution VIII separation (`.specflow/` for operational state)
+
+**Technical approach**:
+- Use `session_id` from CLI JSON output (no polling)
+- Store workflow metadata at `{project}/.specflow/workflows/{session_id}/metadata.json`
+- Link to Claude's JSONL at `~/.claude/projects/{hash}/{session_id}.jsonl` (don't copy)
+- Build index file for quick lookup: `.specflow/workflows/index.json`
+
+**User confirmed**: Yes - 2026-01-19
+
+---
+
+## Recommendations for SPECIFY
+
+### Should Include in Spec
+
+- Session ID comes from CLI JSON output immediately (not polling)
+- Workflow storage moves to `{project}/.specflow/workflows/{session_id}/`
+- Workflow index at `.specflow/workflows/index.json` for quick listing
+- Session Viewer correctly links to clicked session
+- Session history list in project detail
+- Resume capability for any past session
+
+### Should Exclude from Spec (Non-Goals)
+
+- Full session replay/playback
+- Session comparison
+- Export/archive sessions
+- Session search
+- Migration of existing workflow data
+
+### Potential Risks
+
+- Session ID not available until first CLI response completes (~5-30s)
+- UI needs graceful handling of "session pending" state
+- Edge cases: CLI crashes before returning, network issues
+
+### Questions to Address in CLARIFY
+
+- None - scope is clear from phase file and user clarification