Add golden file tests for Markdown formatter variants

[gregpriday]

Summary

The Markdown formatter supports multiple output variants (line numbers, only-tree, git status) but lacks regression tests using golden files to verify stable output across changes.

Problem Statement

The src/services/formatters/MarkdownFormatter.js implements various output modes and flags, but without golden file tests we cannot:

• Verify that output format remains consistent after code changes
• Catch unintended formatting regressions
• Validate that fence language detection works correctly
• Ensure begin/end markers are properly placed
• Test interaction between multiple flags

This affects users who rely on consistent Markdown output for documentation generation, AI tool integration, and automated workflows.

Current State

Formatter Capabilities (Implemented but Untested):

• Base Markdown formatting with code fences
• --with-line-numbers - Line-numbered content
• --only-tree - Tree structure without file contents
• --with-git-status - Git status annotations
• Language detection for syntax highlighting
• Begin/end markers for file boundaries

Existing Test Coverage:

• Unit tests may exist for individual methods
• No golden file tests for integrated output
• No tests for flag combinations

Affected Components

• src/services/formatters/MarkdownFormatter.js - Formatter implementation
• src/ui/components/CopyView.js - Format mapping and options
• tests/fixtures/goldens/ - Will store expected outputs
• tests/helpers/determinism.js - Normalization utilities
• tests/helpers/fixtures.js - Golden file matcher

Implementation Approach

1. Create golden files in tests/fixtures/goldens/markdown/:

   • base.md - Default markdown output
   • with-line-numbers.md - With line numbers
   • only-tree.md - Tree structure only
   • with-git-status.md - With git annotations
   • combined-flags.md - Multiple flags together

2. Create test file tests/e2e/markdown-formatter.test.js:

   • Test each variant against golden files
   • Use normalizeForGolden() for deterministic comparison
   • Test flag combinations
   • Verify fence language detection

3. Follow existing golden file workflow:

   • Initial generation with UPDATE_GOLDEN=true
   • Subsequent runs compare against baselines
   • Git tracks changes to golden files

Tasks

• Create tests/fixtures/goldens/markdown/ directory
• Create tests/e2e/markdown-formatter.test.js with variant tests
• Add test for base Markdown output (no flags)
• Add test for --with-line-numbers output
• Add test for --only-tree output
• Add test for --with-git-status output
• Add test for combined flags (e.g., line numbers + git status)
• Generate initial golden files with UPDATE_GOLDEN=true npm run test:e2e
• Verify fence language detection for various file types (.js, .py, .md, etc.)
• Verify begin/end markers are correctly placed
• Ensure tests pass on CI across different platforms

Acceptance Criteria

• At least 6 golden file tests for Markdown formatter variants
• Tests pass consistently without UPDATE_GOLDEN flag
• Golden files are committed and tracked in git
• Tests fail when formatter output changes
• Language detection verified for common file extensions
• All flag combinations produce deterministic output
• README examples match actual output from golden files
• No test flakiness or platform-specific failures

Additional Context

This issue is a subset of #19 (E2E golden tests) but focuses specifically on the Markdown formatter, which is commonly used for AI tool integration and documentation generation.

README Examples to Verify:

• Common Recipes - Line 98-99 shows markdown streaming
• Quick Start - Shows basic markdown usage

Related Testing Infrastructure:

• Golden File Testing Guide
• Determinism Helpers

[gregpriday]

Goal

Pin the MarkdownFormatter output contract with golden files and enforce deterministic, regression-catching tests across all supported modes (base, withLineNumbers, onlyTree, withGitStatus, and combinations). Pass when current behavior is stably reproduced (no UPDATE_GOLDEN) and performance/observability SLOs are met; fail on any unintended format drift.

Approach Overview

1. Define the formatter surface and option contract from src/services/formatters/MarkdownFormatter.js and the call path in src/ui/components/CopyView.js.
2. Establish deterministic normalization utilities in tests/helpers/determinism.js and golden matcher utilities in tests/helpers/fixtures.js.
3. Create canonical input fixtures and golden outputs under tests/fixtures/goldens/, with a markdown/ subfolder for each variant.
4. Add an end-to-end test driver tests/e2e/markdown-formatter.test.js that renders each variant and asserts against goldens using normalizeForGolden().
5. Cover flag precedence and interactions (e.g., onlyTree vs withLineNumbers, withGitStatus overlays).
6. Validate fence language detection for representative extensions; assert presence/placement of file begin/end markers.
7. Wire CI matrix (Linux/macOS/Windows) to run golden tests and track perf metrics; document update workflow.

Step-by-Step Plan

1. Document the public contract

   • Action: In a header comment of MarkdownFormatter and a short doc (docs/format/markdown.md), enumerate supported options and invariants. Options surface (names must match existing code): withLineNumbers, onlyTree, withGitStatus, detectLanguage (if present).

   • Invariants to record:

     • Stable path ordering (lexicographic by POSIX path).
     • Begin/end markers always emitted around file contents in non-onlyTree modes.
     • Fence language derives from file extension; unknown ⇒ fallback (current behavior).
     • Git status decorations are stable per path and appear in both onlyTree and full modes (current behavior).

   • Outcome: A frozen “as-is” contract that tests will pin.

2. Determinism normalization

   • Action: In tests/helpers/determinism.js, ensure normalizeForGolden() handles:

     • Newlines → \n; path separators → /.
     • Collapse inconsistent whitespace in trailing spaces; strip absolute paths, timestamps, hostnames.
     • Normalize git porcelain output (e.g., map R100/C100 to canonical tokens if emitted).

   • Outcome: Cross-platform stability; no CRLF or platform noise.

3. Fixtures and goldens layout

   • Action: Under tests/fixtures/goldens/, add:

     • markdown/base.md, markdown/with-line-numbers.md, markdown/only-tree.md, markdown/with-git-status.md, markdown/combined-flags.md, markdown/language-detection.md.

   • Input fixture tree (lightweight) co-located or loaded via tests/helpers/fixtures.js using loadFixtureTree('markdown').

   • Outcome: Canonical inputs + expected outputs committed.

4. Inject/make deterministic Git and language sources

   • Action: If MarkdownFormatter pulls git via a shell, provide a gitStatusProvider in options for tests; default to real provider in prod. If language detection is internal (e.g., detectFenceLanguage()), keep as-is but include enough representative files in the fixture to exercise it.
   • Outcome: E2E tests do not require a real Git repo and are deterministic.

5. Golden test driver

   • Action: Add tests/e2e/markdown-formatter.test.js that:

     • Renders each variant by invoking MarkdownFormatter.format(tree, options) through the same API used by CopyView to avoid divergence.
     • Uses normalizeForGolden() then expect(...).toMatchGolden('markdown/<file>.md').
     • Asserts begin/end markers exist and nest correctly when multiple files are concatenated.
     • Asserts fence language tokens for .js, .ts, .py, .md, unknown extension.

   • Outcome: Failing tests on any user-visible format drift.

   Small example (kept <20 lines for clarity):

   test('with line numbers', async () => {
     const tree = await loadFixtureTree('markdown');
     const out = MarkdownFormatter.format(tree, { withLineNumbers: true });
     const norm = normalizeForGolden(out);
     expect(norm).toMatchGolden('markdown/with-line-numbers.md');
     expect(norm).toContain('```'); // fence present
     expect(norm).toMatch(/^\s*1\:/m); // numbered first line
   });

6. Flag precedence & combinations

   • Action: Add explicit tests for combined flags (e.g., withLineNumbers + withGitStatus, onlyTree + withGitStatus). Record current precedence (e.g., if onlyTree suppresses contents, line numbers should be absent; git annotations should still appear on the tree).
   • Outcome: Interactions are pinned and documented.

7. README parity test

   • Action: Parse fenced blocks from README.md “Quick Start” and “Common Recipes” sections, generate current formatter output for the same input, and assert equality. If examples are illustrative rather than exact, snapshot separately and allow opt-out via a marker comment.
   • Outcome: Examples never drift from implementation.

8. Performance & observability hooks

   • Action: Time MarkdownFormatter.format() for the fixture and emit a metric (console or lightweight counter) captured in tests: metric: formatter.markdown.p95_ms. Add log keys formatter.mode, formatter.files_total, formatter.flags when running tests for easier debugging.
   • Outcome: Basic performance/SLO coverage and diagnosability without external infra.

Tests

• Unit

  • Language detection mapping (extensions → fence tags) within MarkdownFormatter or helper: cases for .js, .jsx, .ts, .py, .md, no extension, unknown.
  • Marker emitter: ensure BEGIN/END markers are always paired and include canonical path.

• Integration

  • CopyView → formatter option wiring: render once via CopyView code path to ensure flags are propagated identically to direct calls.

• E2E (Golden)

  • Files: tests/e2e/markdown-formatter.test.js

  • Cases:

    • base.md (no flags).
    • with-line-numbers.md.
    • only-tree.md.
    • with-git-status.md (using stubbed provider).
    • combined-flags.md (line numbers + git status).
    • language-detection.md (multiple files; assert fence tags).
    • README parity (skips if example is marked illustrative).

  • Commands:

    • Generate: UPDATE_GOLDEN=true npm run test:e2e
    • Verify (CI/default): npm run test:e2e
    • Optional perf: npm run test:e2e -- --reporter=dot (ensure timing is printed and parsed).

Edge Cases & Risks

• Paths with spaces/backticks/Unicode; ensure markers and fences remain valid and normalization preserves code points.
• Mixed EOLs (CRLF vs LF); normalized in determinism helper.
• Very long lines causing excessive line number width; ensure alignment logic is stable and tested.
• Files containing triple backticks; formatter must escape or switch fence token (current behavior is pinned—add such a file to fixture).
• Git rename/copy statuses; if surfaced, normalize tokenization to avoid platform git differences or test via stub.
• Non-deterministic inputs (timestamps, absolute paths) leaking into output; must be stripped in normalizeForGolden().

Rollout & Monitoring

• Flag/Toggles

  • Introduce env MARKDOWN_FORMAT_LOCK=1 in CI to fail on any golden drift.
  • Keep UPDATE_GOLDEN for intentional changes only; require PR note in docs/format/markdown.md.

• Telemetry

  • Emit formatter.markdown.p95_ms, formatter.markdown.bytes_out, formatter.markdown.files_total, and formatter.markdown.mode.
  • Log keys: formatter.flags, formatter.version (string), git.provider (real|stub).

• Rollback

  • If tests catch unintended drift: revert the last commit that modified formatter or goldens, or temporarily set MARKDOWN_FORMAT_LOCK=0 to unblock CI while investigating.

• Abort Criterion

  • If >20% of golden files change in one PR without a corresponding doc update, or p95 exceeds SLO by >25% for two consecutive CI runs, block merge and open a follow-up bug.

Acceptance Checklist

• Golden files added for at least 6 variants in tests/fixtures/goldens/; CI runs without UPDATE_GOLDEN.
• Tests assert begin/end markers, fence tags, flag precedence, and README parity.
• Determinism: no CRLF or path separator diffs across CI OSes.
• SLO: formatter.markdown.p95_ms ≤ 300ms for the canonical fixture (≥200 files, ~1MB total). Alert if exceeded in CI logs.
• Observability: metrics/log keys documented in docs/format/markdown.md and summarized in docs/observability/.
• Update workflow documented: how to intentionally refresh goldens and justify changes in PR description (“Markdown format change” section).

Naming note: Keep option names consistent and positive/neutral (withLineNumbers, withGitStatus, onlyTree); metric keys dot-separated lowercase (formatter.markdown.*); event/log keys snake_case (formatter_mode).