Skip to content

Primitives-first AI subtitle post-processor: add MuxDataAdapter, debug artifacts, and workflow updates#8688

Open
lumberman wants to merge 1 commit into01-01-FE-feat-ai-subtitlesfrom
cursor/vm-26-implement-ai-subtitle-post-processor-tasks
Open

Primitives-first AI subtitle post-processor: add MuxDataAdapter, debug artifacts, and workflow updates#8688
lumberman wants to merge 1 commit into01-01-FE-feat-ai-subtitlesfrom
cursor/vm-26-implement-ai-subtitle-post-processor-tasks

Conversation

@lumberman
Copy link
Contributor

Motivation

  • Centralize Mux transcript/storyboard I/O behind @mux/ai/primitives to avoid bespoke fetch logic and reduce integration drift.
  • Make the validator the strict gatekeeper and ensure a deterministic fallback always produces valid WebVTT.
  • Persist debug artifacts (original transcript, AI outputs, validator report, optional storyboard) to aid QA and incident triage.
  • Wire primitives into the Workflow DevKit pipeline so transcript acquisition is a discrete, testable step.

Description

  • Added a primitives-first adapter MuxDataAdapter (apis/api-media/src/lib/aiSubtitles/muxDataAdapter.ts) that uses @mux/ai/primitives (fetchTranscriptForAsset, parseVTTCues, getStoryboardUrl, buildTranscriptUrl) to return normalized Segment[], raw VTT and storyboard/playback info.
  • Implemented a standalone aiSubtitles library (types, language parsing, profiles, normalize/counting, validator, WebVTT parse/serialize, deterministic fallback, AI call helper, processor orchestration) under apis/api-media/src/lib/aiSubtitles/* and exported via index.ts.
  • Updated Workflow DevKit pieces to be primitives-first: added fetch-transcript step and updated workflow bundle/steps/worker to call getTranscriptSegments, pass raw transcript into the processor, persist validation debug artifacts to R2 when failures occur, and use primitives-built VTT URLs when tracks are ready (apis/api-media/src/workers/aiSubtitlePostProcessor/*).
  • Added deterministic golden fixtures and unit tests for the subtitle logic under apis/api-media/src/lib/aiSubtitles/__tests__ to assert fallback determinism and retry-before-fallback AI flow.
  • Extended post-processing types and processor to surface debug artifacts and a PostProcessError carrying artifacts for downstream persistence.
  • Updated PRD and learnings (prds/watch/ai-subtitle-post-processor.md, ...-learnings.md) to require primitives-only Mux reads, mandate a MuxDataAdapter component, and require debug outputs on validation failure.

Testing

  • Ran linter: pnpm nx lint api-media — executed but failed due to pre-existing repository lint errors unrelated to these changes (import/order and other repo-wide issues).
  • Ran package tests scoped to the feature: pnpm nx test api-media --testPathPattern aiSubtitles — the new aiSubtitles test files are present, but the test run was blocked by existing repository test-harness issues (prismaMock/PrismaClient export error) so the suite could not complete.
  • Added deterministic golden fixtures (English/Arabic/Japanese/mixed/non-speech) and unit tests exercising fallback formatting and AI retry semantics; these tests are included in the PR but could not be executed to completion due to the global test configuration issues above.

If you want, I can next (1) run an isolated Jest invocation directly against the new __tests__ folder to demonstrate the aiSubtitles logic in isolation, or (2) help remediate the unrelated lint/test-harness issues so repo-wide lint and test complete end-to-end.


Codex Task

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 4, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • 🔍 Trigger a full review
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch cursor/vm-26-implement-ai-subtitle-post-processor-tasks

Important

Action Needed: IP Allowlist Update

If your organization protects your Git platform with IP whitelisting, please add the new CodeRabbit IP address to your allowlist:

  • 136.113.208.247/32 (new)
  • 34.170.211.100/32
  • 35.222.179.152/32

Reviews will stop working after February 8, 2026 if the new IP is not added to your allowlist.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

Fails
🚫 Please ensure your PR title matches commitlint convention.
🚫 Please assign someone to merge this PR.
🚫 Please request a reviewer for this PR.
Warnings
⚠️ ❗ Big PR (2048 changes)

(change count - 2048): Pull Request size seems relatively large. If Pull Request contains multiple changes, split each into separate PR will helps faster, easier review.

(pr title - Primitives-first AI subtitle post-processor: add MuxDataAdapter, debug artifacts, and workflow updates):

subject may not be empty

type may not be empty

header must not be longer than 100 characters, current length is 110

Generated by 🚫 dangerJS against 9f3df82

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

Fails
🚫 Please ensure your PR title matches commitlint convention.
🚫 Please assign someone to merge this PR.
🚫 Please request a reviewer for this PR.
Warnings
⚠️ ❗ Big PR (2048 changes)

(change count - 2048): Pull Request size seems relatively large. If Pull Request contains multiple changes, split each into separate PR will helps faster, easier review.

(pr title - Primitives-first AI subtitle post-processor: add MuxDataAdapter, debug artifacts, and workflow updates):

subject may not be empty

type may not be empty

header must not be longer than 100 characters, current length is 110

Generated by 🚫 dangerJS against 9f3df82

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant