Skip to content

[plan][agent.command]: Add dry-run mode for ultra-planner and issue-to-impl commands #601

@were

Description

@were

Description

Add a --dry-run flag to the /ultra-planner and /issue-to-impl commands so users can preview what actions will be taken before executing them. This provides a safety mechanism for critical workflow commands that currently execute actions (creating issues, branches, commits, PRs) without a preview option.

Source: Issue #598, P1.2 - Dry-Run Mode for Commands

Related modules:

  • .claude-plugin/commands/ultra-planner.md
  • .claude-plugin/commands/issue-to-impl.md
  • .claude-plugin/lib/workflow.py
  • .claude-plugin/prompts/ultra-planner.txt
  • .claude-plugin/prompts/issue-to-impl.txt

Proposed Solution

Consensus Summary

Implement a command-level --dry-run that prints a plain preview and skips side-effect steps, avoiding new simulation layers or skill changes. Keep ultra-planner's debate/external-consensus flow so plan content remains real, but suppress GitHub issue creation/updates and labels, with explicit cost expectations.

Critical: Handsoff Mode Handling

The continuation prompts in .claude-plugin/prompts/ define completion conditions for the stop hook:

  • ultra-planner.txt: "create a comprehensive plan and post it on GitHub Issue"
  • issue-to-impl.txt: "deliver a PR on GitHub"

In dry-run mode, these conditions will never be met (no Issue/PR created), causing the session to run through all max_continuations without recognizing completion.

Solution: Update continuation prompts to recognize dry-run mode completion:

  1. Add {#dry_run#} variable to workflow.py's _fmt_prompt() function
  2. Update prompts to check: "If dry-run mode, completion = dry-run summary printed (not Issue/PR creation)"
  3. Alternatively: Dry-run commands should NOT enter handsoff continuation at all (single-shot execution)

Goal

Provide a safe preview for /ultra-planner and /issue-to-impl so users can see intended actions without creating issues, branches, commits, or PRs, while preserving existing workflow logic and outputs.

Success criteria:

  • /ultra-planner --dry-run <feature> produces a consensus plan and a clear "no issue/label changes" summary without calling open-issue or label edits.
  • /issue-to-impl <N> --dry-run prints a preview of branch/sync/commit/PR actions plus planned files, without modifying the repo or writing milestone/cache files.
  • Dry-run mode correctly terminates in handsoff mode (does not run through max_continuations looking for non-existent Issue/PR).
  • Documentation reflects the new flag in command summaries and tutorials.

Out of scope:

  • Terraform-style renderers, action checkpoint registries, or skill-level dry-run propagation.
  • A "preview" mode that skips multi-agent execution entirely.

Future work:

  • Add a separate --preview mode for /ultra-planner that skips the debate/external-consensus to reduce token cost when users only want a high-level sketch.

Codebase Analysis

File changes:

File Level Purpose
.claude-plugin/commands/ultra-planner.md medium Add --dry-run parsing, skip issue/label steps, print preview summary
.claude-plugin/commands/issue-to-impl.md medium Add --dry-run parsing, skip branch/sync/commit/milestone/PR steps, print preview summary
.claude-plugin/lib/workflow.py medium Add {#dry_run#} variable to _fmt_prompt(), update issue extraction regex
.claude-plugin/prompts/ultra-planner.txt medium Add dry-run completion condition (dry-run summary = complete, not Issue creation)
.claude-plugin/prompts/issue-to-impl.txt medium Add dry-run completion condition (dry-run summary = complete, not PR creation)
tests/cli/test-workflow-module.sh minor Add extract_issue_no cases for --dry-run variants
docs/feat/core/ultra-planner.md medium Document --dry-run usage and behavior
docs/tutorial/01-ultra-planner.md medium Add dry-run usage and cost note
docs/tutorial/02-issue-to-impl.md medium Add dry-run usage and preview behavior
docs/feat/core/issue-to-impl.md minor Add dry-run behavior note
docs/commands.md minor Note --dry-run support in command list

Interface Design

New interfaces:

  • /ultra-planner --dry-run <feature>: Runs debate/consensus to produce a plan but skips placeholder issue creation, issue updates, and label edits; prints a dry-run summary.
  • /issue-to-impl <issue> --dry-run: Reads the issue plan to list intended file changes, then skips branch/sync/commit/milestone/PR steps and prints a dry-run summary.

Modified interfaces:

- argument-hint: [issue-number]
+ argument-hint: [issue-number] [--dry-run]
- argument-hint: [feature-description] or --force-full [feature-description] or --refine [issue-no] [refine-comments] or --from-issue [issue-no]
+ argument-hint: [feature-description] [--dry-run] or --force-full [feature-description] [--dry-run] or --refine [issue-no] [refine-comments] [--dry-run] or --from-issue [issue-no] [--dry-run]

Continuation Prompt Updates (Critical for Handsoff Mode)

Problem: Current prompts check for Issue/PR creation as completion signal. Dry-run never creates these, causing infinite continuation loops.

Solution: Add dry-run awareness to prompts:

ultra-planner.txt changes:

  The ultimate goal of this workflow is to create a comprehensive plan and post it on GitHub Issue. Have you delivered this?
+ (Note: If running in dry-run mode, completion = printing the dry-run summary, not creating an Issue)
  1. If not, please continue!
  2. If you have already delivered the plan, manually stop further continuations.
+    - In dry-run mode: If you have printed the dry-run summary showing what would be created, stop here.

issue-to-impl.txt changes:

  The ultimate goal of this workflow is to deliver a PR on GitHub that implements the corresponding issue.
+ (Note: If running in dry-run mode, completion = printing the dry-run preview, not creating a PR)
  ...
  4. If the PR is successfully created, manually stop further continuations.
+    - In dry-run mode: If you have printed the dry-run preview showing what would be done, stop here.

workflow.py changes:

# Add dry_run parameter to get_continuation_prompt()
def get_continuation_prompt(workflow, session_id, fname, count, max_count, 
                            pr_no='unknown', transcript_path=None, 
                            plan_path=None, plan_excerpt=None,
                            dry_run=False):  # NEW PARAMETER
    ...
    
# Add to _fmt_prompt()
def _fmt_prompt(template, ..., dry_run=False):
    return (template
            ...
            .replace('{#dry_run#}', 'true' if dry_run else 'false'))

Test Strategy

Test modifications:

  • tests/cli/test-workflow-module.sh — add extract_issue_no cases for /issue-to-impl 42 --dry-run and /issue-to-impl --dry-run 42.

Implementation Steps

Step 1: Update documentation for dry-run usage (Estimated: 80 LOC)
Files: docs/feat/core/ultra-planner.md, docs/tutorial/01-ultra-planner.md, docs/tutorial/02-issue-to-impl.md, docs/feat/core/issue-to-impl.md, docs/commands.md.
Changes: Add --dry-run usage and behavior notes; clarify token-cost expectations for ultra-planner dry-run; annotate command list.

Step 2: Add workflow parsing tests (Estimated: 10 LOC)
Files: tests/cli/test-workflow-module.sh.
Changes: Add two extract_issue_no tests for --dry-run before/after the issue number.

Step 3: Update workflow.py for dry-run support (Estimated: 20 LOC)
Files: .claude-plugin/lib/workflow.py.
Changes:

  • Allow extract_issue_no to match /issue-to-impl --dry-run <N> and /issue-to-impl <N> --dry-run
  • Add dry_run parameter to get_continuation_prompt() and _fmt_prompt()
  • Add {#dry_run#} placeholder substitution

Step 4: Update continuation prompts for dry-run completion (Estimated: 30 LOC)
Files: .claude-plugin/prompts/ultra-planner.txt, .claude-plugin/prompts/issue-to-impl.txt.
Changes: Add dry-run mode completion conditions so stop hook recognizes dry-run summary as "done" (not Issue/PR creation).

Step 5: Add dry-run logic to command workflows (Estimated: 80 LOC)
Files: .claude-plugin/commands/issue-to-impl.md, .claude-plugin/commands/ultra-planner.md.
Changes: Parse --dry-run, remove it from args, and add explicit dry-run branches that skip side-effect steps.

Total estimated complexity: ~220 LOC (Large)
Recommended approach: Single session (docs → tests → workflow.py → prompts → commands).

Success Criteria

  • /ultra-planner --dry-run produces a plan and explicit "no GitHub changes" summary.
  • /issue-to-impl <N> --dry-run produces a preview with planned file lists and no repo changes.
  • Dry-run in handsoff mode terminates correctly after printing summary (does not loop through max_continuations).
  • Docs and tutorials reflect the new flag and its limitations.

Risks and Mitigations

Risk Likelihood Impact Mitigation
Users expect dry-run to reduce token cost for ultra-planner M M Document that agents still run; add explicit output note in dry-run summary
Issue number extraction fails with new flag placement M M Update regex and add tests for both flag positions
Side-effect steps accidentally executed during dry-run L H Add explicit dry-run guards before each side-effect step and early exit with summary
Dry-run loops through max_continuations in handsoff mode H H Update prompts to recognize dry-run summary as completion signal
Preview drift vs actual run due to external state changes M M Note best-effort preview in docs and output summary

Related PR

TBD - will be updated when PR is created

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentize:planPlan created by /ultra-planner command

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions