Skip to content

feat(ask): Add failure detection with actionable guidance #6

@hummat

Description

@hummat

Problem or Motivation

papi ask currently has basic output hygiene (--pqa-raw, -v/--verbose) but lacks proper failure detection. When PaperQA2 crashes, hangs, or encounters bad PDFs, users get unclear errors or silent failures.

Current state:

  • --pqa-raw flag for full passthrough
  • -v/--verbose enables raw output
  • Default behavior suppresses PaperQA2 streaming logs

Missing:

  • Crash loop detection
  • Bad PDF identification
  • Actionable error guidance

Proposed Solution

  1. Detect common failure modes:

    • PaperQA2 crashes (traceback parsing)
    • Hanging/timeout (no progress for N seconds)
    • Bad PDF errors (corrupted, encrypted, scan-only)
  2. Provide actionable guidance:

    • "Paper X has a corrupted PDF. Try: papi attach X --pdf /path/to/fixed.pdf"
    • "PaperQA2 timed out. Try: papi ask --pqa-llm <faster-model> or add --verbose to debug"
    • "Install missing dependency: pip install paperqa[...]"
  3. Exit codes:

    • 0: success
    • 1: query failed but recoverable (bad paper, suggest fix)
    • 2: unrecoverable error (missing deps, bad config)

Alternatives Considered

  • Rely on --pqa-raw for debugging — requires manual log parsing, not user-friendly
  • Wrapper script — moves complexity outside papi, harder to maintain

Area

RAG backends (PaperQA2/LEANN)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions