RFC: Model and Agent Selection by Reasoning Level

[beettlle]

Summary

We propose extending RoboRev so that different agents (and optionally different models within an agent) can be used for each reasoning level (fast, standard, thorough). For example: thorough with an online “thinking” or powerful model (e.g. Claude Code), standard with a fast online or local agent (e.g. Codex), fast with a local or lightweight agent (e.g. Codex + Ollama, or a future Ollama agent). The change is backward-compatible: if no overrides are set, behavior stays as today. No database or API schema changes are required for Phase 1.

---

Motivation

RoboRev currently picks one agent per job from config or --agent. The reasoning level (fast / standard / thorough) only adjusts parameters for that agent (e.g. Codex’s model_reasoning_effort, Droid’s --reasoning-effort). It does not choose a different backend or model.

Users have asked for:

• Thorough: Online “thinking” or high-capability models for deep analysis.
• Standard: Fast online models or capable local models for balanced cost/speed.
• Fast: Local or lightweight models (e.g. Ollama, OpenCode) for quick, low-cost, or offline reviews.

Supporting this requires choosing the agent (and, optionally, the model inside an agent) based on the reasoning level. “Different model per reasoning” can be implemented as a different agent per reasoning, or later as an explicit model override when the CLI supports it (e.g. Codex -m).

---

Current Behavior

• Enqueue: In internal/daemon/server.go, handleEnqueue calls ResolveAgent(req.Agent, repoRoot, globalCfg) and ResolveReviewReasoning(req.Reasoning, repoRoot). The job is stored with (agent, reasoning).
• Worker: In internal/daemon/worker.go, GetAvailable(job.Agent) then WithReasoning(level).WithAgentic(agentic) and Review(...). Reasoning only affects how Codex/Droid are invoked (e.g. -c model_reasoning_effort=high|low in internal/agent/codex.go); Claude, Gemini, Copilot, OpenCode ignore WithReasoning.
• Refine: In cmd/roborev/refine.go, ResolveAgent, ResolveRefineReasoning, then selectRefineAgent with WithReasoning. One agent for the run; reasoning only changes effort where supported.

---

Proposal

The feature is split into two phases. Phase 1 (agent per reasoning) covers “local for fast, cloud for thorough” and needs no changes to the Agent interface, worker, or storage. Phase 2 (model per reasoning) adds “same agent, different model” (e.g. Codex with a large model for thorough and an Ollama model for fast) via a small, backward-compatible extension.

Phase 1: Agent per reasoning

For each reasoning level, an optional override can specify which agent to use. If set (repo or global), the job uses that agent; otherwise it uses the normally resolved agent. The worker already runs job.Agent, so no worker changes.

Where it applies:

• Enqueue: In internal/daemon/server.go handleEnqueue, after ResolveAgent and ResolveReviewReasoning, a new reasoning-aware resolution step would pick the agent. The resolved agentName is stored in the job. All enqueue paths (commit, range, dirty, run, prompt) go through handleEnqueue, so they are covered.
• Refine: In cmd/roborev/refine.go, after ResolveAgent and ResolveRefineReasoning, a similar reasoning-aware resolution before selectRefineAgent. Worker and refine agent invocation are unchanged.
• DB and API: No change. review_jobs.agent continues to store the resolved agent name.

Explicit --agent: When the user passes --agent, we ignore per-reasoning overrides (agent_fast / agent_standard / agent_thorough, or review_agent_* / refine_agent_* under Option A) and use only ResolveAgent(explicit, repoPath, globalCfg). That keeps --agent predictable: one agent for the whole run.

Override agent missing: If the override names an agent that is not installed, we store the override in the job. In the worker, agent.GetAvailable(job.Agent) already falls back to another installed agent. We rely on that existing behavior; no new reasoning-aware fallback in Phase 1.

Phase 2: Model per reasoning (optional)

When the same agent is used (e.g. Codex for both fast and thorough), an optional model override per reasoning level can select a different model (e.g. gpt-5.1-codex-max for thorough, an Ollama model for fast). Codex supports -m and -c model="..."; other CLIs would need to expose a model flag for full support.

Agent interface: To avoid changing the core Agent interface, we would introduce an optional ModelOverridable interface:

type ModelOverridable interface {
    Agent
    WithModelOverride(model string) Agent
}

Codex would implement it (e.g. add -m or -c model="..." in buildArgs when non-empty). Other agents would not implement it and would ignore model overrides. In the worker and refine, after WithReasoning/WithAgentic, if the resolved model is non-empty and the agent implements ModelOverridable, we would call WithModelOverride(model).

Coexistence with Phase 1: e.g. review_agent_fast = "codex" and review_model_fast = "ollama/llama3" so fast reviews use Codex with a local model override.

---

Config Reference

All new keys are optional in ~/.roborev/config.toml and .roborev.toml. Precedence: repo overrides global; unset means “use base agent” (Phase 1) or “agent default model” (Phase 2).

Phase 1 — Agent per reasoning

Option A: Six keys (review and refine can differ)

Key	Scope	Meaning
review_agent_fast	Repo, global	Agent for review/run when reasoning=fast
review_agent_standard	Repo, global	Agent when reasoning=standard
review_agent_thorough	Repo, global	Agent when reasoning=thorough
refine_agent_fast	Repo, global	Agent for refine when reasoning=fast
refine_agent_standard	Repo, global	Agent when reasoning=standard
refine_agent_thorough	Repo, global	Agent when reasoning=thorough

Option B: Three keys (shared by review and refine)

Key	Scope	Meaning
agent_fast	Repo, global	Agent when reasoning=fast (review and refine)
agent_standard	Repo, global	Agent when reasoning=standard
agent_thorough	Repo, global	Agent when reasoning=thorough

agent / default_agent stay the base when a level has no override. refine_agent_* can be added later if refine-specific overrides are needed.

Example (Option A):

review_agent_fast     = "codex"
review_agent_standard = "codex"
review_agent_thorough = "claude-code"

refine_agent_fast     = "codex"
refine_agent_standard = "codex"
refine_agent_thorough = "claude-code"

Example (Option B):

agent_fast     = "opencode"
agent_standard = "codex"
agent_thorough = "claude-code"

Phase 2 — Model per reasoning

Key	Scope	Meaning
review_model_fast	Repo, global	Model override for review when reasoning=fast (agent-specific ID, e.g. gpt-5.1-codex-mini, codellama)
review_model_standard	Repo, global	Model when reasoning=standard
review_model_thorough	Repo, global	Model when reasoning=thorough
refine_model_fast	Repo, global	Model for refine when reasoning=fast
refine_model_standard	Repo, global	Model when reasoning=standard
refine_model_thorough	Repo, global	Model when reasoning=thorough

Empty or unset = use the agent’s default model. Model IDs are agent-specific (Codex, Claude, Ollama, etc.).

---

Key Considerations

1. Agent and CLI capabilities: Codex supports -m/--model and -c model="..."; we already pass -c model_reasoning_effort="high"|"low" in internal/agent/codex.go. --oss and model_providers allow Ollama. Model-per-reasoning is feasible for Codex. Claude, Droid, Gemini, Copilot, and OpenCode: WithReasoning is a no-op for some; model override and extended-thinking are not wired. Model-per-reasoning would initially be Codex-only until their CLIs support model flags.
2. Explicit --agent overrides per-reasoning: When --agent is set, we ignore the new per-reasoning keys. We lean toward this for clear, predictable behavior; do you prefer that or allowing --agent to be overridden per-level?
3. Config naming — 6 vs 3 keys: Option A (6 keys) lets review and refine differ; Option B (3 keys) is simpler if they should share the same mapping. We lean toward 3 keys if you do not need refine-specific overrides; do you?
4. Phase 2 ModelOverridable: We prefer an optional ModelOverridable interface so the core Agent interface stays unchanged; agents that do not implement it simply ignore model overrides.
5. Override agent missing: We store the override and rely on GetAvailable(job.Agent) fallback in the worker. No extra “reasoning-aware” fallback in Phase 1.
6. Local (Ollama): (1) Via Codex: -m plus the user’s Codex/Ollama config (--oss or model_providers.base_url). (2) Optional native internal/agent/ollama.go calling http://localhost:11434/v1/chat/completions for a Codex-free local path.

---

Effort and Complexity

Scope	Effort	Notes
Phase 1 only	~0.5–2 days	Depends on 3 vs 6 keys, single vs two resolvers, tests, and docs. No DB, API, or Agent interface changes.
Phase 1 + Phase 2	+1–2.5 days	Model config, ResolveReviewModel/ResolveRefineModel, ModelOverridable, Codex -m/-c model=, worker and refine wiring. Model-per-reasoning Codex-first; others no-op until CLIs support it.
Native Ollama agent	+2.5–3 days	New internal/agent/ollama.go; optional, for a Codex-free local path.

---

Risks and Dependencies

1. Claude / Droid / Gemini: Model and extended-thinking are not wired today. Model-per-reasoning would be limited to Codex (and any other agent whose CLI exposes a model flag) until verified.
2. Codex -c and model: Multiple -c overrides and model together with model_reasoning_effort should be confirmed. Both -m and -c model="..." are documented in the Codex CLI.
3. CodexCmd / ClaudeCodeCmd: These config keys exist but are not wired to the agent Command; agents use built-in defaults. Independent of this feature.

---

Questions

We seek your feedback on:

• Phase 1 design — Agent per reasoning (optional overrides, explicit --agent overrides per-reasoning). Does this match what you need?
• Config naming — Six keys (review_agent_*, refine_agent_*) vs three keys (agent_fast, agent_standard, agent_thorough) shared by review and refine. Do you need refine-specific overrides?
• Phase 2 and Ollama — Should we pursue model-per-reasoning after Phase 1? Is a native Ollama agent a priority for you?

Please comment in the issue tracker or open an issue with the proposal label. After feedback, we will implement Phase 1 and re-assess Phase 2 and a native Ollama agent.

---

References

External

• Codex CLI — Overview, Models, Config Reference, Command reference (-m, -c, --oss).
• Ollama — OpenAI-compatible API at http://localhost:11434/v1.

In-repo (for implementers)

• internal/daemon/server.go — handleEnqueue, ResolveAgent, ResolveReviewReasoning
• internal/daemon/worker.go — GetAvailable, WithReasoning, Review
• internal/config/config.go — ResolveAgent, ResolveReviewReasoning, ResolveRefineReasoning
• internal/agent/agent.go — Agent, WithReasoning, GetAvailable
• internal/agent/codex.go — buildArgs, model_reasoning_effort
• cmd/roborev/refine.go — ResolveAgent, selectRefineAgent

[wesm]

I am working on a PR right now that tackles some of this! please hang tight

[beettlle]

I was just making a brain dump of what I want to do in the end. I'm still having issued with OpenCode/Ollama in my Python codebase. Trying to figure that out first.

[wesm]

No worries! Just wanted to let you know that I've been grinding on this the last couple hours.

#126 adds a "model" column to the schema allowing us to persist the selected model for each job (+ database migrations for sqlite and the shared postgres). It adds a default_model option to go with default_agent in config.toml.

I think the idea then is that after this is merged, we can create a PR that adds the granular agent and model selection for different reasoning / thoroughness levels -- agent/model selection is determined at CLI invocation time (i.e. when invoking roborev review or roborev refine) and those are stored in the database. I do not think we need any more schema changes other than the addition of a "model" column in the schema, since agent/model selection is just something that happens at the config level.

[beettlle]

One little detail that my RFC brings up is that it should also be something that is configurable, not just when it's called from the CLI, so when the daemon picks something up with a set reasoning level, it should also route to the proper model.

[wesm]

Right, whenever a review (either by the daemon or the CLI — which is where most reviews originate via the git post-commit hook) is triggered the preferences provided first by the repo-level TOML file are respected followed by the global TOML file. That request gets persisted in the reviews database -- currently only the agent and reasoning level is stored, #126 adds model storage (which is needed for opencode anyway, and would have been useful to be able to select different models in codex/claude/gemini, too) and so all that will be left will be to add the configuration options to enable routing to different agent/model combinations based on the reasoning level, so

roborev review --reasoning fast --> agent A / model A
roborev review --reasoning standard --> agent B / model B
roborev review --reasoning thorough --> agent C / model C

of course this can be further overridden by passing additional options beyond just the reasoning level