Skip to content

Add individual evaluator selection with multi-select dropdown (Vibe Kanban)#1

Merged
cteyton merged 4 commits intomainfrom
vk/4fa4-select-specific
Feb 8, 2026
Merged

Add individual evaluator selection with multi-select dropdown (Vibe Kanban)#1
cteyton merged 4 commits intomainfrom
vk/4fa4-select-specific

Conversation

@cteyton
Copy link
Contributor

@cteyton cteyton commented Feb 6, 2026

Summary

  • Replace the 3-option category filter (All / Errors Only / Suggestions Only) with a multi-select dropdown listing all 17 evaluators individually, grouped by issue type
  • Users can now pick any combination of evaluators, with at least 1 required
  • By default all evaluators are selected, preserving existing behavior
  • Cloud mode is unchanged (UI hidden, always runs all evaluators)

Why

Previously, the frontend only allowed filtering evaluators by broad category ("errors" or "suggestions"). Users had no way to run a specific subset of evaluators — for example, only "Security" and "Outdated Documentation" — without running the entire category. This change gives fine-grained control over which evaluators to run when not in cloud mode.

Implementation details

Backend (additive, backward-compatible)

  • New selectedEvaluators?: string[] option on IEvaluationOptions — when set and non-empty, overrides evaluatorFilter. The existing evaluatorFilter type ("all" | "errors" | "suggestions") and CLI --evaluator-filter flag are unchanged.
  • New filterEvaluatorsByIds() function in evaluator-types.ts — filters the evaluator list by specific IDs (filenames without .md), preserving original order.
  • Runner integration — both runAllEvaluators() and runUnifiedEvaluation() accept and use selectedEvaluators, falling back to category-based filtering when not set.
  • Engine pass-throughselectedEvaluators is explicitly forwarded at both runner call sites in engine.ts (per the documented Configuration Options Flow pattern in CLAUDE.md).
  • API enrichmentGET /api/evaluators now returns issueType ("error" | "suggestion") per evaluator, enabling the frontend to group them.

Frontend

  • Multi-select dropdown in RepositoryUrlInput.tsx replaces the old <select>:
    • Button shows "{N}/{total} evaluators" with chevron
    • Dropdown panel with "Select All" / "Deselect All" controls
    • "Errors (13)" and "Suggestions (4)" group headers with group-level checkboxes (supports indeterminate state)
    • Individual checkboxes per evaluator with minimum-1 constraint
    • Click-outside-to-close via useRef + mousedown listener
  • useEvaluationApisubmitJob() accepts and sends selectedEvaluators in the POST body
  • Progress trackinggetFilteredEvaluatorCount() accepts optional selectedCount for accurate progress bar initialization
  • App.tsx — forwards selectedEvaluators through the submission flow

Tests

  • 7 new tests for filterEvaluatorsByIds covering: specific selection, order preservation, empty input, unknown IDs, full selection, and mixed error/suggestion selection

Files changed (11)

Layer File Change
Types src/shared/types/evaluation.ts Add selectedEvaluators field
Backend src/shared/evaluation/evaluator-types.ts Add filterEvaluatorsByIds()
Backend src/shared/evaluation/runner.ts Wire selectedEvaluators in both runners
Backend src/shared/evaluation/engine.ts Pass-through at 2 call sites
API src/api/utils/evaluator-utils.ts Add issueType to evaluator response
Frontend frontend/src/types/evaluator.ts Add issueType field
Frontend frontend/src/hooks/useEvaluationApi.ts Add selectedEvaluators param
Frontend frontend/src/lib/formatters.ts Accept selected count
Frontend frontend/src/components/RepositoryUrlInput.tsx Multi-select dropdown UI
Frontend frontend/src/App.tsx Forward selectedEvaluators
Tests src/shared/evaluation/evaluator-types.test.ts 7 new tests

This PR was written using Vibe Kanban

Replace the 3-option category filter (All/Errors/Suggestions) with a
multi-select dropdown listing all 17 evaluators individually, grouped
by type. Users can pick any combination (minimum 1 required). When all
are selected, behavior is identical to before. Only shown in non-cloud
mode.

Adds selectedEvaluators option that flows through the full stack:
frontend -> API -> engine -> runner. The existing evaluatorFilter and
CLI --evaluator-filter flag remain unchanged for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cteyton cteyton changed the title Select specific evaluator (vibe-kanban) Add individual evaluator selection with multi-select dropdown (Vibe Kanban) Feb 6, 2026
cteyton and others added 3 commits February 6, 2026 20:33
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ment

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The multi-select dropdown was being clipped by the parent .glass-card
container which applies overflow-hidden. Override with overflow-visible
on the form card so the dropdown can extend beyond the card boundary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cteyton cteyton merged commit ed89167 into main Feb 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant