Skip to content

feat(admin): pick auto-routing benchmark models with the shared picker - #5144

Merged
iscekic merged 5 commits into
mainfrom
auto-routing-picker-ae67
Aug 7, 2026
Merged

feat(admin): pick auto-routing benchmark models with the shared picker#5144
iscekic merged 5 commits into
mainfrom
auto-routing-picker-ae67

Conversation

@iscekic

@iscekic iscekic commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Users can select benchmark classifier and manual decider models with the shared model picker. Decider variants come from each model catalog and persist across saves and reloads.

Product managers get consistent model selection for both benchmark tables and variant-aware published routing data. Existing server validation still rejects ineligible or duplicate decider models.

Maintainers get a nullable D1 variant column, legacy effort fallback, canonical variant propagation to the CLI, and publish fidelity for non-enum variants. The benchmark worker must deploy with migration 0009 before or with the web app; its predeploy applies the migration. The services/auto-routing worker gains variant-accurate serving after its deployment, but no rollout order is required because older readers keep today’s default behavior.

Shape decision: the variant picker has no clear item by design. Removing and re-adding a row resets its variant, while a saved catalog key remains visible and editable.

Human steps: before merge, deploy the benchmark worker with migration 0009 before or with the web app. After merge, deploy services/auto-routing to enable variant-accurate serving for non-enum keys.

Visual Changes: Benchmark Config picker card

E2E: bot-e2e — VERIFICATION PASSED. from the fresh local iOS browser round.

@iscekic iscekic self-assigned this Aug 7, 2026
@iscekic
iscekic requested review from RSO and jeanduplessis August 7, 2026 18:40
Comment thread apps/web/src/app/admin/auto-routing/BenchmarksSection.tsx
@kilo-code-bot

kilo-code-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental re-review of 81553cef (empty "chore: retrigger review" commit; tree identical to previously reviewed 8e4d0dd2) found no code changes and no new issues, and the single prior suggestion on BenchmarksSection.tsx was rejected by the author, leaving no unresolved findings.

Files Reviewed (0 files)
  • No files changed since the previous review at 8e4d0dd2 (empty retrigger commit)
Previous Review Summary (commit 8e4d0dd)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 8e4d0dd)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
apps/web/src/app/admin/auto-routing/BenchmarksSection.tsx 546 VariantCombobox offers no way to clear a selected variant back to default/null (the replaced Select had an explicit "None" option)
Files Reviewed (20 files)
  • apps/web/src/app/admin/auto-routing/AutoRoutingAdminContent.tsx
  • apps/web/src/app/admin/auto-routing/BenchmarksSection.test.ts
  • apps/web/src/app/admin/auto-routing/BenchmarksSection.tsx - 1 issue
  • apps/web/src/components/shared/ModelCombobox.tsx
  • apps/web/src/components/shared/VariantCombobox.tsx
  • services/auto-routing-benchmark/migrations/0009_cold_cobalt_man.sql
  • services/auto-routing-benchmark/migrations/meta/0009_snapshot.json
  • services/auto-routing-benchmark/migrations/meta/_journal.json
  • services/auto-routing-benchmark/src/admin.test.ts
  • services/auto-routing-benchmark/src/auto-decider-sync.test.ts
  • services/auto-routing-benchmark/src/config.test.ts
  • services/auto-routing-benchmark/src/config.ts
  • services/auto-routing-benchmark/src/db-schema.ts
  • services/auto-routing-benchmark/src/db.test.ts
  • services/auto-routing-benchmark/src/db.ts
  • services/auto-routing-benchmark/src/profile-runs.test.ts
  • services/auto-routing-benchmark/src/routing-table-builder.test.ts
  • services/auto-routing-benchmark/src/routing-table-builder.ts
  • services/auto-routing-benchmark/src/run-process-job.test.ts
  • services/auto-routing-benchmark/src/run.ts

Verified: the nullable D1 variant column migration is safe and matches db-schema.ts/snapshot; canonical-variant vs legacy-effort round-trips are consistent across config.ts, run.ts, db.ts, and routing-table-builder.ts (enum efforts keep the legacy published shape, non-enum variants publish as variant, never both); findSnapshot throws rather than returning undefined so the dropped ?. guard is safe; the ['openrouter-models'] invalidation prefix-matches the useModelSelectorList query key; no new memory-leak vectors (no new listeners/timers; existing poll effect retains its cleanup).

Fix these issues in Kilo Cloud


Reviewed by kimi-k3 · Input: 55.7K · Output: 3K · Cached: 78.4K

Review guidance: REVIEW.md from base branch main

@iscekic iscekic added the human-ready The PR is ready for human review. label Aug 7, 2026
@iscekic
iscekic merged commit fc0e0b9 into main Aug 7, 2026
13 checks passed
@iscekic
iscekic deleted the auto-routing-picker-ae67 branch August 7, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

human-ready The PR is ready for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants