feat(query): add @aeye/query — LLM-friendly query language, runtime &…#180
Merged
Conversation
… SQL converter New standalone package (packages/query): - Types/fields meta-model + class-based Expr system (one file per kind, Registry dispatch) - Relation-based joins, params, per-FieldType filters, functions (scalar/tabular/aggregate/window), array field type - In-memory runtime + base & postgres SQL converter (RLS/FLS, computed-field backing, named joins/LATERAL, three-valued NULL logic) - Dev-side Type backing (Access/Computed) so the conceptual model stays simple while fields map to columns/SQL/runtime fns - Graduated per-axis LLM schema depth + capability gating; strict/typed-args schemas - Transforms (drill-down, auto-paginate), cost estimation, type-from-data util - Examples folder + interactive CLI; README 100% test coverage (932 tests, base+postgres SQL + runtime), 0 typecheck errors, no any/unknown/casts. Wires packages/query into the workspace (root package.json, tsconfig.base.json). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…GROUP BY / ORDER BY / HAVING
`{ kind:'output', name }` delegates to the referenced select item's expression
(looked up by its `as` / natural name): expands to portable SQL (base+postgres)
and re-evaluates correctly at runtime (group keys over the source row; ORDER BY /
HAVING over the group, incl. aggregate targets). Offered by the LLM schema only
in groupBy/orderBy/having positions (gated out of the general Expr union), with
output.unknown / output.aggregate / output.not-available validation and drill-down
expansion. README "Output references" section. 100% coverage maintained (960 tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…generic Tool and Prompt gain an optional `parse` that fully REPLACES Zod validation (JSON.parse → parse → validate), so a caller can decode raw args into a built domain object and return rich compiler-style errors instead of Zod's. When `parse` is supplied its RETURN TYPE drives a new appended-last `TDecoded` generic (default = the wire type) that types call/validate/metadataFn and the structured-output value — the handler receives the decoded value with no cast. Backward-compatible: absent `parse` ⇒ identical Zod behavior. Docs + guides + reference updated to the 7-param signatures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
AIToolInput/AIPromptInput and the AI.tool()/AI.prompt() factories carry a trailing TDecoded generic (appended-last-with-default) so a custom parse's return type surfaces through the convenience layer; all existing callers compile unchanged. Also fixes a pre-existing parse-ctx hydration/contravariance mismatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…lf-describing, core Tool wiring - Cross-type semantic pairing (bound-source query form) + numeric text-score (ts_rank); "top N by similarity/relevance" via ORDER BY score DESC LIMIT N. - Function library expanded to 60+ (date/array/scalar/agg/window), renamed camelCase, per-dialect SQL (postgres native, base ANSI/degrade); an inline raw-literal arg mechanism for date-field selectors. - e.* expression builder (returns real Expr instances; engine.evaluateExpr / exprToSQL; parseExpr passes an already-built Expr through). - Search/semantic backing (hidden tsvector/pgvector vectorField) + backing alias-correctness (run factories receive the bound alias). - `output` expr (reference a SELECT field in groupBy/orderBy/having); filters teardown of the op/clause machinery + query.filters() introspection. - Self-describing: FunctionDef.instructions + static Expr INSTRUCTIONS + describeExprs/describeEngine + generated Type/Field descriptions; the CLI prompt is fully informed from the live engine. - buildQueryTool returns a wired @aeye/core Tool (custom parse → built Query / QueryToolError, bypassing Zod). - Cost-estimation accuracy (join fan-out + per-outer-row subquery multiplication). - New aeye-query.md (LLM-facing) + README; 100% coverage gate (vitest+v8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
A custom `parse` can decode raw args into ANY value (a number, Date, array, boolean, class instance, …), not just object/string — so `TDecoded`'s constraint on Tool/Prompt and the ai.tool()/ai.prompt() wrappers is now `extends unknown` (defaults = the wire type, unchanged). Component's `TInput` is widened to match, since a component's input IS the decoded value. No runtime change and no new casts (the existing wire-fallback bridge stayed valid). Tests prove a primitive `parse` return (e.g. `() => 42`) types the handler input as `number` end-to-end through Tool.parse, Prompt structured output, and the ai wrappers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…te + field exprs/default)
Types declare insertable?/updatable?/deletable? (default true); fields declare
insertable?/updatable? and exprs?: {not|only} (narrow which expr kinds may
target them, bounded by the field type). FieldBacking.default (value or factory)
makes a field optional-on-insert and materializes it at runtime — no hasDefault;
a computed field defaults to non-insertable/updatable.
Enforced in BOTH layers: validation (insert/update/delete type+field readonly,
the insert-requiredness rule, field.expr-denied at direct-subject + operator
sites) AND schema-building (drop insert/update/delete kinds when no permitted
Type; filter into/type/from enums; paired-mode required-vs-optional insert
fields and updatable-only set; expr operand field enums honor not/only +
capability-gate a kind away when all fields exclude it). Runtime materializes
defaults (value/sync/async); describe surfaces the write-model tersely so the
self-describing prompt reflects it. aeye-query.md + README + 26 tests; 100%
coverage gate held.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…fied cases) A separate `packages/query/integration/` harness — NOT run with the unit tests and excluded from the coverage gate — that measures whether an LLM builds CORRECT queries from natural-language requests: - a 20-Type ERP model + deterministic, distractor-laden JSON data (a wrong query returns a wrong answer); - 100 NL request cases across filter/join/aggregate/group/having/top-n/date/ text/subquery/set-op/cte/window/pagination/write-model, each with a hand-written correct `oracle` query whose result IS the expected (derived from the data, never hand-guessed); - run.ts: LLM mode (needs OPENROUTER_API_KEY; @aeye/ai + @aeye/openrouter; buildQueryTool + a describeEngine-informed prompt) generates → runs → compares → reports a pass rate; a key-free `--check` validates every oracle is valid/ deterministic/discriminating; `--only`/`--category`/`--limit` for cheap per-case iteration; - gitignored logs/latest.json (keyed by test id: emitted query + diagnostics + pass/fail + diff) and logs/failures.md for iterating on failures. npm scripts `integration` / `integration:check`. Does not affect the unit suite, the 100% coverage gate, or the src typecheck. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
FieldBacking.relation lets a relation field declare its physical FK key column(s) (hidden from the LLM) so the join ON is synthesized from them (source.local = target.foreign, composite ANDed; foreign defaults to the target's identity), plus a custom ON hook (expr/sql/run, access/computed-style precedence) for dynamic predicates. Wired through every ON site — join SQL + runtime applyHop, relation-walk/planner, relation-path, and TypeBacking.joins relation specs — alias-correct, with JoinDef.and still ANDed on; the inverseRelation has-many reuses the owning relation's FK. No backing ⇒ the current name convention (byte-identical). Also fixes the has-many LEFT JOIN anti-join case surfaced by the integration eval (a wrong convention FK produced zero matches → all rows). 25 tests; 100% coverage gate held. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
… physical FKs) The integration eval's 20-type ERP now models relations like a real schema: clean LLM-facing relation fields (customer / order / product / category / parent / creator / …) with the physical FK columns (customer_id / order_id / category_id / parent_id / …, snake_case) HIDDEN in FieldBacking.relation.keys — so the joins resolve via the backing, not the name convention. Includes a composite-FK relation (salesReturn.invoice, keyed by order + customer) and a custom relation.on.expr (salesOrderLine.order), both exercised through the in-memory runtime by integration:check. Data regenerated (snake_case FKs, every trap/distractor preserved, deterministic; also fixed a stale generator vs the committed 4-level category tree); 100 → 101 cases; integration:check 101/101. src suite / 100% coverage / typecheck unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ed scope)
TypeBacking.defaultConditions ANDs a predicate into every read/update/delete for
a bound source BY DEFAULT (reusing the RLS injection path, so SQL + runtime
agree), and LIFTS for that source when a `without` field is referenced in a
CONDITION position (WHERE / HAVING / JOIN `and`) — not select/order/group.
`without` defaults to the fields the `where` predicate references; `ops` default
to select+update+delete (never insert). Alias-correct per bound source,
subquery-scoped, and ANDs alongside RLS (which is never suppressed).
describeType surfaces each condition + its reveal mechanism so the model knows
how to view the hidden rows.
e.g. a files table: `{ where: (a) => e.isNull(e.ref(a,'archivedAt')) }` → live
files by default; archived become visible the moment a filter references
archivedAt. 23 tests; 100% coverage gate held.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ed, sensible-only)
TypeBacking.defaultOrder gives a Type a natural sort applied to an unsorted
SELECT's ORDER BY — { by: [{ by: Computed, dir?, nulls? }], applyTo }. The sort
key is a Computed (dual expr/sql/run), resolved alias-correct so the SQL ORDER BY
and the runtime sort agree. Applied only when sensible: FROM binds the backed
Type, the query has no explicit order, it is not aggregated (no groupBy / bare
aggregate) and not DISTINCT, and it is in scope per applyTo — 'result' (root
query + any SELECT with LIMIT/OFFSET) | 'paginated' (limit/offset only) | 'all'.
Root vs nested is an isRoot marker threaded from engine.run/toSQL (subqueries,
CTE bodies, and set-op branches are non-root). SELECT-only; pairs with
autoPaginate for deterministic pagination; describeType surfaces a terse note.
24 tests; 100% coverage gate held.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
A custom `parse` can decode into ANY value (a number, Date, array, class instance, …), not just object/string — so the Tool/Prompt/Component generic signatures + notes in aeye-core-tools.md, aeye-core-prompts.md, the core README, and docs/public/llms.txt now show `TDecoded extends unknown` (and `Component.TInput extends unknown`) instead of `extends object` / `object | string`. Docs-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…map) New src/json-source.ts: a positioned canonical JSON stringifier that maps every node's structural path to its [start,end] char range in JSON.stringify(v,null,2). Code.fromJson(value) registers those spans, so reportFor underlines the offending value in the model's own JSON — for BOTH structural (zod) and semantic (validateWalk) problems. flattenZodIssues descends the .or-folded union tree, prunes branches that reject the `kind` discriminant, and keeps the deepest match — collapsing hundreds of union sub-issues to the single offending value. Result: a malformed model query renders as concise, localized `^^^` diagnostics instead of zod union noise. 12 tests; 100% coverage gate held. (Phase A of the own-the-parser plan.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…nostics)
Every generated schema node now carries an `aid` (the identifier used in
recursive schema generation) plus a per-schema directed zod error, so a
malformed query renders domain-specific text instead of zod's generic types —
still underlined at the offending JSON (Phase A):
"left": "oops" → expected an expression
"op": "equals" → expected a comparison operator: =, <>, <, <=, >, >=, like, notLike, ilike
"args": "total" → expected named arguments, an object of { argName: <expr> }, got a string
"kind": "comparise"→ unknown expression kind `comparise` — did you mean `comparison`? (available: …)
"limit": "three" → expected a number or a param
unknown Type name → expected a registered Type name
New src/aids.ts: withAid(schema, aid, opts) (captures a zod error map surviving
.meta()/.describe()/lazy clones + stamps aid metadata), an aid→{label,noun}
registry, directedMessage() per issue code, and Levenshtein nearestKind for the
"did you mean". flattenZodIssues surfaces the union's aid-directed message.
Wire schema content unchanged except the aid metadata + friendlier text. zod
stays the validator (no owned-parser rewrite). 17 tests; 100% gate held.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…a $defs PART A — the LLM problem report is now configurable: BuildQueryToolOptions.report (FormatProblemsOptions) threads through parseQueryInput → reportFor → formatProblems, so a dev sets contextLines / sectionHeaders / lineNumbers / maxProblems (default unchanged: 2 context lines, headers + gutter on). PART B — factor the largest repeated schema fragments into shared, aid-named $defs (a per-generation SchemaCache reuses one instance so the converter emits $def + $refs; withAid gains `id` to stamp meta.id, salted per generation to stay unique in zod's global registry): per-Type field-name enums (Fields_<Type>, ~43 inlined copies → 1), param (~14→1), Limit, and typed function-args by signature (104 builtins → ~35 distinct arg signatures). Minified z.toJSONSchema: open 21,295 → 20,235 (−5%) paired 105,609 → 95,543 (−9.5%) The residual paired bulk is the ~100 genuinely-distinct per-function branches of `typed` mode (irreducible without changing acceptance). Golden test pins 10 valid + 10 invalid queries to the SAME accept/reject + directed messages, plus a size-threshold guard against re-inlining. 100% gate held; 6 new tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Generalized the suggester (aids.ts): didYouMean(input, candidates, {max}) +
nearest() + suggestionBudget(len) = min(3, max(1, floor(len/3))), case-
insensitive, TYPO-only (never an unrelated word). Applied everywhere a name can
be wrong — appended to the existing directed, underlined messages:
- validation: unknown field → nearest of the Type's fields (field-ref, filters,
relation-path, semantic + query-field, text-search/score, excluded,
insert/update); unknown source → nearest bound source (new QueryScope.sources());
unknown Type → nearest registered Type (insert/update/delete/source/relation-path);
unknown relation hop → nearest relation; unknown function → nearest builtin of
that shape; unknown named-arg → nearest param; unknown output ref → nearest
select output;
- schema enums: a near-miss op / dir / nulls / function / Type / field value
appends the nearest allowed value via directedMessage;
- (unknown expr/query kind already suggested — nearestKind now wraps nearest()).
e.g. op:"notlike" → "…, notLike, ilike — did you mean `notLike`?";
ref('u','nam') → "Type 'user' has no field 'nam'. — did you mean `name`?".
26 new tests; 100% coverage gate held.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…+ result)
Each eval case now declares `assert: Assertion[]` (ALL must hold) mixing
STRUCTURE with RESULT, so a case verifies the model built the right SHAPE — not
just that its rows happened to match. New integration/cases/assert.ts `a.*`:
structural (walk the model's queryDef, never run it) kind/from/joins(to)/
filtersOn/groupBy/having/aggregate(fn)/orderBy({by,dir})/limit(n)/offset/
distinct/window/setOp/cte/selects/custom/refused; result (lazy cached run)
resultOf(oracle,{match,tolerance})/rowCount/rows. `a.joins(to)`/`a.from` resolve
a relation hop's target Type via the registry so e.path(...) joins are
recognized.
EvalCase → { id, category, request, note, assert }. Runner evaluates every
assertion (case passes iff all pass) and logs per-assertion pass/fail + reason.
--check validates each resultOf oracle (valid/deterministic/non-degenerate) +
each refused sample fails; --only/--category/--limit kept. All 101 cases
rewritten to assert shape + result per request; a self-check (each oracle fed
back as the model query) confirms the 97 oracle-bearing cases pass their own
assertions. integration:check 101/101; src suite/coverage/typecheck unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…to bool New condition.non-bool check (src/queries/_condition.ts checkBoolCondition): a top-level predicate that isn't a param and doesn't resolve to bool is flagged "Expected a boolean condition; got <category>." at its own path (underlined) — matching how logical operands and case `when` are already checked (params exempt/inferred). Wired into SelectQuery where[i] + having[i], the join `and` (new QueryJoin.validateWalk — which also newly validates the and predicate's inner exprs, previously unvalidated), UpdateQuery where, DeleteQuery where. A comparison/in/between/exists/filters/bool-field predicate passes; a money field-ref / literal number / arithmetic expr is flagged. 17 tests; 100% gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Step 1 of replacing zod's structural error path with a query-owned parser that
each expr owns, never throws, and accumulates one-or-more shape problems per pass
(like validateWalk does for semantics). New src/shape/: Shape<T> combinators
(lit/str/num/int/bool/scalar/enumOf/obj/optional/list/exprRef) whose check() is
total — on bad input it records an aid-directed problem (+ didYouMean) and returns
the INVALID sentinel, never throwing. obj is fully typed with NO cast via an
inverted generic (F = built-value record, field map = { [K]: Shape<F[K]> }) + a
completeness type guard. registry.parseCheckedExpr(json: unknown, problems)
defensively dispatches (shape.not-object / missing-kind / unknown-kind + didYouMean)
to each class's new static SHAPE. Exemplars: comparison, field-ref, literal, param,
logical (with the not-arity cross-field rule). Parallel + unit-tested only — zod
still gates parseQueryInput; nothing flipped yet. 42 tests; 100% gate (1371 total).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ision) + rewire eval to the query parser
The integration eval crashed EVERY case before reaching the model with
`Error: ID param_g2 already exists in the registry`.
Root cause: `SchemaCache` tagged the shared `$def` fragments (`Fields_*`,
`Args*`, `param`, `Limit`) with `.meta({ id })`, which registers into zod's
PROCESS-GLOBAL registry — and that registry THROWS on any duplicate id. Core's
`strictify` (openai.ts `convertResponseFormat`, used to build the model wire
schema) CLONES every node and re-applies its `.meta()`; the instant the wire
schema is strictified it re-registers those ids and collides. Salting the ids
per generation cannot help — strictify re-registers the SAME id, not a new one.
Fix: route the shared-fragment ids through a NEW process-LOCAL `z.registry()`
(`sharedIdRegistry` in aids.ts) instead of zod's global registry. `.meta({ aid })`
is unaffected — zod's `add()` only guards the `"id"` key, so re-registering an
aid never throws, and core's converter (which factors by `aid`, or by re-hitting
the same memoized instance for the aid-less `param`/`Args`) still emits a fully
factored model-facing wire schema. A plain `z.toJSONSchema` is told to read ids
from the local registry via its `metadata` option, so the `$def`/`$ref`
factoring — and the size win — is preserved without any id touching the global
registry. Verified via a no-network repro: `strictify(querySchema(erp))` no
longer throws, and the factored open/paired sizes stay well under the re-inline
bounds.
The factoring test now reads the shared ids from `sharedIdRegistry` and asserts
the `$def` names by PATTERN (`Fields_*` / `Args*` / `param(_g<n>)?` /
`Limit(_g<n>)?`), since the ids are salted per schema-generation; the size
thresholds are unchanged.
Also rewires the eval harness (`integration/run.ts`): the prompt now validates
model output through core's `parse` hook running the QUERY PARSER (`tool.parse`),
so the output-retry loop is driven by the package's own compiler-style,
aid-directed diagnostics instead of raw zod union noise — zod stays only as the
wire schema the model emits against.
typecheck 0; coverage 100% (1371 tests, --testTimeout=45000); examples exit 0;
integration:check 101/101; integration tsc 0. A small real eval
(openai/gpt-4o-mini) confirms the `already exists` crash is GONE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Migrates the remaining non-query-referencing exprs to the zod-free structural parser (after C1's foundation): binary, unary, is-null, between, in (list form), case, array-op, aggregate, window, function-call, relation-path, output, excluded, text-search, text-score. Each SHAPE.check(def).toJSON() deep-equals from(def), accumulates every shape problem in one pass, never throws. New combinator record(valueShape, aid) for named-arg maps (ordered Map, per-key accumulation) + file-local array-value / text-query shapes matching existing behavior. Deferred to C3 (need queryRef/sourceRef): subquery, exists, in-with-subquery, semantic, tabular-function-call, filters (in-with-subquery records a shape.todo, list form works). Still parallel — parseQueryInput's zod gate untouched (flips in C4). New AIDs OutputName/Not/Distinct/CaseBranch/Order. 26 tests; 100% gate (1397 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
… (full tree) Completes the zod-free structural parser's coverage. New combinators queryRef / queryDefRef / sourceRef; dispatches registry.parseCheckedQuery + parseCheckedSource (mirror parseCheckedExpr — never throw, shape.not-object/missing-kind/unknown-kind + didYouMean). Migrated: 8 query kinds (select, insert +onConflict, update, delete, union/intersect/except, cte, expr) and 4 source kinds (type, aliased, subquery, function); finished the 6 C2-deferred exprs (subquery, exists, in-with-subquery — shape.todo gone, semantic, tabular-function-call, filters). Shared shapes in new src/queries/_shape.ts + QueryOrder/QueryJoin SHAPE. parseCheckedExpr/Query/Source now cover the WHOLE tree: a complex SELECT parses to a Query deep-equal to registry.parseQuery, and a query with errors in ≥2 clauses surfaces them all in one pass. Still parallel — parseQueryInput's zod gate flips in C4. 54 tests; 100% gate (1451 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
The query-owned structural parser is now the ACTIVE gate. parseQueryInput no longer runs schema.safeParse: it validates the envelope zod-free (missing `query` → shape.required), preserves the string-prose fallback, then registry.parseCheckedQuery does the structural parse (accumulating, aid-directed, didYouMean, never throws), and engine.validateQuery runs the semantics on a sound tree. Deleted problemsFromZod / flattenZodIssues / FlatZodIssue / the discriminant helpers / the safeParse gate. zod is now WIRE-SCHEMA-ONLY — querySchema/buildSchemas still generate the schema the model emits against (tool.schema / compile / strict mode); the golden test still proves the wire schema accepts/rejects the same fixtures. A malformed query now surfaces ALL its structural problems in one pass, each underlined + aid-directed, with zero zod in the diagnosis; unknown names/types fall through to validateWalk's semantic messages (structure-vs-semantics split). Tests updated to shape.* codes/messages; +1 accumulation test. 100% gate (1452 tests); examples 0; integration:check 101/101. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…NSchema overflow) array-op's toSchema wrapped its child `target` as `opts.Expr.describe(...)`. Calling .describe() on a z.lazy CLONES it into a fresh wrapper carrying no `aid`, so a converter that caches lazies by instance-identity + wrapper aid (e.g. core's toJSONSchema) misses BOTH keys, re-evaluates the getter, and re-expands the whole recursive Expr union — which contains array-op again — until the stack overflows. Every other expr threads childExprSchema(opts.Expr) in bare (the shared cached instance), so only array-op looped, and only when an array field made it appear in the union (via exprKindApplicable). Use the child directly: a recursive child converts to a bare $ref anyway, so the description was dropped regardless. Unblocks toJSONSchema on any engine with an array field (e.g. the 20-Type integration ERP); the eval now reaches the model instead of crashing. No core changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ch-all
A prompt can intercept each tool's result before it's handed to the model and
return what the model sees instead. Single discriminated handler:
onToolResult?: (event: ToolResultEvent<TContext, TMetadata, TTools>) => unknown | Promise<unknown>
ToolResultEvent is a discriminated union over the tools tuple (mirrors
PromptToolOutput) so narrowing on event.tool types BOTH event.result and
event.args per tool; the default branch is the catch-all. The return value IS
what's presented to the model (serialized identically to today); return
event.result to pass through. Model-facing only — get('tools')/streamTools/the
raw toolOutput result are unchanged, and the toolOutput event gains a `toModel`
field carrying the presented value. Runs inside newToolExecution.run() on the
success path; a throwing handler cleanly converts the slot to a normal tool
error (pairing guarantee preserved); errored/suspended/interrupted/synthetic
markers bypass. v1 = success results only (error transforms are a trivial v2 via
a status discriminant). Fully type-safe: wrong tool-name / field access fail to
compile (@ts-expect-error tests). 14 tests; core suite 533 pass, typecheck 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…nostic) A custom parse (Tool.input.parse / Prompt.input.parse) replaces zod validation, which also dropped core's strictify DECODE preprocesses — so a custom parser saw the raw provider wire shapes (OpenAI array-of-pairs records, null-for-optional, numeric-key tuples) and rejected valid model output. Now core decodes the wire response back to the conceptual shape BEFORE invoking the custom parse, symmetric with how strictify encodes the request. New in schema.ts: relaxValidation(schema) — a generic zod-tree walk that KEEPS every transform wrapper (preprocess/codec/pipe/transform) and every container (object all-optional+passthrough, array/tuple/record/map/set) so nested transforms are reached, and RELAXES validating leaves to z.any(); handles optional/nullable/ default/catch/readonly/branded, unions/intersections, and lazy recursion (memoized). decodeWire(schema, value, descriptor) = relaxValidation(strictify(schema, descriptor)).safeParse(value) → best-effort conceptual value, NEVER throws (a bad sibling passes through for the custom parser to report). Cached per descriptor.id. Wired into Tool.parse (~331) and Prompt output-parse (~1204) — decode with the descriptor already pinned on the request. The decode is DERIVED from the same strictify'd schema, so encode and decode can't drift: adding a new FormatDescriptor gives both wire-schema and decode with zero new code. @aeye/query's owned parser now stays provider-agnostic (no query changes). 65 new tests (all zod arrangements + per-descriptor symmetry round-trips + partial/malformed leniency); core 598 pass, typecheck 0, no new casts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Document that a custom Tool/Prompt `parse` now receives the structured value already wire-decoded to the conceptual shape (core applies the request's FormatDescriptor decodeWire first — array-of-pairs→record, null→undefined, numeric-key→tuple), so a custom parser is provider-agnostic. Follows 029bf1e. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ant tool builds
The parse pipeline (parseQueryInput) was only reachable via buildQueryTool(...).parse,
so integration/run.ts and examples/cli.ts built a whole Tool just to parse a def.
Expose it as standalone public fns sharing the ONE implementation:
parseQueryRequest(engine, raw, opts?) -> { query, problems, report }
parseQueryTool(engine, raw, opts?) -> Query | QueryToolError
buildQueryTool's custom parse is now a one-liner over parseQueryTool (no dup).
integration/run.ts: the prompt parse hook calls parseQueryTool(engine, raw) directly
(no Tool built) — correct now that core wire-decodes before the custom parse, so the
hook receives the conceptual value; querySchema stays as the prompt's wire schema.
examples/cli.ts: the four Tool-just-to-parse sites call parseQueryTool + engine.run
directly. buildQueryTool kept for 08-llm-tool.ts where a runnable Tool is demoed.
All query tool/prompt code now routes through the one owned-parser entry. 10 new
tests; 100% gate (1462); examples 0; integration:check 101/101.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ential stall) The eval ran cases sequentially with output piped through tail (buffered until EOF), so a case that fails PARSE and re-prompts through outputRetries (2-3 calls) stalled the whole run silently — one case hit 99s, and the full suite took ~70 min with no visibility. Now: - run cases CONCURRENTLY (pool of --concurrency, default 8) — a few minutes total; - stream per-case results AS THEY COMPLETE with wall-time + model-call count, so slow/retrying cases surface immediately (e.g. `[ 11/101] PASS 6.9s 3c …`); - ask() catches the prompt's retry-exhaustion throw so a failed case still reports its call count + diagnostics (was showing `0c`) instead of losing both. Same assertions/logs/--check; integration tsconfig 0, integration:check 101/101. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…gory The harness counted model requests per case (Nc) but never aggregated them. Add avgCalls + avgCallsPassed to report.json/report.md/console, per-category avg, and per-case calls — so a higher pass rate bought with more retries is visible as cost. MODELS.md gains an avg-attempts column: Sonnet 4/4.6 win at ~1.2 (near one-shot) while GPT-5.1's 75% costs 2.25 (Mode-4 fallback re-issues every case). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Read the provider-reported usage.cost (OpenRouter returns it per request; core's accumulateUsage sums it across retries) — no local pricing math. Add totalCostUsd/avgCostUsd to report.json + console + report.md, and per-case tokensIn/tokensOut/costUsd. So pass rate, avg attempts, AND $ cost are tracked together (e.g. gemini-2.5-flash-lite ~$0.21/100 cases). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Full 101 runs with cost: GPT-5 mini 92% @ $0.43 (value leader, ties frontier Sonnet 4.6); Claude Haiku 4.5 83% but $3.42 (poor value, mid-tier pricing); Gemini 3.1/2.5 Flash Lite 81/75% at $0.35-0.51 (cheapest). Cost is input-dominated (~20K tok/case: schema + all examples). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Aggregate the per-case durationMs into avgDurationMs (wall-clock per case, cases run concurrently) — a speed signal alongside pass rate, attempts, and cost. Added to report.json + console + report.md; MODELS.md documents all four axes and the per-case fields. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…nfigurable
Positional INSERT (`fields: string[]` + `values: ExprDef[][]`, aligned by index)
was unformattable by LLMs (they emit 1 value for 5 fields). Replace with
field-keyed objects:
- InsertDef.rows: InsertRowDef[] (each { [field]: WriteValueDef })
- UpdateDef.set: SetDef ({ [field]: WriteValueDef }); OnConflictDef.update too
- WriteValueDef = JsonValue | ExprDef (a raw typed value OR a full expr)
Omit vs set-NULL (OpenAI-safe, since OpenAI can't omit a field): absent key OR
JSON null = OMIT (backing default / unset); an explicit literal-null expr
{kind:'literal',value:null} is the ONLY way to set SQL NULL. A raw scalar is
wrapped in a LiteralExpr; a raw array/object → write.unsupported-value.
New `writes` depth axis (open|names|typed, mirrors FnDepth): open = record of
exprs; names = per-Type field-name enum + expr values; typed = per-Type union,
each field's value = its FieldType.toValueSchema() | Expr, required-on-insert
enforced. paired→typed; degrades typed→names→open by field count.
Multi-row INSERT requires homogeneous keys (insert.row-shape). INSERT…SELECT
derives target columns from the SELECT output names.
Green: tsc 0 (src+integration); vitest 1484 @ 100% coverage; examples 0-err;
integration:check 101/101. Oracles + ~14 test suites migrated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
reject tool: a proper `reject(reason)` affordance for declining an impossible
write, with reason a CONTROLLED ENUM (field-read-only / type-read-only /
append-only-no-delete / immutable-field / reference-data) drawn from the
write-model policy codes — so the model can only reject for a genuine permission
reason, never "the JSON is hard to format" (the give-up that made an earlier
free-text attempt over-reject). The call ToolInterrupts; ask() reports it as a
clean refusal with the enum reason (per-case `rejected`).
reasoning: QUERY_EVAL_REASONING=low|medium|high (unset = off) sets the prompt's
`config` → `{ reason: { effort } }`; OpenRouter maps it to each family's thinking
API and reasoning tokens/cost flow through usage.cost (already tracked).
Together with the keyed-object INSERT schema, write-model on gemini-2.5-flash-lite
4/6 -> 5/6: all 4 refusals pass with correct enum reasons, the allowed insert is
NOT over-rejected (built first try), and the lone fail is a real projection error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…oisy/costly) reasoning=high: 75->81 (+6) but 15 gains / 9 regressions, 1.7x cost, ~22s/case. Doesn't reach the no-reasoning frontier (89-91% @ ~1.2 tries) — poor ROI vs just using a frontier model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…cy error GPT-5 used reject as a give-up on VALID inserts (bailing with a nonsensical enum reason). Gate it: reject only fires AFTER the model attempted the write and hit a genuine *-readonly validation code; otherwise the tool returns 'not a policy failure, build the query' and the model retries. A valid insert never produces a -readonly code, so it can't be rejected; a forbidden write does. gemini-2.5-flash-lite write-model 5/6 -> 6/6 (the allowed inserts now build, no over-rejection; refusals still pass). GPT-5 verification running. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…atrix Rebuild MODELS.md from a fresh 10-model sweep (all 101 cases, current schema: post relation-path removal, post insert/update redesign, with the enum reject tool). Supersedes the schema-delivery matrix and the named-join table (kept for history). - New authoritative "Current value leaderboard": composite value score (0.60*accuracy + 0.20*cost + 0.20*speed, min-max normalized) ordered by value. gemini-3-flash-preview is the champion (92%, $1.17/100, 3.4s); both Anthropic frontier models rank mid-pack despite 91% accuracy because ~$10/100 tanks their value. - New cross-model failure matrix (fails/10 per case) + root-cause analysis: the old universal subquery/set-op failures are fixed by the join refactor; the hard core is now recursive CTE (43% miss), window ties/frames (38%), write-refusal (35%, insert-id sticky), and the distinct flag. Each hard case documented with the specific reason models miss. - Flash-lite 59% shown to be variance (59/75/81% on same code), not a regression; qwen documented as an unrankable outlier (10% auto, 135s/case, OOM-prone). - Updated per-model notes and the ceiling section to fresh numbers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
A `relation→X` field-ref resolves to the whole related row (possibly composite-keyed), never a single value. Used as a scalar VALUE — a select field, aggregate/window value, partitionBy, orderBy, group-by key, or function arg — it previously validated with ZERO problems and silently read as null at runtime, so a natural query like `count(distinct line.product)` or `partitionBy: [salesOrder.customer]` produced wrong/empty results with no feedback. Now such a use raises `ref.relation-not-value` with the join-it hint, so the model's re-prompt loop can self-correct to the named-join form. A ValidateContext `relationValueOk` flag, set only by the FK-comparison operators (comparison / in / between / binary — which run their own relation-vs-relation / relation-vs-scalar checks), permits a relation operand there; everywhere else a bare relation ref is rejected. Surfaced by the SQL-first experiment: models produce correct SQL for the hard cases, then translate to structurally-correct AST that this footgun silently sank. Lifts pure query mode (gemini-3-flash-preview 92%→93%, avg attempts 1.41→1.32 — more accurate and cheaper). 1484 tests pass; all 101 integration oracles still coherent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ostic modes Two opt-in diagnostic tools that separate "can the model reason about the problem" from "is our query language the obstacle": - `sql-review.ts`: gives a model the same schema (describeTypes) but asks for a PostgreSQL statement instead of our AST, and writes a review doc per case (request + expected rows + our oracle→Postgres reference SQL + the model's SQL). Not executed — for eyeball review. Showed gemini/sonnet produce CORRECT SQL for ~11/13 of the cases they fail in our AST — i.e. the language, not model capability, is the main obstacle. - `run.ts` `QUERY_EVAL_OUTPUT=sql-then-query`: a two-stage mode that generates SQL first, appends it to the request (with a note on how named-relation joins differ), then asks for the AST. Verdict (kept as documented diagnostics, NOT a recommended mode): translation is net-NEGATIVE — gemini 93%→87% — because SQL idioms for arrays / string funcs / set-ops don't map to our native operators and mislead the model on cases it otherwise gets right. Its real value was surfacing the relation-as-value footgun (see the preceding fix), which lifted pure query mode instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…older A small CLI (`npx @aeye/context [dir]`) that copies every installed `@aeye/*` package's flat agent-reference markdown (`aeye-<pkg>.md` + topic sub-docs) into one folder and generates an `aeye.md` index, so the cross-linked flat sibling docs resolve on GitHub, in an editor, and for a coding agent pointed at them. Registered in the workspace list. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
MODELS.md documents model behavior against the CURRENT schema; the `relation-path` construct and its named-join replacement no longer exist, so the archaeology isn't useful to a reader. Removed the historical schema-delivery results matrix and the entire named-join-refactor before/after section, plus the inline relation-path / "post-refactor" / "pre-refactor" qualifiers, reframing the few current facts (subquery/set-op are no longer whole-category failures; the frontier reaches 91-93%) to stand on their own. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Add a `reason` column to the Current value leaderboard and fold in the 6 reasoning runs (gemini-3-flash-preview + gpt-5-mini at low/medium/high, QUERY_EVAL_REASONING), with value scores recomputed over the full 16-row set. The two base rows for those models now use their post-footgun-fix no-reasoning run so the reasoning comparison is apples-to-apples. Finding: every reasoning tier ranks BELOW its own no-reasoning base — gemini none 95% vs low/high 94% (medium 96% is noise) at 4-5x latency; gpt-5-mini none 91% beats all tiers (it reasons by default, so an explicit effort disrupts it), and high blows latency to 25.8 s/case. Reasoning buys no accuracy, only a latency/cost tax. Expanded the Reasoning section with the two-model grid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
A SELECT with a GROUP BY previously did NOT validate that select / order-by / having columns are grouped or aggregated: `SELECT region, total ... GROUP BY region` passed validation, then Postgres errors at execution and the in-memory runtime silently returns an ARBITRARY row's `total`. Same class of silent-wrong footgun as the relation-as-value one. Add a top-down, pruning check in SelectQuery.validateWalk: for each select / order / having expr, a subtree that IS a group key (matched by `toCode`, with an `output` key expanded to the select field it names) — or an aggregate / window — is covered; a bare column reached any other way raises `group.ungrouped-column` with an "add it to groupBy or aggregate it" hint. Composite exprs recurse, so `a || b` needs both grouped while a grouped `dateTrunc(...)` selected verbatim is covered as a whole. Also fixes the same gap for the coming dynamic `sorter` — its sort exprs validate through this path. 1490 tests (6 new), 100% coverage, 101/101 oracles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Container types (list / map / obj / tuple) sum `elementComplexity` over their ELEMENT types — i.e. call it on OTHER `Type` instances via a base-typed reference, which TypeScript forbids for a `protected` member (TS2446). Widen to `public`; it is an internal cost helper meant to be called across the type hierarchy. Unblocks `npm run typecheck` (was 5 errors). 850 tests still pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
An integration/eval harness for gin mirroring packages/query/integration: the
model generates a gin function `(args) => output` (an ExprDef body reading its
params via `args.<name>`); correctness is proven by running it over SEVERAL test
inputs and comparing each output to a plain-JS ORACLE, exactly like ginny's
designer flow. Improves on ginny's prompting where our query work showed it
matters: worked EXAMPLES in the prompt (ginny ships none) and an auto-RE-PROMPT
loop on `engine.validate()` problems (ginny is one-shot).
- `run.ts`: `--check` fixture gate (oracles run twice for determinism + output
type-validated + fn probes executed), LLM eval over OpenRouter, concurrency
pool, per-run archived logs, cost/latency/by-category report.
- `model.ts` / `asker.ts` / `describe.ts`: fresh registry+engine per case,
`describeGin` (type docs + expr grammar + examples + this case's fns), the
generated-function invoker, `compareValues`.
- `cases/`: `EvalCase { setup?, fns?, argsType, returnType, inputs[], assert[] }`
+ the `a` builder (`produces` / `usesFn` / `usesKind` / `returnsType` /
`refused`). 61 cases across num, text, obj, map, control, lambda, date, list,
functions/domain — each category includes a fns-WITH-DISTRACTORS case (some
provided functions are red herrings — tests function selection) and a
CUSTOM-TYPE case (nested objs / lists / enums).
Custom types use `extend(obj({...}), {name})`, NOT `extend('obj', {props})` —
the latter silently drops fields at runtime (a gin footgun; --check can't catch
it since it never executes gin). typecheck 0, --check 61/61, 0/15 custom-type
cases drop fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Add a `sorter` expr (valid only in a SELECT `order`), the ORDER BY analogue of
`filters`: the query author (or LLM) declares a CATALOG of named sortable
expressions (`sorts`) + optional multi-key `defaultSort`; the CALLER supplies the
sort SELECTION at execution time (`RuntimeOptions.sort` / `engine.toSQL({ sort })`),
so an end-user can re-sort a live result. The selection is never LLM-authored.
- `SorterExpr` (mirrors `FiltersExpr`): `expand(spec)` maps each `{sort,dir?}`
(dir⇒asc, in order) to `sorts[name]` → `QueryOrder[]`; empty/absent spec falls
back to `defaultSort`; an unknown selected name throws a loud `sort.unknown-name`.
Both the runtime sort and `toSQL` ORDER BY run the EXPANDED terms through the
existing comparator / emission — no duplicated logic.
- A `sorts` value may be any expr including an `output` ref (sort by a select item
without restating it). Each catalog expr is validated exactly like an order term
(same scope/ctx), so the SQL-92 GROUP BY rule applies to sorts too.
- Validation: `sorter.misplaced` (anywhere but a select `order`), `sorter.empty`,
`sorter.unknown-default`. Introspection: `Query.sorters(engine)` (mirrors
`Query.filters`) → each sort name's resolved orderable type. LLM schema folds
the sorter into the `order` position only; a worked example added to describe.
typecheck 0, coverage 100% (1524 tests, cov-exprs-sorter.test.ts +34),
examples exit 0, integration:check 101/101.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
`SorterExpr.evaluate`/`toSQL` return NULL and read like stubs, but a sorter is never a VALUE: the real runtime + SQL sorting lives in `SelectQuery`, which calls `SorterExpr.expand(spec)` via `SelectQuery.expandOrder(ctx.sortSpec)` (the ORDER BY phase of both `run` and `toSQL`) to expand the execution-time sort spec into concrete `QueryOrder[]` and run them through the existing order-by machinery. Point the two dead methods at `SelectQuery.expandOrder` so the next reader isn't misled. Comment-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…ne query Add `dynamic-customer-directory`: a request for a REUSABLE, caller-parameterized query that must compose all three execution-time constructs at once — a `param` (`minCredit`) in the WHERE, a `filters` placeholder on the customer source, and a dynamic `sorter` catalog (name / creditLimit, with a defaultSort) in ORDER BY. Structural (the param/filter/sort VALUES are supplied by the caller at run time, not authored), gated with `a.require(a.hasParam/hasFilters/hasSorter)`. Supporting harness changes: - `a.hasFilters()` / `a.hasParam()` / `a.hasSorter()` structural assertions. - The query-def walker now handles a `sorter` order entry (collects it as an expr and walks its `sorts` catalog) via a shared `walkOrderEntry` — previously it assumed every `order` entry had an `.expr` and would crash on a sorter. All 3 top models compose the trio correctly: gemini-3-flash-preview (1 call), sonnet-4.6 (1 call), gpt-5-mini (2 calls, incl. an `output`-ref sort). --check 102/102, typecheck 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…le sorter `drillDown` un-ravels an aggregating SELECT into row-level detail, where an aggregate-based sort has no meaning. It already dropped a plain aggregate ORDER BY term, but for a `sorter` it dropped the ENTIRE catalog if ANY one sort was an aggregate — losing the still-valid group-key sorts a caller could use on the drilled rows. Now `pruneSorterForDrill` drops only the aggregate entries from a sorter's `sorts` (and any `defaultSort` naming them), keeping the rest; the whole sorter is dropped only when that empties it. `output` refs in the catalog are already expanded to their underlying exprs first, so an `output`-ref sort that pointed at an aggregate select item is pruned correctly. Each drop is a `drill.order-dropped` warning. Also converts the order-keeping loop from a filter to explicit sorter/term handling. 3 new tests (mixed → prune, all-aggregate → drop whole, all-scalar → keep). 1527 tests, 100% coverage, 102/102 oracles, examples clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…rops
Follow-up to the sorter drill handling: rather than DROP an aggregate ORDER BY /
sorter sort, CONVERT it to an `output` ref to the column that carries it. When
`drillDown` un-ravels `sum(total) AS revenue` into the `total` column, the
`revenue` column survives — so a sort by `sum(total)` (or `output('revenue')`)
becomes `output('revenue')` and keeps sorting the un-ravelled rows, instead of
being lost.
- Un-ravelling records `aggByCanon` (each converted aggregate's canonical form →
its output name) and gives every converted aggregate an explicit alias so the
column keeps a STABLE name an `output` ref can target.
- `convertSort` maps a matching aggregate → `output` ref, keeps a non-aggregate,
and returns null (drop) only for an aggregate with NO surviving column (an
expanded `count(*)`, a nested aggregate). `convertSorterForDrill` applies it
per catalog sort, trimming `defaultSort` for any dropped sort and dropping the
whole sorter only when none survive. So a dropped select column (count(*))
correctly drops any order/sorter expr that referenced it.
End-to-end verified: a drilled query's converted `output`-ref sort runs + re-sorts
(0 validation errors). 1528 tests, 100% coverage, 102/102 oracles, examples clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Each aggregate function def now carries a serializable template describing
how it un-aggregates to a row-level value, using {kind:"arg",name} placeholders
that are substituted with the call args. Because the templates are plain ExprDef
they survive being sent over the wire alongside a function def.
- schema.ts: add ArgExprDef placeholder to the ExprDef union; add
FunctionDef.unaggregate / unaggregateEmpty (the arg-less variant).
- function.ts: thread unaggregate/unaggregateEmpty through QueryFunction.
- aggregate.ts: AggregateExpr.unaggregate() picks the arg-less template when
the call has no value arg, substitutes call args into the template, and parses
the result. sum(x)=>x, count(*)=>1 (droppable), count(x)=>CASE WHEN x IS NULL
THEN 0 ELSE 1 END, max(a)-min(b)=>a-b.
- builtins.ts: register the value / count / arg-less templates on the builtins.
- drill-down.ts: unaggregateDef walks any expr and un-aggregates every aggregate
it contains (windows and template-less aggregates => non-invertible). Applied
in SELECT (drop the item if it cannot be un-aggregated or references no field)
and ORDER (drop the sort term likewise); count(*) in SELECT still expands to
rows. Sorter placeholders are un-aggregated per-sort.
100% coverage; 1533 unit tests, 13 examples, 102/102 integration all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…urvive parse
A type-ref with `extends: 'obj'` (e.g. `{ name: 'Widget', extends: 'obj',
props: {...} }`) stranded its props in the Extension local. Because an
Extension's value ops delegate entirely to the base — and `obj`'s parse is
driven by the base's own fields — the added fields were advertised by props()
but never landed in the parsed value: every value parsed to `{}`.
registry.parseInner now folds the fields a built-in base natively consumes
(obj -> props; interface -> props/get/call) INTO a base built from that class,
leaving only non-structural fields in the Extension local. Generic extensions
are excluded: their props reference type-parameter placeholders that must stay
in the local to resolve against the specialization scope.
Adds a regression test; all gin tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
A living record of how models perform on the gin integration eval (61 cases, 10 categories, distractor fns, re-prompt-on-diagnostics loop). Value-scored leaderboard (same 0.60/0.20/0.20 acc/cost/speed methodology as query) over a partial 5-model sweep, plus the failure analysis: the two cases that fail on every capable model are JUDGMENT cases (refuse an impossible request; resist a distractor fn), and `lambda` is the capability discriminator that separates the tiers. Mirrors query's MODELS.md structure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
…xamples flag
Coverage gap: the 4 original worked examples showed `new` only for scalars, so
object CONSTRUCTION had no demonstration — models fell back to
`new {type:{name:'X', extends:'obj'}}` with no props and silently dropped every
field (obj-discount-product, obj-build-person). Adds:
- Example 5 — composite `new`: build an object, copy unchanged fields, and the
note "reference a registered type by BARE name; do NOT re-add extends/props".
- Example 6 — `if` branching.
- Example 7 — `reduce` accumulation (fn lambda + initial).
All three validated end-to-end (parse + validate + run) before adding.
`GIN_EVAL_NO_EXAMPLES=1` drops all worked examples to measure their lift.
Measured on google/gemini-3-flash-preview, 4 seeds/condition, auto delivery:
no examples 31/61 · 4 examples ~55 · 7 examples ~57.25 (+2 vs 4).
The gain is concentrated in object construction — obj-discount-product 0/4->4/4,
obj-build-person 1/4->4/4 — with only single-seed noise flips elsewhere.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Adds a measured "How much the worked examples matter" section: no examples 31/61 (51%) → 4 examples 55.25 (91%) → 7 examples 57.25 (94%), 4 seeds/condition on gemini-3-flash-preview. Examples are load-bearing for the wire format (9 cases can't emit valid ExprDef without them); the 3 added examples buy a targeted +2 concentrated in object construction (obj-discount-product 0/4→4/4, obj-build-person 1/4→4/4). Notes the leaderboard predates the additions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
A method with required parameters could be called without them — `n.lt` (no
{other}), `n.lt({})`, `text.replace` (no search/replacement) — and gin would
silently bind the missing params to nothing and return a WRONG value (`n.lt`
reads as a false bool; `replace` returns the string unchanged). Because nothing
errored, the eval's retry loop never saw it, producing programs that flip between
correct and silently-wrong across seeds (control-sign, text-slugify).
validateWalk now errors in three spots:
- a method + `{args:…}` step that omits a required param → `call.missing-arg`;
- a bare CALL (`fns.f({...})`) that omits a required param → `call.missing-arg`;
- a method with required params read WITHOUT any `{args:…}` step → `call.uncalled`
("needs arguments — add an {args:{…}} step to call it").
Auto-callable zero-required-arg methods (`opt.has`, `text.upper`) and correct
calls are unaffected; set-mode bare methods (which may route through call.set)
are left alone. Turns a silent-wrong answer into a diagnostic the retry loop can
fix. Updates the one test that codified the old lenient behavior.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
Ports query's typo-suggester (Levenshtein + length-scaled budget, fires only on a genuine misspelling) into gin as src/aids.ts, and appends it to every "unknown name" error so a model or human sees the fix inline: unknown variable 'titl' -> unknown variable 'titl' — did you mean `title`? no prop 'titl' on type 'obj' -> … — did you mean `title`? registry.parse: unknown type 'tex' -> … — did you mean `text`? Wired sites (suggestion appended, no code/message otherwise changed): - path.ts validate: var.unknown (scope var names), prop.unknown (type prop names) - path.ts runtime throws: unknown variable, no prop (added Scope.names()) - registry.ts parse throws: unknown type / extends / satisfies (typeNameCandidates = built-in class names + registered named types) New src/aids.ts exports didYouMean/nearest/editDistance/suggestionBudget; algorithm is byte-identical to @aeye/query. Adds did-you-mean-suggestions.test.ts (10 tests: helper units + end-to-end via engine.validate, incl. a no-false-positive case). typecheck 0; full gin suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… SQL converter
New standalone package (packages/query):
100% test coverage (932 tests, base+postgres SQL + runtime), 0 typecheck errors, no any/unknown/casts. Wires packages/query into the workspace (root package.json, tsconfig.base.json).
Claude-Session: https://claude.ai/code/session_012aWFXmP2zN1wnX43EwRjBS