Skip to content

feat: add kimi-k2.6 and harden structured output#40

Merged
CollierKing merged 1 commit intomainfrom
fix/gemma-structured-output
Apr 21, 2026
Merged

feat: add kimi-k2.6 and harden structured output#40
CollierKing merged 1 commit intomainfrom
fix/gemma-structured-output

Conversation

@CollierKing
Copy link
Copy Markdown
Collaborator

@CollierKing CollierKing commented Apr 16, 2026

Summary

  • add @cf/moonshotai/kimi-k2.6 to the tested model matrices and Worker example supported-model list
  • harden structured output handling by making method="json_schema" a first-class option, fixing prompt-value schema injection, and normalizing dict-shaped model responses before constructing AIMessage content
  • remove the older llama vision variants from general test coverage and keep the integration model lists aligned with the current supported set
  • harden Vectorize readiness and namespace search tests for eventual consistency
  • make Worker test startup reliable on local machines by using non-interactive Wrangler startup and preferring a modern ~/.nvm Node runtime when the system node is too old for Wrangler 4
  • bump langchain-cloudflare to 0.3.4 and update the changelog

Test plan

  • cd libs/langchain-cloudflare && make test
  • cd libs/langchain-cloudflare && set -a && source ../../.env && set +a && uv run pytest tests/integration_tests/test_workersai_models.py -v -s
  • cd libs/langchain-cloudflare && set -a && source ../../.env && set +a && uv run pytest tests/integration_tests/test_worker_integration.py -v -s
  • cd libs/langchain-cloudflare && make lint
  • pre-commit run --all-files

Validation notes

  • make test: 101 passed, 2 skipped
  • Full Worker integration file: 145 passed, 6 skipped, 4 failed
  • The remaining Worker failures were not K2.6-specific:
    • TestWorkerChat::test_chat_batch[@cf/mistralai/mistral-small-3.1-24b-instruct]
    • TestWorkerStructuredOutputBatch::test_structured_output_batch[@cf/openai/gpt-oss-20b]
    • TestWorkerMultiModal::test_multi_modal_image[@cf/mistralai/mistral-small-3.1-24b-instruct]
    • TestWorkerSessionAffinity::test_session_affinity_basic
  • During one long REST full-file run, test_structured_output_json_schema_method_invoke[@cf/moonshotai/kimi-k2.6] failed once, but the isolated rerun passed immediately, so it appears flaky rather than a deterministic K2.6 capability gap.

@theobouwman
Copy link
Copy Markdown

I also received the OutputParserException with the Gemma model.

Installed this version of the package. And can confirm it works out of the box and directly fixes the issue 🚀

@theobouwman
Copy link
Copy Markdown

One questions. Does this significantly increase system message token usage compared to other models?

@CollierKing
Copy link
Copy Markdown
Collaborator Author

Yes, it does add tokens — in my testing ~3x more input tokens (145 vs 466 for a typical nested schema).

Here's why: by default I automatically inject the full JSON schema into a system message so Gemma knows exactly what fields to return. This is what makes it reliable.

If you want to reduce token usage you can pass method="json_mode" to with_structured_output. In this mode I don't inject anything — Gemma just knows it needs to return valid JSON, but has no idea what fields you want. That means you're responsible for communicating the structure yourself in your prompt, for example:

llm.with_structured_output(Data, method="json_mode").invoke(
    "Extract announcements and return JSON with fields: type, context, entities (name, ticker, role)"
)

Just note that this is essentially trading reliability for token efficiency — unreliable field coverage is the original problem this fix was solving.

@CollierKing CollierKing changed the title fix: use json_object mode for Gemma structured output feat: add method="json_schema" to with_structured_output() and expand vision model support Apr 19, 2026
@CollierKing CollierKing force-pushed the fix/gemma-structured-output branch from 7880ddf to 7d0d819 Compare April 21, 2026 18:50
@CollierKing CollierKing changed the title feat: add method="json_schema" to with_structured_output() and expand vision model support feat: add kimi-k2.6 and harden structured output Apr 21, 2026
@CollierKing CollierKing force-pushed the fix/gemma-structured-output branch from 7d0d819 to d9765c3 Compare April 21, 2026 18:52
@CollierKing CollierKing merged commit a9ebdde into main Apr 21, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants