feat: add kimi-k2.6 and harden structured output#40
Conversation
|
I also received the OutputParserException with the Gemma model. Installed this version of the package. And can confirm it works out of the box and directly fixes the issue 🚀 |
|
One questions. Does this significantly increase system message token usage compared to other models? |
|
Yes, it does add tokens — in my testing ~3x more input tokens (145 vs 466 for a typical nested schema). Here's why: by default I automatically inject the full JSON schema into a system message so Gemma knows exactly what fields to return. This is what makes it reliable. If you want to reduce token usage you can pass llm.with_structured_output(Data, method="json_mode").invoke(
"Extract announcements and return JSON with fields: type, context, entities (name, ticker, role)"
)Just note that this is essentially trading reliability for token efficiency — unreliable field coverage is the original problem this fix was solving. |
7880ddf to
7d0d819
Compare
7d0d819 to
d9765c3
Compare
Summary
@cf/moonshotai/kimi-k2.6to the tested model matrices and Worker example supported-model listmethod="json_schema"a first-class option, fixing prompt-value schema injection, and normalizing dict-shaped model responses before constructingAIMessagecontent~/.nvmNode runtime when the systemnodeis too old for Wrangler 4langchain-cloudflareto0.3.4and update the changelogTest plan
cd libs/langchain-cloudflare && make testcd libs/langchain-cloudflare && set -a && source ../../.env && set +a && uv run pytest tests/integration_tests/test_workersai_models.py -v -scd libs/langchain-cloudflare && set -a && source ../../.env && set +a && uv run pytest tests/integration_tests/test_worker_integration.py -v -scd libs/langchain-cloudflare && make lintpre-commit run --all-filesValidation notes
make test:101 passed, 2 skipped145 passed, 6 skipped, 4 failedTestWorkerChat::test_chat_batch[@cf/mistralai/mistral-small-3.1-24b-instruct]TestWorkerStructuredOutputBatch::test_structured_output_batch[@cf/openai/gpt-oss-20b]TestWorkerMultiModal::test_multi_modal_image[@cf/mistralai/mistral-small-3.1-24b-instruct]TestWorkerSessionAffinity::test_session_affinity_basictest_structured_output_json_schema_method_invoke[@cf/moonshotai/kimi-k2.6]failed once, but the isolated rerun passed immediately, so it appears flaky rather than a deterministic K2.6 capability gap.