feat(ai-proxy): support OpenAI Chat Completions clients with native Anthropic Messages API upstreams#13569
Open
GeorgelPreput wants to merge 1 commit into
Open
Conversation
…nthropic Messages API upstreams ai-proxy/ai-proxy-multi could translate Anthropic-format clients to OpenAI-compatible upstreams, but not the reverse. There was no way to accept an OpenAI Chat Completions request and forward it to an upstream that only speaks the native Anthropic Messages API (/v1/messages), such as Azure AI Foundry Claude, which exposes no OpenAI-compatible endpoint. Add the missing direction (non-streaming): - New `anthropic-compatible` provider exposing only the `anthropic-messages` capability. The existing `anthropic` provider declares both `openai-chat` and `anthropic-messages`, so OpenAI clients always hit native passthrough and never reach a converter; a provider with only the Messages capability lets the converter run. This mirrors how `openai-compatible` is the vendor-neutral counterpart used as a conversion target. - New `openai-chat` -> `anthropic-messages` converter: request (system role to top-level `system`, tools, tool_choice, tool messages to user tool_result blocks, images, reasoning_effort to thinking, max_tokens), non-streaming response (content/tool_use/thinking to choices, stop reason and usage mapping including cache tokens), and headers. - Streaming (stream=true) is rejected with a clear error for now; it is left as a follow-up since it requires the Anthropic Messages SSE adapter to pass raw event data through to the converter. - Tests (t/plugin/ai-proxy-openai-to-anthropic.t) and docs. Ref: apache#13566 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Georgel Preput <georgelpreput@mailbox.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ai-proxy/ai-proxy-multican translate Anthropic-format clients to OpenAI-compatible upstreams (via theanthropic-messages-to-openai-chatconverter), but not the reverse. There was no way to accept an OpenAI Chat Completions request and forward it to an upstream that only speaks the native Anthropic Messages API (/v1/messages) — e.g. Azure AI Foundry Claude, whose only surface ishttps://<resource>.services.ai.azure.com/anthropic/v1/messages(no OpenAI-compatible/chat/completions). Today an OpenAI body sent through theanthropicprovider is passed through unchanged and rejected by the upstream (messages.0: use the top-level 'system' parameter ...).This PR adds the missing direction (non-streaming).
Changes
anthropic-compatibleprovider exposing only theanthropic-messagescapability. This is the crux:ai-proxy/base.luaselects native passthrough whenever the provider declares a capability matching the client protocol, before looking for a converter. The existinganthropicprovider declares bothopenai-chatandanthropic-messages, so OpenAI clients always passthrough and never reach a converter. A provider exposing only the Messages capability lets the converter run — mirroring howopenai-compatibleis the vendor-neutral counterpart used as a conversion target.openai-chat→anthropic-messagesconverter:systemrole → top-levelsystem;tools/tool_choice;toolmessages → usertool_resultblocks (coalesced); image parts;reasoning_effort→thinking;max_tokens/max_completion_tokens→max_tokens(defaulted since Anthropic requires it).tool_use/thinking→choices[].message; stop-reason and usage mapping (incl. cache tokens) → OpenAI shape.anthropic-version, strips OpenAI/stainless telemetry.stream: true) is rejected with a clear error for now (see follow-ups).t/plugin/ai-proxy-openai-to-anthropic.t, unit + e2e) and docs (ai-proxy.md,ai-proxy-multi.md, en + zh).Verified end-to-end against a live Azure AI Foundry Claude deployment via
ai-proxy-multi+anthropic-compatible: basic chat (with asystemprompt), tool-call requests (tool_use→tool_calls), and a multi-turn tool-result round-trip all returned correct OpenAI-shaped responses;stream: truereturned a clean 400.Open questions for maintainers (documented as potential follow-ups rather than decided here):
stream: truein this direction needs the Anthropic Messages SSE adapter (ai-protocols/anthropic-messages.luaparse_sse_event) to pass raw event data through to the converter (it currently digests/dropstool_useandfinish_reasondetail). Happy to do this in a follow-up — kept this PR focused to get directional feedback first.Authorization: Bearer <key>tox-api-key. This works for Azure/Anthropic key-based auth, but would break a Microsoft Entra ID bearer token. Options: make the rewrite conditional, drop it and rely solely onauth.header, or document it. Guidance welcome.Which issue(s) this PR fixes:
Partially addresses #13566 (non-streaming; streaming tracked as a follow-up).
Checklist