Skip to content

feat(ai-proxy-multi): add anthropic-native provider for native Anthropic Messages API#13050

Open
iakuf wants to merge 1 commit intoapache:masterfrom
iakuf:feat/ai-proxy-multi-anthropic-native
Open

feat(ai-proxy-multi): add anthropic-native provider for native Anthropic Messages API#13050
iakuf wants to merge 1 commit intoapache:masterfrom
iakuf:feat/ai-proxy-multi-anthropic-native

Conversation

@iakuf
Copy link

@iakuf iakuf commented Feb 28, 2026

Summary

Add a new anthropic-native driver that speaks the native Anthropic Messages API protocol (/v1/messages) directly.

The existing anthropic provider uses the OpenAI-compatible driver, which works for basic use cases but cannot handle Anthropic-specific protocol details. The new anthropic-native provider addresses this.

Key differences from openai-base driver

  • SSE event types: handles message_start, content_block_delta, message_delta, message_stop (no [DONE] sentinel)
  • Token usage fields: reads input_tokens/output_tokens instead of prompt_tokens/completion_tokens
  • Request format: removes stream_options (not supported by Anthropic), injects anthropic-version header automatically
  • Response format: extracts text from content[].text array instead of choices[].message.content
  • System prompt: accepts top-level system field and flexible content types (string or array of content blocks)

Changes

  • apisix/plugins/ai-drivers/anthropic-native.lua: new driver implementation
  • apisix/plugins/ai-drivers/schema.lua: register anthropic-native provider with its own request schema
  • t/assets/anthropic-native-response.json: mock response fixture for tests
  • t/plugin/ai-proxy-multi.anthropic-native.t: 26 test cases covering non-streaming, streaming, token usage, TTFT, error handling, and schema validation
  • docs/en/latest/plugins/ai-proxy-multi.md: document the new provider with a usage example

Testing

All 26 tests pass:

…pic Messages API

Add a new anthropic-native driver that speaks the native Anthropic Messages
API protocol (/v1/messages) directly, as opposed to the existing anthropic
provider which uses the OpenAI-compatible driver.

Key differences from the openai-base driver:
- Handles Anthropic-specific SSE event types: message_start,
  content_block_delta, message_delta, message_stop (no [DONE] sentinel)
- Reads token usage from input_tokens/output_tokens fields
- Removes stream_options from requests (not supported by Anthropic)
- Injects anthropic-version header automatically
- Extracts response text from content[].text array

Also updates schema.lua to register the new provider and its request
schema (which allows top-level system field and flexible content types),
and adds documentation with a usage example.
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. enhancement New feature or request labels Feb 28, 2026
end


return _M
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the pattern of anthropic.lua or vertex-ai.lua, make the module entry point return openai-base.new(...) or an equivalent instance object.

@@ -0,0 +1,399 @@
--
-- anthropic-native.lua
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requires the inclusion of the Apache Software Foundation standard license header.

@Baoyuantop Baoyuantop requested a review from Copilot March 12, 2026 06:44
@Baoyuantop Baoyuantop added the wait for update wait for the author's response in this issue/PR label Mar 12, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new anthropic-native AI driver to ai-proxy-multi so APISIX can proxy directly to Anthropic’s native Messages API (/v1/messages) rather than using the OpenAI-compatible protocol, including native SSE event handling and token usage extraction.

Changes:

  • Introduces a new anthropic-native driver implementing native request/response and SSE semantics.
  • Registers anthropic-native in the AI drivers schema with a dedicated (minimal) request schema.
  • Adds tests, fixtures, and documentation for the new provider.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
apisix/plugins/ai-drivers/anthropic-native.lua New native Anthropic Messages API driver (request shaping, SSE parsing, usage/TTFT extraction).
apisix/plugins/ai-drivers/schema.lua Adds anthropic-native to provider registry and assigns it a request schema.
t/plugin/ai-proxy-multi.anthropic-native.t Adds an extensive test suite covering non-streaming, streaming SSE, usage, and error handling.
t/assets/anthropic-native-response.json Adds a mock native Anthropic response fixture used by tests.
docs/en/latest/plugins/ai-proxy-multi.md Documents the new provider and adds a usage example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

## Description

The `ai-proxy-multi` Plugin simplifies access to LLM and embedding models by transforming Plugin configurations into the designated request format for OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs. It extends the capabilities of [`ai-proxy`](./ai-proxy.md) with load balancing, retries, fallbacks, and health checks.
The `ai-proxy-multi` Plugin simplifies access to LLM and embedding models by transforming Plugin configurations into the designated request format for OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, Anthropic-native, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs. It extends the capabilities of [`ai-proxy`](./ai-proxy.md) with load balancing, retries, fallbacks, and health checks.
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation clarity: this section lists providers using brand names, but "Anthropic-native" is the config key and is shown elsewhere as anthropic-native. Consider using consistent formatting (e.g., backticks and lowercase anthropic-native) to avoid confusion about the exact provider value.

Copilot uses AI. Check for mistakes.
Comment on lines +314 to +316
local scheme = parsed_url and parsed_url.scheme or "https"
local host = parsed_url and parsed_url.host or self.host or "api.anthropic.com"
local port = parsed_url and parsed_url.port
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anthropic-native is required to work with the ai-proxy-multi health-check / endpoint resolution logic. ai-proxy-multi.resolve_endpoint() falls back to reading ai_driver.host/ai_driver.port when override.endpoint is not set, but this driver module doesn't define those fields (unlike the other providers). This will make endpoint resolution/health checks fail when users rely on the default Anthropic endpoint. Define host, port (and ideally path) on the exported driver object (or return an instantiated driver like the other providers) so the default endpoint can be resolved consistently.

Copilot uses AI. Check for mistakes.
- AI
- LLM
description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health chekcs, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.
description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health chekcs, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, Anthropic-native, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling: "health chekcs" should be "health checks" (also appears in the page description metadata).

Suggested change
description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health chekcs, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, Anthropic-native, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.
description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health checks, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, Anthropic-native, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.

Copilot uses AI. Check for mistakes.
Copy link
Member

@moonming moonming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @iakuf, thank you for the substantial work on the anthropic-native provider! This is a significant contribution (1233 lines).

Context: There's an ongoing architectural discussion about how to support Anthropic's API:

  • This PR (#13050): Adds a native Anthropic driver that speaks the Anthropic Messages API directly
  • #13048: Takes a protocol translation approach (Claude ↔ OpenAI format conversion)

Both approaches have merit:

  • Native driver: More efficient, preserves Anthropic-specific features (tool use, vision, etc.)
  • Protocol translation: Simpler to maintain, single code path for request/response handling

Before we can proceed, we need to decide on the technical direction. I'll make a decision on this and update both PRs. In the meantime, could you:

  1. Address any existing review comments
  2. Ensure tests cover the key Anthropic API features (streaming, tool use, vision)
  3. Rebase if needed

Thank you for your patience while we sort out the architecture! Your implementation quality is appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XXL This PR changes 1000+ lines, ignoring generated files. wait for update wait for the author's response in this issue/PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants