feat: add langfuse plugin for LLM observability#13095
Open
sihyeonn wants to merge 6 commits intoapache:masterfrom
Open
feat: add langfuse plugin for LLM observability#13095sihyeonn wants to merge 6 commits intoapache:masterfrom
sihyeonn wants to merge 6 commits intoapache:masterfrom
Conversation
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Add auth.source option to ai-proxy and ai-proxy-multi plugins:
- source: "config" (default) - use auth values from plugin config directly
- source: "consumer_label" - lookup auth values from consumer labels
This allows per-consumer API keys for AI providers like OpenAI, Gemini, etc.
Example usage:
- Consumer labels: {"openai_api_key": "Bearer sk-xxx"}
- Plugin config: {"auth": {"source": "consumer_label", "header": {"Authorization": "openai_api_key"}}}
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Consumer labels ending with '_secret' suffix are now automatically encrypted when stored in etcd and decrypted when loaded. This enables secure storage of sensitive data like API keys in consumer labels without external secret management infrastructure. Example: - Label "openai_api_key_secret" will be encrypted in etcd - Decrypted automatically when consumer is loaded Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Add a new langfuse plugin that sends AI request traces to Langfuse via its ingestion API using batch-processor-manager pattern. Features: - Auto-detect AI endpoints (configurable patterns) - Generate trace-create and generation-create batch items per request - W3C traceparent header parsing and propagation - Token usage extraction (ai-proxy ctx > nginx vars > response body) - Support for X-Langfuse-Tags and X-Langfuse-Metadata headers - Basic auth with encrypt_fields for secret key protection - Connection keepalive for Langfuse API calls Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
dosubot
bot
added
size:XXL
Move langfuse_host, langfuse_public_key, langfuse_secret_key, ssl_verify, timeout, detect_ai_requests, and ai_endpoints from per-route schema to metadata_schema (plugin_attr), following the opentelemetry plugin pattern. Per-route schema now only contains include_metadata. Remove encrypt_fields (custom feature, not in OSS). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Remove /v1-prefixed entries since has_suffix matching already covers them via shorter suffixes (e.g. /chat/completions matches both /chat/completions and /v1/chat/completions). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Contributor
|
This PR contains three fundamentally different features:
Apache APISIX community practice requires each PR to focus on a single feature. These three features have different review dimensions, testing requirements, and regression risks; please separate them into independent PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a new
langfuseplugin that sends LLM request traces to Langfuse for AI observability. The plugin uses the batch-processor-manager pattern (same as http-logger) to send trace data via Langfuse's ingestion API.Key features:
trace-createandgeneration-createbatch items per requesttraceparentheader parsing and propagation for distributed tracingctx.ai_token_usage(ai-proxy) > nginx vars > response body parsingX-Langfuse-TagsandX-Langfuse-Metadatacustom headersencrypt_fieldsfor secret key protection in etcdWhich issue(s) this PR fixes:
Fixes #
Checklist