feat(aio): declare session id and per-turn traceparent to the llm gateway - #3927
Conversation
Generated-By: PostHog Code Task-Id: 1119b267-6fcd-4dba-bc80-f15665f7a9b1
Generated-By: PostHog Code Task-Id: 1119b267-6fcd-4dba-bc80-f15665f7a9b1
Generated-By: PostHog Code Task-Id: 1119b267-6fcd-4dba-bc80-f15665f7a9b1
|
😎 Merged successfully - details. |
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
| OTEL_TRACES_EXPORTER: process.env.OTEL_TRACES_EXPORTER ?? "otlp", | ||
| OTEL_EXPORTER_OTLP_PROTOCOL: | ||
| process.env.OTEL_EXPORTER_OTLP_PROTOCOL ?? "http/json", |
There was a problem hiding this comment.
Inherited settings disable turn tracing
When a gateway-routed Claude session inherits OTEL_TRACES_EXPORTER=none or an incompatible OTEL_EXPORTER_OTLP_PROTOCOL, this block preserves that configuration even though tracer initialization is required to stamp traceparents, causing generations to remain untraced or incorrectly grouped.
Knowledge Base Used: Agent SDK (packages/agent)
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/agent/src/adapters/claude/session/options.ts
Line: 202-204
Comment:
**Inherited settings disable turn tracing**
When a gateway-routed Claude session inherits `OTEL_TRACES_EXPORTER=none` or an incompatible `OTEL_EXPORTER_OTLP_PROTOCOL`, this block preserves that configuration even though tracer initialization is required to stamp traceparents, causing generations to remain untraced or incorrectly grouped.
**Knowledge Base Used:** [Agent SDK (`packages/agent`)](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/code/-/docs/agent-sdk.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Prompt To Fix All With AI### Issue 1
packages/agent/src/adapters/claude/session/options.ts:202-204
**Inherited settings disable turn tracing**
When a gateway-routed Claude session inherits `OTEL_TRACES_EXPORTER=none` or an incompatible `OTEL_EXPORTER_OTLP_PROTOCOL`, this block preserves that configuration even though tracer initialization is required to stamp traceparents, causing generations to remain untraced or incorrectly grouped.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "feat: forward the task id as the codex a..." | Re-trigger Greptile |
| // leaking to a local collector on the default port. Export never matters. | ||
| const gatewayTracing: Record<string, string> = gateway?.anthropicBaseUrl | ||
| ? { | ||
| CLAUDE_CODE_ENABLE_TELEMETRY: "1", |
There was a problem hiding this comment.
Medium: Repository settings can exfiltrate tool content through telemetry
Project .claude/settings.json files are loaded by default and can define environment variables. A malicious repository can set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (which takes precedence over the dead generic endpoint) and enable tool-content attributes; this new telemetry switch then exports tool inputs and outputs, including source code read by the agent, to the attacker's collector. Pin or strip the signal-specific OTel endpoints, exporters, headers, and content-enabling variables after project settings are resolved, or prevent repository settings from configuring telemetry for these embedded sessions.
There was a problem hiding this comment.
Isn't this a pre-existing issue? You just looked at this code now, but nothing was stopping anyone before from just setting ALL of these env vars on the JSON file and exfiltrating data right? I repro'd it locally on main.
Also the attack vector for this is a compromised repo that gets trusted by Claude Code, in which case there's a million different ways to be evil, like scripts, skills, etc. Am I missing something here that I caused?
There was a problem hiding this comment.
Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.
PR overviewThis PR adds session ID and per-turn traceparent propagation to the LLM gateway and updates Claude session telemetry options to support the new tracing behavior. One security issue remains open. A malicious repository can use project-level Claude settings to redirect telemetry and include tool inputs and outputs, potentially exposing source code and other agent-accessed content to an attacker-controlled collector. Open issues (1)
Fixed/addressed: 0 · PR risk: 7/10 |
tatoalo
left a comment
There was a problem hiding this comment.
agree on veria comment but stamping!
Generated-By: PostHog Code Task-Id: 409fb75d-ab0b-4dca-b960-df1f721982c1
|
👋 Visual changes detected for this PR. Review and approve in PostHog Visual Review If these changes are unexpected, they may be caused by a flaky test or a broken snapshot on master. Don't approve — rerun the job or wait for a fix. |
Problem
Every
$ai_generationa PostHog Code task produces lands mis-shaped in AI Observability: the Claude path collapses a whole task (observed: 1,500+ generations over 45h) into one$ai_trace_id, the codex path scatters each LLM call into its own singleton trace, and$ai_session_idis null on 100% of events — so the AIO sessions UI shows nothing for Code.Changes
packages/agent/.../options.ts):buildEnvironmentnow emitsx-posthog-property-$ai_session_id: <ACP session id>inANTHROPIC_CUSTOM_HEADERS(resume keeps the id, fork mints a new one). For gateway-routed sessions only, it sets the four env vars that make CLI 2.1.197 stamp a per-turn W3Ctraceparentheader (verified locally: trace-id rotates per user turn, survives a dead OTLP endpoint), and strips inheritedTRACEPARENT/TRACESTATEso ambient CI/sandbox trace context can't collapse turns. BYOK sessions byte-identical.$ai_session_id: <task id>— cloud viaconfigureEnvironment'sopenaiCustomHeaders, desktop viacodexOptions.httpHeaders(previously unset). Record form only; the Claude header string never gains the key (test-locked against double-send). Codex has no per-turn signal, so its traces stay per-call; the session is its task-level grouping.Inert until the gateway counterpart (PostHog/posthog
feat(aio)/llm-gateway-turn-traces) deploys — today's gateway ignores these headers. Once both are live, posthog_code trace cardinality rises ~5-20× on the Claude path (dashboards treating trace=task should be audited) and Code tasks appear in the AIO sessions UI.How did you test this?
Automated only: 6 new/updated Vitest cases — session-header ordering in
ANTHROPIC_CUSTOM_HEADERS, tracing vars gated on gateway routing, userOTEL_*overrides honored, ambientTRACEPARENTstripped, BYOK untouched, codex header present/absent by task id, TOML$-key serialization. Full@posthog/agentsuite 1665/1665, typecheck clean, Biome warnings unchanged from base. The traceparent flag recipe itself was verified against the real pinned CLI run in stream-json mode against a header-logging fake gateway (two turns → two trace-ids, shared within turn). End-to-end SQL/UI verification (two-prompt repro) deferred to rollout — needs the gateway change deployed.Automatic notifications
Created with PostHog Code