BP-69: Print slog context attrs in log4j2; add OTel JSON example#4770
Open
merlimat wants to merge 3 commits intoapache:masterfrom
Open
BP-69: Print slog context attrs in log4j2; add OTel JSON example#4770merlimat wants to merge 3 commits intoapache:masterfrom
merlimat wants to merge 3 commits intoapache:masterfrom
Conversation
Slog attributes are forwarded to log4j2 as ContextData (the same channel SLF4J's MDC uses). Add %X to every PatternLayout in the user-facing log4j2 configs so the structured attributes attached via .attr(...) appear in log output by default. Drop the now-redundant *MDC variant appenders and the directentrylogger override that targeted them. Also add conf/log4j2.otel-json.xml plus conf/otel-log-template.json as an example layout that emits each event as a JSON object aligned with the OpenTelemetry log data model (severity_text, severity_number, body, attributes, resource, instrumentation_scope, trace_id/span_id from MDC). The example uses log4j-layout-template-json, which must be added to the classpath alongside log4j-core to enable JsonTemplateLayout.
Replace the previous template with the structure used by Pulsar
(apache/pulsar/blob/master/conf/OtelLogLayout.json):
* Top-level fields use the spec's PascalCase naming (Timestamp,
SeverityText, Body, TraceId, SpanId, TraceFlags, Attributes).
* thread.name, thread.id, code.namespace, exception.{type,message,
stacktrace}, and the flattened MDC are placed under "Attributes",
using OTel semantic-convention keys.
* Drop SeverityNumber, ObservedTimestamp, InstrumentationScope, and
Resource — the first two are derivable / redundant for app-emitted
logs, the logger name is "code.namespace" not InstrumentationScope,
and Resource is normally attached by the collector pipeline.
* The MDC resolver with flatten=true now sits inside Attributes, so
slog .attr(...) keys merge as siblings of thread.name etc. (the
previous placement at the top level didn't merge into "attributes"
as intended).
Update the explanatory comment block and drop the now-unused
otel.service.name / otel.host.name properties.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
%Xto every PatternLayout in the user-facing log4j2 configs (conf/,bookkeeper-server/src/main/resources/,stream/conf/,bookkeeper-benchmark/conf/) so that the structured attrs attached via.attr(...)show up in log output by default.*MDCvariant appenders inconf/log4j2.xmland thedirectentryloggeroverride that targeted them — every appender renders the context map now.conf/log4j2.otel-json.xmlplusconf/otel-log-template.jsonas an example layout that emits each event as a single JSON object aligned with the OpenTelemetry log data model:timestamp,severity_text,severity_number,body,instrumentation_scope.name,resource.{service.name,host.name},attributes(the full ContextData map),trace_id/span_id/trace_flagsfrom MDC, plusexception.{type,message,stack_trace}.JsonTemplateLayout, which requiresorg.apache.logging.log4j:log4j-layout-template-jsonon the classpath at runtime (not bundled by default — the example header notes this).Test plan
xmllint --nooutvalidates every modified XML config.python3 -m json.toolvalidatesconf/otel-log-template.json.