feat(writer): implement native writer [APMSP-1875] #13071

VianneyRuhlmann · 2025-04-04T14:37:21Z

TLDR

This PR adds a new Writer which uses the trace exporter from libdatadog to send traces to the agent.

Description

Add PyO3 bindings to expose the rust TraceExporter to python
Add a new NativeWriter using the bindings
Add a config option to toggle between the AgentWriter and the NativeWriter
Update tests accessing private methods/attributes of the writer to support both implementations
Register fork hooks for the writer required as the trace exporter rely on a multi-threaded runtime

Motivation

This PR will allow dogfooding of the Trace Exporter in dd-trace-py

Testing

The full test suite has been run on with NativeWriter enable by default, when it is disabled only writer tests and integration tests are run on the NativeWriter.

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

ddtrace/internal/writer/writer.py

github-actions · 2025-05-28T12:29:02Z

CODEOWNERS have been resolved as:

.riot/requirements/1550212.txt                                          @DataDog/apm-python
src/native/data_pipeline/exceptions.rs                                  @DataDog/apm-core-python
src/native/data_pipeline/mod.rs                                         @DataDog/apm-core-python
.gitlab/benchmarks/bp-runner.microbenchmarks.fail-on-breach.yml         @DataDog/python-guild @DataDog/apm-core-python
ddtrace/_trace/processor/__init__.py                                    @DataDog/apm-sdk-api-python
ddtrace/_trace/tracer.py                                                @DataDog/apm-sdk-api-python
ddtrace/internal/native/__init__.py                                     @DataDog/apm-core-python
ddtrace/internal/native/_native.pyi                                     @DataDog/apm-core-python
ddtrace/internal/writer/__init__.py                                     @DataDog/apm-core-python
ddtrace/internal/writer/writer.py                                       @DataDog/apm-core-python
ddtrace/settings/_config.py                                             @DataDog/apm-core-python
riotfile.py                                                             @DataDog/apm-python
src/native/Cargo.lock                                                   @DataDog/apm-core-python
src/native/Cargo.toml                                                   @DataDog/apm-core-python
src/native/lib.rs                                                       @DataDog/apm-core-python
tests/contrib/kafka/test_kafka.py                                       @DataDog/apm-core-python @DataDog/apm-idm-python
tests/integration/test_debug.py                                         @DataDog/apm-core-python
tests/integration/test_integration.py                                   @DataDog/apm-core-python
tests/integration/test_integration_snapshots.py                         @DataDog/apm-core-python
tests/integration/test_priority_sampling.py                             @DataDog/apm-sdk-api-python
tests/integration/test_trace_stats.py                                   @DataDog/apm-core-python
tests/integration/utils.py                                              @DataDog/apm-core-python
tests/suitespec.yml                                                     @DataDog/python-guild @DataDog/apm-core-python
tests/telemetry/test_writer.py                                          @DataDog/apm-python
tests/tracer/test_processors.py                                         @DataDog/apm-sdk-api-python
tests/tracer/test_writer.py                                             @DataDog/apm-sdk-api-python
tests/utils.py                                                          @DataDog/python-guild

github-actions · 2025-06-03T09:23:14Z

Circular import analysis

🚨 New circular imports detected 🚨

The following circular imports among modules have been detected on this PR, when compared to the base branch:

ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches -> ddtrace.contrib.internal.subprocess.patch -> ddtrace._trace.pin -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._publishers -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.filters -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.processor.endpoint_call_counter -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.schema.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.internal.peer_service.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.trace
ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig.client -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.trace
ddtrace -> ddtrace.trace -> ddtrace._trace.tracer -> ddtrace.appsec._iast.processor -> ddtrace._trace.processor -> ddtrace.internal.writer.writer -> ddtrace.appsec._remoteconfiguration -> ddtrace.internal.remoteconfig.worker -> ddtrace.internal.remoteconfig._pubsub -> ddtrace.internal.remoteconfig._subscribers -> ddtrace

Please consider refactoring your changes in accordance to the Separation of Concerns principle.

github-actions · 2025-06-11T13:25:08Z

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 267 ± 6 ms.

The average import time from base is: 267 ± 5 ms.

The import time difference between this PR and base is: -0.4 ± 0.2 ms.

The difference is not statistically significant (z = -1.92).

Import time breakdown

The following import paths have grown:

ddtrace.auto 1.078 ms (0.40%)

ddtrace 1.078 ms (0.40%)

ddtrace.trace 0.620 ms (0.23%)

ddtrace._trace.filters 0.503 ms (0.19%)

ddtrace._trace.processor 0.503 ms (0.19%)

ddtrace.internal.writer 0.399 ms (0.15%)

ddtrace.internal.writer.writer 0.399 ms (0.15%)

ddtrace.settings.asm 0.107 ms (0.04%)

ddtrace.internal.dogstatsd 0.101 ms (0.04%)

ddtrace.vendor.dogstatsd 0.101 ms (0.04%)

ddtrace.vendor.dogstatsd.base 0.101 ms (0.04%)

queue 0.101 ms (0.04%)

heapq 0.101 ms (0.04%)

ddtrace._logger 0.458 ms (0.17%)

ddtrace.internal.telemetry 0.458 ms (0.17%)

ddtrace.internal.telemetry.writer 0.306 ms (0.11%)

http.client 0.306 ms (0.11%)

ssl 0.306 ms (0.11%)

ddtrace.settings._agent 0.152 ms (0.06%)

ddtrace.settings._core 0.152 ms (0.06%)

ddtrace.internal.native 0.152 ms (0.06%)

ddtrace.internal.native._native 0.152 ms (0.06%)

The following import paths have shrunk:

ddtrace.auto 2.538 ms (0.95%)

ddtrace.bootstrap.sitecustomize 1.304 ms (0.49%)

ddtrace.bootstrap.preload 1.200 ms (0.45%)

ddtrace.internal.remoteconfig.client 0.618 ms (0.23%)

ddtrace.bootstrap 0.104 ms (0.04%)

ddtrace 1.234 ms (0.46%)

ddtrace.trace 0.302 ms (0.11%)

ddtrace._trace.filters 0.198 ms (0.07%)

ddtrace._trace.processor 0.198 ms (0.07%)

ddtrace.internal.writer 0.198 ms (0.07%)

ddtrace.internal.writer.writer 0.198 ms (0.07%)

ddtrace.settings.asm 0.106 ms (0.04%)

ddtrace.appsec._constants 0.106 ms (0.04%)

ddtrace.internal.dogstatsd 0.092 ms (0.03%)

ddtrace.vendor.dogstatsd 0.092 ms (0.03%)

ddtrace.vendor.dogstatsd.base 0.092 ms (0.03%)

queue 0.092 ms (0.03%)

heapq 0.092 ms (0.03%)

_heapq 0.092 ms (0.03%)

ddtrace._trace.pin 0.104 ms (0.04%)

ddtrace._logger 0.263 ms (0.10%)

ddtrace.internal.telemetry 0.263 ms (0.10%)

ddtrace.internal.telemetry.writer 0.263 ms (0.10%)

http.client 0.263 ms (0.10%)

ssl 0.263 ms (0.10%)

_ssl 0.263 ms (0.10%)

ddtrace.internal._unpatched 0.027 ms (0.01%)

json 0.027 ms (0.01%)

json.decoder 0.027 ms (0.01%)

re 0.027 ms (0.01%)

enum 0.027 ms (0.01%)

types 0.027 ms (0.01%)

ddtrace/_trace/processor/__init__.py

ddtrace/internal/forksafe.py

wconti27

Changes are okay for apm-idm-python files 👍

riotfile.py

src/native/Cargo.toml

VianneyRuhlmann closed this Apr 9, 2025

VianneyRuhlmann reopened this Apr 9, 2025

datadog-datadog-prod-us1 bot reviewed Apr 22, 2025

View reviewed changes

ddtrace/internal/writer/writer.py Outdated Show resolved Hide resolved

VianneyRuhlmann force-pushed the vianney/add-native-writer branch from f6a7894 to 6da28c4 Compare April 23, 2025 12:26

VianneyRuhlmann force-pushed the vianney/add-native-writer branch from 23e676c to c369549 Compare April 30, 2025 15:32

VianneyRuhlmann changed the title ~~Add Native Writer~~ feat(writer): [APMSP-1875] implement native writer May 12, 2025

VianneyRuhlmann mentioned this pull request May 28, 2025

[APMSP-1961] Add fork support by dropping runtime DataDog/libdatadog#1056

Merged

VianneyRuhlmann force-pushed the vianney/add-native-writer branch from 086ed3d to 62bf790 Compare May 28, 2025 12:28

VianneyRuhlmann force-pushed the vianney/add-native-writer branch from 2ac64de to 089e2d3 Compare June 11, 2025 09:30

VianneyRuhlmann marked this pull request as ready for review June 13, 2025 11:44

VianneyRuhlmann requested review from a team as code owners June 13, 2025 11:44

VianneyRuhlmann requested review from avara1986, erikayasuda and rachelyangdog June 13, 2025 11:44

VianneyRuhlmann marked this pull request as draft June 13, 2025 11:45

VianneyRuhlmann self-assigned this Jun 13, 2025

VianneyRuhlmann force-pushed the vianney/add-native-writer branch 2 times, most recently from dcfcc90 to 486beb8 Compare July 1, 2025 12:03

brettlangdon changed the title ~~feat(writer): [APMSP-1875] implement native writer~~ chore(writer): implement native writer Jul 1, 2025

brettlangdon added the changelog/no-changelog A changelog entry is not required for this PR. label Jul 1, 2025

brettlangdon mentioned this pull request Jul 1, 2025

ci(crashtracker): refactor crash report listening logic #13839

Merged

2 tasks

P403n1x87 reviewed Jul 1, 2025

View reviewed changes

ddtrace/_trace/processor/__init__.py Outdated Show resolved Hide resolved

P403n1x87 reviewed Jul 1, 2025

View reviewed changes

ddtrace/_trace/processor/__init__.py Outdated Show resolved Hide resolved

P403n1x87 reviewed Jul 1, 2025

View reviewed changes

ddtrace/internal/forksafe.py Outdated Show resolved Hide resolved

P403n1x87 reviewed Jul 1, 2025

View reviewed changes

ddtrace/internal/forksafe.py Outdated Show resolved Hide resolved

VianneyRuhlmann closed this Aug 6, 2025

VianneyRuhlmann force-pushed the vianney/add-native-writer branch from c569fb4 to a158aea Compare August 6, 2025 12:32

VianneyRuhlmann reopened this Aug 6, 2025

VianneyRuhlmann added 9 commits August 6, 2025 14:42

feat(native): Add data-pipeline bindings

409bf54

feat(writer): Add NativeWriter

a40e17a

test(tracer): Add NativeWriter to tracer tests

d52378d

feat(tests): Add NativeWriter support to DummyWriter

efedd26

test(integration): Fix integration test for NativeWriter

136c552

fix(test): Adapt kafka tests to NativeWriter

4f7b49e

test(telemetry): Add NativeWriter configuration option

8b26855

test(integration): Run integration tests on the NativeWriter

b71383e

chore(benchmarks): Increase max_rss SLO

c816826

VianneyRuhlmann force-pushed the vianney/add-native-writer branch from c569fb4 to c816826 Compare August 6, 2025 13:36

VianneyRuhlmann changed the title ~~chore(writer): implement native writer~~ APMSP-1875 feat(writer): implement native writer Aug 6, 2025

VianneyRuhlmann changed the title ~~APMSP-1875 feat(writer): implement native writer~~ feat(writer): implement native writer [APMSP-1875] Aug 6, 2025

wconti27 approved these changes Aug 6, 2025

View reviewed changes

taegyunkim reviewed Aug 6, 2025

View reviewed changes

riotfile.py Show resolved Hide resolved

erikayasuda requested a review from brettlangdon August 7, 2025 14:23

chore(native): Bump to Libdatadog v20.0.0

bf73aad

brettlangdon reviewed Aug 7, 2025

View reviewed changes

src/native/Cargo.toml Outdated Show resolved Hide resolved

Merge branch 'main' into vianney/add-native-writer

e1f86dc

brettlangdon approved these changes Aug 7, 2025

View reviewed changes

VianneyRuhlmann added 2 commits August 8, 2025 10:20

chore(native): Update build-dependencies

6caa529

Merge branch 'main' into vianney/add-native-writer

15e4d02

rachelyangdog approved these changes Aug 8, 2025

View reviewed changes

taegyunkim approved these changes Aug 8, 2025

View reviewed changes

brettlangdon enabled auto-merge (squash) August 8, 2025 15:06

brettlangdon merged commit 100d8cc into main Aug 8, 2025
969 of 1018 checks passed

brettlangdon deleted the vianney/add-native-writer branch August 8, 2025 16:02

VianneyRuhlmann mentioned this pull request Aug 27, 2025

chore(riotfile): use correct python version in native writer scenario #14419

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(writer): implement native writer [APMSP-1875] #13071

feat(writer): implement native writer [APMSP-1875] #13071

Uh oh!

VianneyRuhlmann commented Apr 4, 2025 •

edited by brettlangdon

Loading

Uh oh!

Uh oh!

github-actions bot commented May 28, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jun 3, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jun 11, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wconti27 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

feat(writer): implement native writer [APMSP-1875] #13071

feat(writer): implement native writer [APMSP-1875] #13071

Uh oh!

Conversation

VianneyRuhlmann commented Apr 4, 2025 • edited by brettlangdon Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Description

Motivation

Testing

Checklist

Reviewer Checklist

Uh oh!

Uh oh!

github-actions bot commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Circular import analysis

🚨 New circular imports detected 🚨

Uh oh!

github-actions bot commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bootstrap import analysis

Summary

Import time breakdown

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wconti27 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

VianneyRuhlmann commented Apr 4, 2025 •

edited by brettlangdon

Loading

github-actions bot commented May 28, 2025 •

edited

Loading

github-actions bot commented Jun 3, 2025 •

edited

Loading

github-actions bot commented Jun 11, 2025 •

edited

Loading