Skip to content

Conversation

@Melonps
Copy link

@Melonps Melonps commented Nov 24, 2025

Which problem is this PR solving?

#3159

Description of the changes

  • Fixes a bug where the reducer crashes when metrics payload has an invalid structure (missing metrics propaties)

How was this change tested?

  • Existing unit tests pass
  • Manually tested using the configuration below (storage: OpenSearch). You should see warnings in the console such as Invalid metrics response for service_operation_latencies.
  • If the payload being received is malformed, it’s likely an issue on the OpenSearch side. That said, I’m happy to add unit tests or adjust the types if needed.

docker-compose.yaml

services:
  opensearch:
    networks:
      - backend
      - middleware
    image: opensearchproject/opensearch:2.11.0
    container_name: opensearch
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=false"
      - plugins.security.ssl.http.enabled=true
      - plugins.security.ssl.http.pemcert_filepath=esnode.pem
      - plugins.security.ssl.http.pemkey_filepath=esnode-key.pem
      - plugins.security.ssl.http.pemtrustedcas_filepath=root-ca.pem
      - plugins.security.ssl.transport.pemcert_filepath=esnode.pem
      - plugins.security.ssl.transport.pemkey_filepath=esnode-key.pem
      - plugins.security.ssl.transport.pemtrustedcas_filepath=root-ca.pem
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "9200:9200"
      - "9600:9600"

  jaeger:
    networks:
      backend:
        # This is the host name used in Prometheus scrape configuration.
        aliases: [ spm_metrics_source ]
      middleware:
    image: jaegertracing/jaeger:${JAEGER_VERSION:-latest}
    volumes:
      - "./jaeger-ui.json:/etc/config/config-ui.json"
      - "../../cmd/jaeger/config-spm.yaml:/etc/jaeger/config.yml"
    command: ["--config", "/etc/jaeger/config.yml"]
    ports:
      - "16686:16686"
      - "8888:8888"
      - "8889:8889"
      - "4317:4317"
      - "4318:4318"
      - "6831:6831/udp"
      - "6832:6832/udp"
      - "14250:14250"
    depends_on:
      - opensearch

  microsim:
    networks:
      - backend
    image: yurishkuro/microsim:v0.5.0@sha256:b7ee2dee51d2c9fd94de08a80278cfbf5a144ad0f22efce50f3d3be15cbfa2c7
    command: "-d 24h -s 500ms"
    environment:
      - OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://jaeger:4318/v1/traces
    depends_on:
      - jaeger

  prometheus:
    networks:
      - backend
    image: prom/prometheus:v3.6.0@sha256:76947e7ef22f8a698fc638f706685909be425dbe09bd7a2cd7aca849f79b5f64
    volumes:
      - "./prometheus.yml:/etc/prometheus/prometheus.yml"
    ports:
      - "9090:9090"

networks:
  backend:
  middleware:

config-issue.yaml from #3159

service:
  extensions: [jaeger_storage, jaeger_query, healthcheckv2]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger_storage_exporter]
  telemetry:
    resource:
      service.name: jaeger
    metrics:
      level: detailed
      readers:
        - pull:
            exporter:
              prometheus:
                host: 0.0.0.0
                port: 8888
    logs:
      level: debug
    # TODO Initialize telemetry tracer once OTEL released new feature.
    # https://github.com/open-telemetry/opentelemetry-collector/issues/10663

extensions:
  healthcheckv2:
    use_v2: true
    http:

  jaeger_query:
    storage:
      traces: some_storage
      traces_archive: another_storage
      metrics: some_storage
    ui:
      config_file: /etc/config/config-ui.json

  jaeger_storage:
    backends:
      some_storage: &opensearch_config
        opensearch:
          server_urls:
            - https://opensearch.middleware:9200
          auth:
            basic:
              username: "admin"
              password: "admin"
          tls:
            insecure_skip_verify: true  ###跳过opensearch的tls认证
          indices:
            index_prefix: "jaeger-main"
            spans:
              date_layout: "2006-01-02"
              rollover_frequency: "day"
              shards: 3
              replicas: 1
            services:
              date_layout: "2006-01-02"
              rollover_frequency: "day"
              shards: 3
              replicas: 1
            dependencies:
              date_layout: "2006-01-02"
              rollover_frequency: "day"
              shards: 3
              replicas: 1
            sampling:
              date_layout: "2006-01-02"
              rollover_frequency: "day"
              shards: 3
              replicas: 1
      another_storage:
        opensearch:
          server_urls:
            - https://opensearch.middleware:9200
          auth:
            basic:
              username: "admin"
              password: "admin"
          tls:
            insecure_skip_verify: true  ###跳过opensearch的tls认证
          indices:
            index_prefix: "jaeger-archive"
    metric_backends:
      some_storage: *opensearch_config

connectors:
  spanmetrics:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  jaeger:
    protocols:
      grpc:
        endpoint: 0.0.0.0:14250
      thrift_compact:
        endpoint: 0.0.0.0:6831
      thrift_binary:
        endpoint: 0.0.0.0:6832

processors:
  batch:

exporters:
  jaeger_storage_exporter:
    trace_storage: some_storage

Checklist

@Melonps Melonps requested a review from a team as a code owner November 24, 2025 08:52
@Melonps Melonps requested review from joe-elliott and removed request for a team November 24, 2025 08:52
@Melonps Melonps changed the title Handle invalid metrics payload type in reducer (metrics.tsx) Handle error with invalid metrics payload type in reducer (metrics.tsx) Nov 24, 2025
@Melonps Melonps changed the title Handle error with invalid metrics payload type in reducer (metrics.tsx) Handle error with invalid metrics payload type in reducer Nov 24, 2025
@Melonps Melonps changed the title Handle error with invalid metrics payload type in reducer fix(bug): Handle error with invalid metrics payload type in reducer Nov 24, 2025
@Melonps Melonps marked this pull request as draft November 25, 2025 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant