Skip to content

exporter/awsxray does not set namespace remote for producer spans #40995

Open
@strowk

Description

@strowk

Component(s)

exporter/awsxray

What happened?

Description

When producer span is communicated (f.e when producing to Kafka), it is expected that trace in AWS X-Ray would have namespace=remote (since communicating to messaging system is downstream call, from AWS docs: remote for other downstream calls.).

Steps to Reproduce

  1. setup aws account with xray and adot collector addon
  2. configure awsxray exporter, f.e like this - https://aws-otel.github.io/docs/getting-started/x-ray#sample-collector-configuration-putting-it-together
  3. send opentelemetry trace that has a span with span kind=producer

Expected Result

Segment in AWS X-Ray should have namespace=remote, X-Ray should identify these segments with separate node (so that traces with this node could be searched and trace map would correctly display that node).

Actual Result

The segment does not have remote segment, this results in node not being identified, refiner by Node cannot locate it and in trace map it looks like this communication does not happen.

Collector version

ADOT Collector v0.43.2

Environment information

Environment

OS: Amazon Linux 2
Docker image: public.ecr.aws/aws-observability/aws-otel-collector:v0.43.2

OpenTelemetry Collector configuration

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
        tls:
          cert_file: /etc/x509/tls.crt
          key_file: /etc/x509/tls.key

processors:
  groupbytrace:
    wait_duration: 60s
    num_traces: 2000
  tail_sampling:
    decision_wait: 5s # This value should be smaller than wait_duration
    policies:
      - name: sample_rate
        type: rate_limiting
        rate_limiting:
          spans_per_second: 5
  batch/tracesampling:
    timeout: 0s # No need to wait more since this will happen in previous processors
    send_batch_max_size: 8196 # This will still allow us to limit the size of the batches sent to subsequent exporters
  memory_limiter:
    limit_mib: 100
    check_interval: 5s

exporters:
  awsxray:
    region: eu-central-1

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter,groupbytrace,tail_sampling,batch/tracesampling]
      exporters: [awsxray]

Log output

Additional context

I am not sure I understand precisely everything that goes on in that code, but it appears to me that this part is wrong:

if namespace == "" && span.Kind() == ptrace.SpanKindClient {
namespace = "remote"
}

It currently sets "remote" namespace when namespace was not determined by something else before that, but only if opentelemetry span kind is client. It should do the same for all kinds except maybe internal, I believe.

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions