Closed
Description
Component(s)
exporter/datadog
What happened?
Description
If Datadog API key is passed in with single and double quotations, e.g. '"my-api-key"', Datadog exporter fails to validate but does not produce an error message. This causes 403 errors to be thrown when telemetry is sent to Datadog API and doesn't leave it clear what is happening.
This also happens if it is passed in via env var and that env var contains double quotations
testfile.txt:
"my-api-key"
var=$(cat testfile.txt)
export DATADOG_API=$var
config.yaml:
...
exporters:
datadog:
api:
key: ${env:DATADOG_API}
...
Steps to Reproduce
follow export steps above, or manually set api: '"my-api-key"' in the config.yaml
Expected Result
API key validates and exporter sends telemetry
Actual Result
api key validation starts but never completes. never fails either.
Collector version
v0.111.0 and onward
Environment information
Environment
OS: Ubuntu:noble
Compiler used linux/amd64 ubuntu package, collector contrib
OpenTelemetry Collector configuration
receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318
grpc:
endpoint: 0.0.0.0:4317
processors:
batch:
connectors:
datadog/connector:
exporters:
# debug:
datadog/exporter:
#hostname: ${env:OTEL_COLLECTOR}
api:
key: ${env:DATADOG_API}
site: datadoghq.com
# fail_on_invalid_key: true
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [datadog/connector, datadog/exporter]
metrics:
receivers: [datadog/connector, otlp]
processors: [batch]
exporters: [datadog/exporter]Log output
2024-11-22T20:36:07.168Z info [email protected]/service.go:136 Setting up own telemetry...
2024-11-22T20:36:07.172Z info telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2024-11-22T20:36:37.345Z info provider/provider.go:71 Resolved source {"kind": "exporter", "data_type": "metrics", "name": "datadog/exporter", "provider": "system", "source": {"Kind":"host","Identifier":"e4eac38e404a"}}
2024-11-22T20:36:37.353Z info clientutil/api.go:40 Validating API key. {"kind": "exporter", "data_type": "metrics", "name": "datadog/exporter"}
2024-11-22T20:36:37.357Z info [email protected]/factory.go:359 Trace metrics are now disabled in the Datadog Exporter by default. To continue receiving Trace Metrics, configure the Datadog Connector or disable the feature gate. {"kind": "exporter", "data_type": "traces", "name": "datadog/exporter", "documentation": "https://docs.datadoghq.com/opentelemetry/guide/migration/", "feature gate ID": "exporter.datadogexporter.DisableAPMStats"}
2024-11-22T20:36:37.358Z info [email protected]/zaplogger.go:38 Starting Agent with processor trace buffer of size 0 {"kind": "exporter", "data_type": "traces", "name": "datadog/exporter"}
2024-11-22T20:36:37.362Z info [email protected]/zaplogger.go:38 Receiver configured with 8 decoders and a timeout of 0ms {"kind": "exporter", "data_type": "traces", "name": "datadog/exporter"}
2024-11-22T20:36:37.366Z info [email protected]/zaplogger.go:38 Trace writer initialized (climit=5 qsize=1 compression=gzip) {"kind": "exporter", "data_type": "traces", "name": "datadog/exporter"}
2024-11-22T20:36:37.368Z info clientutil/api.go:40 Validating API key. {"kind": "exporter", "data_type": "traces", "name": "datadog/exporter"}
2024-11-22T20:36:37.369Z info [email protected]/zaplogger.go:38 Processing Pipeline configured with 16 workers {"kind": "exporter", "data_type": "traces", "name": "datadog/exporter"}
2024-11-22T20:36:37.373Z info [email protected]/factory.go:62 Datadog connector using the native OTel API to ingest OTel spans and produce APM stats. To revert to the legacy processing pipeline, disable the feature gate {"kind": "connector", "name": "datadog/connector", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "metrics", "feature gate": "connector.datadogconnector.NativeIngest"}
2024-11-22T20:36:37.373Z info [email protected]/connector_native.go:62 Building datadog connector for traces to metrics {"kind": "connector", "name": "datadog/connector", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "metrics"}
2024-11-22T20:36:37.381Z info [email protected]/service.go:208 Starting otelcol-contrib... {"Version": "0.111.0", "NumCPU": 16}
2024-11-22T20:36:37.381Z info extensions/extensions.go:39 Starting extensions...
2024-11-22T20:36:37.384Z info [email protected]/connector_native.go:91 Starting datadogconnector {"kind": "connector", "name": "datadog/connector", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "metrics"}
2024-11-22T20:36:37.385Z warn [email protected]/warning.go:40 Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. {"kind": "receiver", "name": "otlp", "data_type": "metrics", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2024-11-22T20:36:37.387Z info [email protected]/otlp.go:112 Starting GRPC server {"kind": "receiver", "name": "otlp", "data_type": "metrics", "endpoint": "0.0.0.0:4317"}
2024-11-22T20:36:37.387Z warn [email protected]/warning.go:40 Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. {"kind": "receiver", "name": "otlp", "data_type": "metrics", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2024-11-22T20:36:37.388Z info [email protected]/otlp.go:169 Starting HTTP server {"kind": "receiver", "name": "otlp", "data_type": "metrics", "endpoint": "0.0.0.0:4318"}
2024-11-22T20:36:37.389Z info [email protected]/service.go:234 Everything is ready. Begin running and processing data.Additional context
something must have changed between v0.110.0 and v0.111.0+ on how environment variables are read into configuration maps.