-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Description
Component(s)
exporter/elasticsearch
What happened?
Description
I have encountered an issue with counter metrics sent via the collector. We currently configure the EDOT SDK's to use delta temporality. However, these end up mapped with "time_series_metric": "counter"
. This then causes an error when used in a dashboard summing the overall count, which warns this "may be meaningless" (doc reference) as it expects the counter to be "A cumulative metric that only monotonically increases or resets to 0 (zero).".
The counter metric documentation expects the counter to be monotonically incrementing, which is not the case here with delta temporality. We think the temporality setting needs to be used in the mapping to switch instead to gauge when using delta.
Steps to Reproduce
Send counter metrics using delta temporality via the collector, which are then exported to ES.
Expected Result
These metrics should be aggregatable by sum
in dashboard panels without causing a warning.
Actual Result
When used, the panel displays a warning:
Collector version
fab2130
Environment information
Environment
OS: Windows 11
Compiler(if manually compiled): I'm not sure how to identify this
OpenTelemetry Collector configuration
extensions:
bearertokenauth:
scheme: "APIKey"
token: "REDACTED"
apmconfig:
source:
elasticsearch:
endpoint: "http://localhost:9200"
auth:
authenticator: bearertokenauth
opamp:
protocols:
http:
endpoint: "localhost:4320"
receivers:
# Receiver for logs, traces, and metrics from SDKs
otlp/fromsdk:
protocols:
grpc:
http:
connectors:
elasticapm:
processors:
elastictrace:
exporters:
elasticsearch/otel:
endpoints: [ "http://localhost:9200" ]
auth:
authenticator: bearertokenauth
mapping:
mode: otel
logs_dynamic_index:
enabled: true
metrics_dynamic_index:
enabled: true
traces_dynamic_index:
enabled: true
service:
telemetry:
logs:
level: debug
extensions: [bearertokenauth, apmconfig]
pipelines:
traces/fromsdk:
receivers: [otlp/fromsdk]
processors: [elastictrace]
exporters: [elasticapm, elasticsearch/otel]
metrics/fromsdk:
receivers: [otlp/fromsdk]
processors: []
exporters: [elasticsearch/otel]
metrics/aggregated-metrics:
receivers: [elasticapm]
processors: []
exporters: [elasticsearch/otel]
logs/fromsdk:
receivers: [otlp/fromsdk]
processors: []
exporters: [elasticapm, elasticsearch/otel]