We are seeing duplicate logs after installing Splunk otel collector

[suresh0204]

Is there any setting that can be changed in the values.yaml file to stop getting duplicate logs?

[rmfitzpatrick]

@suresh0204 are you able to provide an example of the duplicates and their origin?

[suresh0204]

Below is the query that we are running to find the duplicate logs

index=<index_name> cluster_name::<cluster_name> container_name::* pod::*
| fields source _indextime index pod sourcetype
| stats count values(_indextime) AS indextime dc(_indextime) AS indexed_count by _time _raw index pod source sourcetype
| where count>1
| eval _time=strftime(_time,"%F %T.%3N %Z"), indextime=strftime(indextime,"%F %T.%3N %Z")

[jvoravong]

We could really use more details about your collector setup . Can you please post details about the following?

• Which logs engine you are using?
• Do you have any custom log configurations you deployed that are not default?
• What version of the collector you are using?
• Which Kubernetes cluster distribution and version are you using?

[suresh0204]

I have attached the sample configuration for you, please take a look and let me know.
sample config.txt

[suresh0204]

Can you please let me know if there is something that I need to change in the configuration?

[suresh0204]

Please let me know if there is any update on this.