Skip to content

[processor/batchprocessor] Improve batch processor edge case performance #13272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

asweet-confluent
Copy link

Description

As the batch size approaches one, the time complexity of the batch processor approaches O(n^2). This is caused by repeatedly counting the number of data points in a message. This commit introduces a "short-circuit" behavior that allows the core loop to exit quickly when the desired number of data points has been found.

Testing

Performance

I added a new benchmark for splitMetrics that tests performance with various numbers of metrics and a batch size of one. This is the before and after comparison produced by benchstat:

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/collector/processor/batchprocessor
cpu: 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
                             │   old.txt    │               new.txt               │
                             │    sec/op    │   sec/op     vs base                │
TraceSizeSpanCount-16          3.326n ± 13%   3.181n ± 5%   -4.37% (p=0.011 n=20)
BatchMetricProcessor-16        366.9µ ±  8%   416.7µ ± 7%  +13.58% (p=0.003 n=20)
MultiBatchMetricProcessor-16   419.6µ ±  6%   404.8µ ± 3%   -3.54% (p=0.021 n=20)
SplitMetrics1k-16              4.063m ±  7%   1.264m ± 4%  -68.88% (p=0.000 n=20)
SplitMetrics16k-16             841.3m ±  3%   189.4m ± 2%  -77.48% (p=0.000 n=20)
SplitMetrics64k-16             17.599 ±  5%    3.203 ± 9%  -81.80% (p=0.000 n=20)
geomean                        1.771m         862.5µ       -51.29%
Correctness

I'm relying on the existing test suite to catch issues here.

@asweet-confluent asweet-confluent requested a review from a team as a code owner June 26, 2025 04:09
@asweet-confluent asweet-confluent requested a review from jmacd June 26, 2025 04:09
As the batch size approaches one, the time complexity of the batch
processor approaches O(n^2). This is caused by repeatedly counting the
number of data points in a message. This commit introduces a
"short-circuit" behavior that allows the core loop to exit quickly when
the desired number of data points has been found.
@asweet-confluent asweet-confluent force-pushed the improve-batch-processor-perf branch from 8d11cbe to 051a987 Compare June 26, 2025 04:13
Copy link

codecov bot commented Jun 26, 2025

Codecov Report

Attention: Patch coverage is 91.66667% with 8 lines in your changes missing coverage. Please review.

Project coverage is 91.56%. Comparing base (a33fdf3) to head (051a987).

Files with missing lines Patch % Lines
processor/batchprocessor/splitlogs.go 88.00% 2 Missing and 1 partial ⚠️
processor/batchprocessor/splittraces.go 87.50% 2 Missing and 1 partial ⚠️
processor/batchprocessor/splitmetrics.go 95.74% 1 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (91.66%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #13272      +/-   ##
==========================================
- Coverage   91.57%   91.56%   -0.01%     
==========================================
  Files         522      522              
  Lines       29089    29162      +73     
==========================================
+ Hits        26639    26703      +64     
- Misses       1933     1939       +6     
- Partials      517      520       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant