Update Pipeline Component Telemetry RFC

[jade-guiton-dd]

Description

This PR updates the Pipeline Component Telemetry RFC with the following changes:

• Reflect implementation choices that have been made since the RFC was written:

  1. using instrumentation scope attributes instead of datapoint attributes to identify component instances
     (see discussion in System for managing own telemetry attributes within pipeline components #12217 and Attribute injection in the Collector opentelemetry-go#6404)
  2. automatically injecting these attributes, without changes to component code
  3. changing the instrumentation scope name used for pipeline metrics

• Slightly change the semantics of outcome = refused:

  The current planned behavior (from Amend Pipeline Component Telemetry RFC to add a "rejected" outcome #11956) is that, in the case of a pipeline A → B where component B returns an error, the "consumed" metric for B and the "produced" metric for A should both have outcome = failure.

  I fear that this may lead users to think that a failure occurred in A, and would like to restrict outcome = failure to only be associated with the component that "failed", ie. component B. The "produced" metric associated with A would instead have outcome = refused.

  This incidentally makes implementation slightly easier, since an instrumentation layer will not need different error wrapping behavior between the "producer" layer and the "consumer" layer.

  See draft PR Emit outcome: failure in obsconsumer #13234 for an example implementation.

As this is a non-trivial change to an RFC, it may need to follow the RFC process.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.55%. Comparing base (7f957ed) to head (e4765d4).
Report is 25 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #13260      +/-   ##
==========================================
- Coverage   91.57%   91.55%   -0.03%     
==========================================
  Files         522      522              
  Lines       29089    29089              
==========================================
- Hits        26639    26631       -8     
- Misses       1933     1939       +6     
- Partials      517      519       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[songy23]

LGTM

[jmacd]

This sounds great. I think it is useful and important to identify the original failure as distinct from the subsequent refusals. Is this really a change, relative to what's already written for failure and refused? (Referring to lines 100-101.)

[jade-guiton-dd]

The current version of the RFC already allows users to identify where the original failure occurred. It's just that they need to be careful in interpreting the metrics:

• a metric point like otelcol.processor.consumed.items (otelcol.component.id = transform, outcome = failure) means that a failure occurred in the Consume call where the transform processor consumed items, ie. it happened in the transform processor's code
• a metric point like otelcol.processor.produced.items (otelcol.component.id = transform, outcome = failure) means that a failure occurred in the Consume call where the transform processor produced items, ie. the transform processor succeeded in transforming data, but the next component in the pipeline returned an error.

The change in this PR just amounts to saying that the second case should be labeled as outcome = refused instead, like for all the components further upstream. As a side effect, it means that we never see outcome = failure on a "produced" metric.

Hope that clears things up.

[jade-guiton-dd]

I'll announce this RFC amendment on Wednesday's SIG meeting, then we can start the waiting period.

[mx-psi]

If there are no further objections, this can be merged on or after July 9th (added one more day to account for a lot of folks in the US)