[pkg/ottl] Create "e2e" tests in the package

[TylerHelmuth]

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

Currently OTTL depends on the transformprocessor's processor_test.go tests in order to actually test parsing and execution of complete statements. The OTTL pkg only tests parsing and individual functions, but it does not test everything all together.

Describe the solution you'd like

A new set of tests that create a payload and tests execution of statements.

Describe alternatives you've considered

No response

Additional context

We should not remove the transformprocessor tests.

[bryan-aguilar]

Do you think this is a good fit for adding to the testbed?

[jiekun]

+1 for adding to testbed. I tried it with tailsampling processor, and I think it's simple (enough) to add new components.

(BTW I feel like it's a good-first-issue if someone wants to contribute?)

[TylerHelmuth]

I'm gonna level with ya, I don't know anything about testbed. Can you explain how it helps solve our problem?

I was imagining a similar test structure that we use in the transformprocessor where we create the data using pdata and then pass it in to OTTL. In this instance instead of calling .Consume, which is a processor thing, we would create a new parser, call ParseStatements, call NewStatements, and then call Execute. For the purposes of OTTL this tests our primary external functions and the ability to manipulate data without requiring a mock collector/loadgen behind the scenes.

[TylerHelmuth]

@evan-bradley how extensive do you think we should make these tests? Should we test every function via every context or is sticking with a single context enough? Should we test every path of every context, or is the context's existing newPathGetSetter tests enough?

[TylerHelmuth]

I do not believe we need to e2e tests the different OTTL Contexts we provide. Each Context already has extensive testing that the Getters and Setters returned by the context manipulate the underlying telemetry correctly.

Similarly, I do not believe we need to e2e test parsing statements. The tests in parser_test.go extensively test each part of our grammar.

What we do need to test with e2e tests is that:

1. The OTTL features behave/interact correctly. This includes things like Where Clauses gating function execution, complex indexing, function composition, etc.
2. Each one of our ottlfuncs functions can be parsed and invoked (the ottlfuncs unit tests do not check this).