Remove attributes from body and add them as log record attributes for FrontDoorWebApplicationFirewallLog log category

[constanca-m]

Component(s)

pkg/translator/azurelogs

Is your feature request related to a problem? Please describe.

Parent issue is #39186.

See related issue for a different category AzureCdnAccessLog: #39811.

Describe the solution you'd like

When the record attributes are part of the body, it is not possible to query the record by these attributes. To achieve that, we need to add explicit support for the current categories.

This issue approaches the category FrontDoorWebApplicationFirewallLog (the goal is to handle all the other categories currently upstream as well).

This is an example of a log of this category.

{
  "records":[
    {
      "time":"2025-04-24T15:35:06.0000000Z",
      "resourceId":"/SUBSCRIPTIONS/OPENTELEMETRY-AZURE-SUB/RESOURCEGROUPS/OPENTELEMETRY-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/OPENTELEMETRY-FRONTDOOR-PROFILE",
      "category":"FrontDoorWebApplicationFirewallLog",
      "operationName":"Microsoft.Cdn/Profiles/WebApplicationFirewallLog/Write",
      "properties":{
        "clientIP":"2001:1c00:3280:6700:fbfa:bf04:1296:ebfc",
        "clientPort":"57660",
        "socketIP":"2001:1c00:3280:6700:fbfa:bf04:1296:ebfc",
        "requestUri":"http://opentelemetry-test-fmagg0exgdcfhefq.z01.azurefd.net:80/",
        "ruleName":"Rule1",
        "policy":"policy",
        "action":"Block",
        "host":"opentelemetry-test-fmagg0exgdcfhefq.z01.azurefd.net",
        "trackingReference":"20250424T153506Z-1756f49cc78nldhvhC1DUS3dhw000000080000000000d207",
        "policyMode":"prevention",
        "details":{
          "matches":[
            {
              "matchVariableName":"Method",
              "matchVariableValue":"GET"
            }
          ],
          "msg":"",
          "data":""
        }
      }
    }
  ]
}

The proposed log attributes are:

Original Field (JSON)	Log Record Attribute
clientIP	client.address
clientPort	client.port
socketIP	source.address
requestUri	url.orginal Also parses it to get fields: 1.url.scheme 2.url.fragment 3.url.query 4.url.path 5.url.port
ruleName	azure.frontdoor.waf.rule
policy	azure.frontdoor.waf.policy.name
action	azure.frontdoor.waf.action
host	http.request.header.host
trackingReference	azure.ref
policyMode	azure.frontdoor.waf.policy.mode
details	Ignore for now

This means that the example of the log I put above, will end up as:

resourceLogs:
  - resource:
      attributes:
        - key: cloud.provider
          value:
            stringValue: azure
        - key: cloud.resource_id
          value:
            stringValue: /SUBSCRIPTIONS/OPENTELEMETRY-AZURE-SUB/RESOURCEGROUPS/OPENTELEMETRY-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/OPENTELEMETRY-FRONTDOOR-PROFILE
        - key: event.name
          value:
            stringValue: az.resource.log
    scopeLogs:
      - logRecords:
          - attributes:
              - key: client.port
                value:
                  intValue: "57660"
              - key: url.original
                value:
                  stringValue: http://opentelemetry-test-fmagg0exgdcfhefq.z01.azurefd.net:80/
              - key: url.port
                value:
                  intValue: "80"
              - key: url.scheme
                value:
                  stringValue: http
              - key: url.path
                value:
                  stringValue: /
              - key: client.address
                value:
                  stringValue: 2001:1c00:3280:6700:fbfa:bf04:1296:ebfc
              - key: source.address
                value:
                  stringValue: 2001:1c00:3280:6700:fbfa:bf04:1296:ebfc
              - key: azure.ref
                value:
                  stringValue: 20250424T153506Z-1756f49cc78nldhvhC1DUS3dhw000000080000000000d207
              - key: http.request.header.host
                value:
                  stringValue: opentelemetry-test-fmagg0exgdcfhefq.z01.azurefd.net
              - key: azure.frontdoor.waf.policy.name
                value:
                  stringValue: policy
              - key: azure.frontdoor.waf.policy.mode
                value:
                  stringValue: prevention
              - key: azure.frontdoor.waf.rule.name
                value:
                  stringValue: Rule1
              - key: azure.frontdoor.waf.action
                value:
                  stringValue: Block
              - key: azure.category
                value:
                  stringValue: FrontDoorWebApplicationFirewallLog
              - key: azure.operation.name
                value:
                  stringValue: Microsoft.Cdn/Profiles/WebApplicationFirewallLog/Write
            body: {}
            spanId: ""
            timeUnixNano: "1745508906000000000"
            traceId: ""
        scope:
          name: otelcol/azureresourcelogs
          version: 1.2.3

For the current implementation, the log record ends up like this.

resourceLogs:
  - resource:
      attributes:
        - key: cloud.provider
          value:
            stringValue: azure
        - key: cloud.resource_id
          value:
            stringValue: /SUBSCRIPTIONS/OPENTELEMETRY-AZURE-SUB/RESOURCEGROUPS/OPENTELEMETRY-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/OPENTELEMETRY-FRONTDOOR-PROFILE
        - key: event.name
          value:
            stringValue: az.resource.log
    scopeLogs:
      - logRecords:
          - body:
              kvlistValue:
                values:
                  - key: category
                    value:
                      stringValue: FrontDoorWebApplicationFirewallLog
                  - key: operation.name
                    value:
                      stringValue: Microsoft.Cdn/Profiles/WebApplicationFirewallLog/Write
                  - key: properties
                    value:
                      kvlistValue:
                        values:
                          - key: policyMode
                            value:
                              stringValue: prevention
                          - key: clientIP
                            value:
                              stringValue: 2001:1c00:3280:6700:fbfa:bf04:1296:ebfc
                          - key: socketIP
                            value:
                              stringValue: 2001:1c00:3280:6700:fbfa:bf04:1296:ebfc
                          - key: trackingReference
                            value:
                              stringValue: 20250424T153506Z-1756f49cc78nldhvhC1DUS3dhw000000080000000000d207
                          - key: details
                            value:
                              kvlistValue:
                                values:
                                  - key: matches
                                    value:
                                      arrayValue:
                                        values:
                                          - kvlistValue:
                                              values:
                                                - key: matchVariableName
                                                  value:
                                                    stringValue: Method
                                                - key: matchVariableValue
                                                  value:
                                                    stringValue: GET
                                  - key: msg
                                    value:
                                      stringValue: ""
                                  - key: data
                                    value:
                                      stringValue: ""
                          - key: clientPort
                            value:
                              stringValue: "57660"
                          - key: requestUri
                            value:
                              stringValue: http://opentelemetry-test-fmagg0exgdcfhefq.z01.azurefd.net:80/
                          - key: action
                            value:
                              stringValue: Block
                          - key: ruleName
                            value:
                              stringValue: Rule1
                          - key: policy
                            value:
                              stringValue: policy
                          - key: host
                            value:
                              stringValue: opentelemetry-test-fmagg0exgdcfhefq.z01.azurefd.net
            spanId: ""
            timeUnixNano: "1745508906000000000"
            traceId: ""
        scope:
          name: otelcol/azureresourcelogs
          version: 1.2.3

Describe alternatives you've considered

N/A

Additional context

Internally (at Elastic), we are using ingest pipelines to parse this body and put everything as attributes. This would not be necessary if we fix this upstream.

See, for example, the AWS logs encoding extension upstream that parses a log and puts the field as log record attributes.

[constanca-m]

/labels -needs-triage

[github-actions]

Pinging code owners:

• pkg/translator/azurelogs: @atoulme @cparkins @MikeGoldsmith @constanca-m

See Adding Labels via Comments if you do not have permissions to add labels yourself.