Skip to content

Add CEL validation test for targetRef in ClientSettingsPolicy #3623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 60 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
dbb5b5c
Add CEL validation test for targetRef in ClientSettingsPolicy
shaun-nx Jul 14, 2025
48ec86b
gofumpt
shaun-nx Jul 14, 2025
c7129bd
Add tests for targetRefGroup
shaun-nx Jul 14, 2025
87a4e0e
Rename tests
shaun-nx Jul 14, 2025
29f201e
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 15, 2025
d921ed2
Move tests into clientsettingspolicy_test.go
shaun-nx Jul 15, 2025
f22269a
Update tests to create a ClientSettingsPolicy resource during tests
shaun-nx Jul 15, 2025
7fddffa
make lint in tests
shaun-nx Jul 15, 2025
dd66a79
Update tests to create a ClientSettingsPolicy object during validation
shaun-nx Jul 17, 2025
d664f35
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 17, 2025
cd9bada
Update TargetRegGroup tests
shaun-nx Jul 17, 2025
7d9989d
Fix lint errors
shaun-nx Jul 17, 2025
9274e57
Group valid and invalid test cases into single test function
shaun-nx Jul 21, 2025
506b01c
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 21, 2025
f99ab29
Revert dependency version
shaun-nx Jul 21, 2025
ad80b70
Move imports
shaun-nx Jul 21, 2025
e5479fb
Define constants for Kinds and Groups
shaun-nx Jul 21, 2025
1789701
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 22, 2025
aa4a6b2
Use controller-runtime library to get cluster information
shaun-nx Jul 22, 2025
8f74214
Add Makefile targets to run CEL test and re format test
shaun-nx Jul 22, 2025
4486bd5
Ensure TestClientSettingsPoliciesTargetRefKind and TestClientSettings…
shaun-nx Jul 22, 2025
1adf168
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 22, 2025
7c6dba4
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 23, 2025
69f485b
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 23, 2025
1ad24c4
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 25, 2025
aa5adc6
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 28, 2025
29f26ef
Update test to use `gomega` test library
shaun-nx Jul 28, 2025
f26eea3
Add ci workflow job for cel tests
shaun-nx Jul 28, 2025
efa4a81
Add new line
shaun-nx Jul 28, 2025
d25661a
Add `Deploy Kubernetes` step
shaun-nx Jul 28, 2025
c413895
Add step to apply CRDs in cel-tests CI job
shaun-nx Jul 28, 2025
9962e03
Add `--server-side` to apply command
shaun-nx Jul 28, 2025
feca4ed
Add `working-directory` to test run and remove code coverage steps
shaun-nx Jul 28, 2025
8d84e7b
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 28, 2025
ddce60f
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 29, 2025
d518ada
Add t.Parallel()
shaun-nx Jul 29, 2025
39a6b38
Use g.Expect for errors and ensure unique policy names
shaun-nx Jul 29, 2025
5079c21
Return and assert errors from `getKubernetesClient`
shaun-nx Jul 29, 2025
ce0960b
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 29, 2025
248df53
Remove nil error checks
shaun-nx Jul 29, 2025
c5860e2
Create and return client on same line
shaun-nx Jul 29, 2025
83139ce
Use `rand.Prime` to attempt to keep policy names unique for each para…
shaun-nx Jul 29, 2025
a4e58ee
Remove unused dependency
shaun-nx Jul 29, 2025
339e6b8
Fix imports
shaun-nx Jul 29, 2025
716d051
Add helper functions to gernated unique resource names
shaun-nx Jul 30, 2025
d273281
Move `getKubernetesClient` function to helpers
shaun-nx Jul 30, 2025
4a6e84e
Fix test failures
shaun-nx Jul 30, 2025
66b7105
Update `TestMustCreateKubernetesClient` to only assert client creation
shaun-nx Jul 30, 2025
510aac1
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 30, 2025
c45403a
Remove test for creating kubernetes cluster
shaun-nx Jul 30, 2025
e284740
Add common finctions for cel tests
shaun-nx Jul 30, 2025
06adc0e
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 30, 2025
cb43b52
Move common constatns to `common.go` and add v1alpha2 to client schema
shaun-nx Jul 30, 2025
9217339
Move helper functions to common.go and add tests
shaun-nx Jul 30, 2025
f30f117
unexport consnts and functions in cel common file
shaun-nx Jul 30, 2025
e18b9dc
unexport randomPrimeNumber and uniqueResourceName
shaun-nx Jul 30, 2025
869a091
Fix casing in comments
shaun-nx Jul 30, 2025
92caf12
Remove duplicate error checks
shaun-nx Jul 31, 2025
81fd868
Initialise ginko and k8sClient at the start of each test
shaun-nx Jul 31, 2025
94a8dd0
Merge branch 'main' into tests/cel-clientsettingspolicies
shaun-nx Jul 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -311,3 +311,32 @@ jobs:
- name: Push to GitHub Container Registry
run: |
helm push ${{ steps.package.outputs.path }} oci://ghcr.io/nginx/charts

cel-tests:
name: CEL Tests
runs-on: ubuntu-24.04
needs: vars
steps:
- name: Checkout Repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Setup Golang Environment
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version: stable
cache-dependency-path: |
go.sum
.github/.cache/buster-for-unit-tests

- name: Deploy Kubernetes
id: k8s
run: |
kind create cluster --name ${{ github.run_id }} --image=kindest/node:${{ needs.vars.outputs.k8s_latest }}

- name: Apply CustomResourceDefinition
run: |
kubectl kustomize config/crd | kubectl apply --server-side -f -

- name: Run Tests
run: make test-cel-validation
working-directory: ./tests
12 changes: 12 additions & 0 deletions tests/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ STANDARD_CONFORMANCE_PROFILES = GATEWAY-HTTP,GATEWAY-GRPC
EXPERIMENTAL_CONFORMANCE_PROFILES = GATEWAY-TLS
CONFORMANCE_PROFILES = $(STANDARD_CONFORMANCE_PROFILES) # by default we use the standard conformance profiles. If experimental is enabled we override this and add the experimental profiles.
SKIP_TESTS =
CEL_TEST_TARGET =

# Check if ENABLE_EXPERIMENTAL is true
ifeq ($(ENABLE_EXPERIMENTAL),true)
Expand Down Expand Up @@ -186,3 +187,14 @@ uninstall-ngf: ## Uninstall NGF on configured kind cluster
-make uninstall-gateway-crds
-kubectl delete namespace nginx-gateway
-kubectl kustomize ../config/crd | kubectl delete -f -

# Run CEL validation integration tests against a real cluster
.PHONY: test-cel-validation
test-cel-validation:
@if [ -z "$(CEL_TEST_TARGET)" ]; then \
echo "Running all tests in ./cel"; \
go test -v ./cel; \
else \
echo "Running test: $(CEL_TEST_TARGET) in ./cel"; \
go test -v ./cel -run "$(CEL_TEST_TARGET)"; \
fi
166 changes: 166 additions & 0 deletions tests/cel/clientsettingspolicy_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
package cel

import (
"context"
"testing"

. "github.com/onsi/gomega"
controllerruntime "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"

ngfAPIv1alpha1 "github.com/nginx/nginx-gateway-fabric/v2/apis/v1alpha1"
)

func TestClientSettingsPoliciesTargetRefKind(t *testing.T) {
t.Parallel()
g := NewWithT(t)
k8sClient, err := getKubernetesClient(t)
g.Expect(err).ToNot(HaveOccurred())
tests := []struct {
policySpec ngfAPIv1alpha1.ClientSettingsPolicySpec
name string
wantErrors []string
}{
{
name: "Validate TargetRef of kind Gateway is allowed",
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: gatewayKind,
Group: gatewayGroup,
},
},
},
{
name: "Validate TargetRef of kind HTTPRoute is allowed",
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: httpRouteKind,
Group: gatewayGroup,
},
},
},
{
name: "Validate TargetRef of kind GRPCRoute is allowed",
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: grpcRouteKind,
Group: gatewayGroup,
},
},
},
{
name: "Validate Invalid TargetRef Kind is not allowed",
wantErrors: []string{expectedTargetRefKindError},
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: invalidKind,
Group: gatewayGroup,
},
},
},
{
name: "Validate TCPRoute TargetRef Kind is not allowed",
wantErrors: []string{expectedTargetRefKindError},
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: tcpRouteKind,
Group: gatewayGroup,
},
},
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
validateClientSettingsPolicy(t, tt, g, k8sClient)
})
}
}

func TestClientSettingsPoliciesTargetRefGroup(t *testing.T) {
t.Parallel()
g := NewWithT(t)
k8sClient, err := getKubernetesClient(t)
g.Expect(err).ToNot(HaveOccurred())
tests := []struct {
policySpec ngfAPIv1alpha1.ClientSettingsPolicySpec
name string
wantErrors []string
}{
{
name: "Validate gateway.networking.k8s.io TargetRef Group is allowed",
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: gatewayKind,
Group: gatewayGroup,
},
},
},
{
name: "Validate invalid.networking.k8s.io TargetRef Group is not allowed",
wantErrors: []string{expectedTargetRefGroupError},
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: gatewayKind,
Group: invalidGroup,
},
},
},
{
name: "Validate discovery.k8s.io/v1 TargetRef Group is not allowed",
wantErrors: []string{expectedTargetRefGroupError},
policySpec: ngfAPIv1alpha1.ClientSettingsPolicySpec{
TargetRef: gatewayv1alpha2.LocalPolicyTargetReference{
Kind: gatewayKind,
Group: discoveryGroup,
},
},
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
validateClientSettingsPolicy(t, tt, g, k8sClient)
})
}
}

func validateClientSettingsPolicy(t *testing.T, tt struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to just keep in mind for when we write more CEL tests for other policies, is that a lot of this function (and getKubernetesClient) will likely be re-used. So we can probably make it more generic in a separate file to handle all types. No action needed in this PR.

policySpec ngfAPIv1alpha1.ClientSettingsPolicySpec
name string
wantErrors []string
}, g *WithT, k8sClient client.Client,
) {
t.Helper()

policySpec := tt.policySpec
policySpec.TargetRef.Name = gatewayv1alpha2.ObjectName(uniqueResourceName(testTargetRefName))
policyName := uniqueResourceName(testPolicyName)

clientSettingsPolicy := &ngfAPIv1alpha1.ClientSettingsPolicy{
ObjectMeta: controllerruntime.ObjectMeta{
Name: policyName,
Namespace: defaultNamespace,
},
Spec: policySpec,
}

err := k8sClient.Create(context.Background(), clientSettingsPolicy)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the k8sClient is using the actual Kubernetes API Server, I wonder if we need to add a timeout

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It probably is fine without it but just curious

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, when creating we typically do have a context to make sure it doesn't block there. Like these constants https://github.com/nginx/nginx-gateway-fabric/blob/main/tests/framework/timeout.go


// Clean up after test
defer func() {
_ = k8sClient.Delete(context.Background(), clientSettingsPolicy)
}()

if len(tt.wantErrors) == 0 {
g.Expect(err).ToNot(HaveOccurred())
} else {
g.Expect(err).To(HaveOccurred())
for _, wantError := range tt.wantErrors {
g.Expect(err.Error()).To(ContainSubstring(wantError), "Expected error '%s' not found in: %s", wantError, err.Error())
}
}
}
78 changes: 78 additions & 0 deletions tests/cel/common.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package cel

import (
"crypto/rand"
"fmt"
"testing"

"k8s.io/apimachinery/pkg/runtime"
controllerruntime "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"

ngfAPIv1alpha1 "github.com/nginx/nginx-gateway-fabric/v2/apis/v1alpha1"
ngfAPIv1alpha2 "github.com/nginx/nginx-gateway-fabric/v2/apis/v1alpha2"
)

const (
gatewayKind = "Gateway"
httpRouteKind = "HTTPRoute"
grpcRouteKind = "GRPCRoute"
tcpRouteKind = "TCPRoute"
invalidKind = "InvalidKind"
)

const (
gatewayGroup = "gateway.networking.k8s.io"
invalidGroup = "invalid.networking.k8s.io"
discoveryGroup = "discovery.k8s.io/v1"
)

const (
expectedTargetRefKindError = `TargetRef Kind must be one of: Gateway, HTTPRoute, or GRPCRoute`
expectedTargetRefGroupError = `TargetRef Group must be gateway.networking.k8s.io.`
)

const (
defaultNamespace = "default"
)

const (
testPolicyName = "test-policy"
testTargetRefName = "test-targetRef"
)

// getKubernetesClient returns a client connected to a real Kubernetes cluster.
func getKubernetesClient(t *testing.T) (k8sClient client.Client, err error) {
t.Helper()
// Use controller-runtime to get cluster connection
k8sConfig, err := controllerruntime.GetConfig()
if err != nil {
return nil, err
}

// Set up scheme with NGF types
scheme := runtime.NewScheme()
if err = ngfAPIv1alpha1.AddToScheme(scheme); err != nil {
return nil, err
}
if err = ngfAPIv1alpha2.AddToScheme(scheme); err != nil {
return nil, err
}
// Create a new client with the scheme and return it
return client.New(k8sConfig, client.Options{Scheme: scheme})
}

// randomPrimeNumber generates a random prime number of 64 bits.
// It panics if it fails to generate a random prime number.
func randomPrimeNumber() int64 {
primeNum, err := rand.Prime(rand.Reader, 64)
if err != nil {
panic(fmt.Errorf("failed to generate random prime number: %w", err))
}
return primeNum.Int64()
}

// uniqueResourceName generates a unique resource name by appending a random prime number to the given name.
func uniqueResourceName(name string) string {
return fmt.Sprintf("%s-%d", name, randomPrimeNumber())
}
26 changes: 26 additions & 0 deletions tests/cel/common_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package cel

import (
"testing"

. "github.com/onsi/gomega"
)

func TestMustGenerateRandomPrimeNumber(t *testing.T) {
t.Parallel()
g := NewWithT(t)
g.Expect(func() {
_ = randomPrimeNumber()
}).ToNot(Panic())
}

func TestMustReturnUniqueResourceName(t *testing.T) {
t.Parallel()
g := NewWithT(t)

name := "test-resource"
uniqueName := uniqueResourceName(name)

g.Expect(uniqueName).To(HavePrefix(name))
g.Expect(len(uniqueName)).To(BeNumerically(">", len(name)))
}
Loading