NGINX ingress operator creates SCC: nginx-ingress-scc and sets groups: system:authenticated

[lniesz]

Describe the bug
After installing NGINX ingress operator no new POD can start in OKD and Openshift cluster.

A new SCC is created by nginx ingress operator: nginx-ingress-scc in which the groups attribute is set to system:authenticated. This results that all new POD gets the SCC: nginx-ingress-scc in annotation: openshift.io/scc and because of this so far running PODs can not start. When scaling the deployment up I get the following error:
pods "productpage-v1-55679b7dc5-" is forbidden: unable to validate against any security context constraint: [spec.volumes[1]: Invalid value: "emptyDir": emptyDir volumes are not allowed to be used spec.containers[1].securityContext.securityContext.runAsUser: Invalid value: 102: must be: 101 spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 101: must be in the ranges: [1000910000, 1000919999] capabilities.add: Invalid value: "NET_BIND_SERVICE": capability may not be added spec.containers[1].securityContext.securityContext.runAsUser: Invalid value: 102: must be in the ranges: [1000910000, 1000919999] fsGroup: Invalid value: []int64{1000910000}: 1000910000 is not an allowed group spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 101: must be

To Reproduce

1. Create a new namespace: nginx-operator
2. Install NGINX ingress operator in that namespace
3. Check that a new SCC is created: nginx-ingress-scc
4. Scale down a so far running POD
5. Scale up the POD again and check the error message
6. As the POD is not started it is not even possible to check that annotation: openshift.io/scc, so I had to enable all stuff in the nginx-ingress-scc to be able to start the POD with that, and then realized that because of the attribute groups is set to system:authenticated in the SCC the SCC was assigned to all new POD creating on the cluster. Now the old SCC: restricted was set to groups: system:authenticated also.

Expected behavior
I think it would be enough to set the users attribute of the new SCC to

• 'system:serviceaccount:nginx-operator:my-nginx-ingress-controller'
  This way the ingress controller POD in nginx-operator namespace would get the SCC which is sufficient for working the ingress controller.

Your environment
I've tested this on OKD 4.5 and after many hours of test and fail I've tested on Openshift 4.5 also.
NGINX ingress operator is 0.0.6

[nccurry]

This looks hard coded here:
https://github.com/nginxinc/nginx-ingress-operator/blob/release-0.0.6/pkg/controller/nginxingresscontroller/scc.go#L38

The ideal way of configuring this would be to apply the SCC only to the service account the nginx-ingress-operator is using or directly on the pod manifests nginx-ingress uses to function.

[lucacome]

Hi @lniesz, thanks for reporting the bug, and @nccurry thanks for the suggestion.

We'll look into this and let you know once we resolve the issue.