nginx start can't get cpu limit and change worker_processes numbers

[Darren-wh]

nginx start can't get cpu limit and change worker_processes numbers.
I think you can add follow scripts to docker-entrypoint.sh.

CPU_CFS_QUOTA_US=`cat /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us`
CPU_CFS_PERIOD_US=`cat /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us`
if [ "$CPU_CFS_PERIOD_US" -ne "-1" ]
then
WORKER_PROCESSES=`awk 'BEGIN{printf "%.2f\n",'''$CPU_CFS_QUOTA_US'''/'''$CPU_CFS_PERIOD_US'''}'|awk '{print int($1+1)}'`
sed  -i "s/worker_processes  2/worker_processes  $WORKER_PROCESSES/g" /etc/nginx/nginx.conf
fi

add this script,The nginx can start with the worker_processes auto.

[thresheek]

Hi @Darren-wh!

Thanks for the suggestion, this fits nicely into entrypoints IMO.

I've just pushed a branch called "tune-worker-processes" to https://github.com/nginxinc/docker-nginx/tree/tune-worker-processes - it would be nice if you can rebuild the image & check if it works for your usecase. This tuning is not done by default, you'd need to set an environment variable NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE to any value to have an effect when launching a container.

Here's a list of actions needed to build & run it locally:

$ git clone https://github.com/nginxinc/docker-nginx
$ cd docker-nginx
$ git checkout -f tune-worker-processes
$ cd mainline/debian
$ docker build -t nginx-tuned:1.19.5 .
$ docker run -e NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE=1 -d --name nginx-tuned nginx-tuned:1.19.5

At this point the container named "nginx-tuned" should be running, and you can check if the limits were correctly applied via:

$ docker exec nginx-tuned cat /etc/nginx/nginx.conf

In my case it resulted in something like:

user  nginx;
# Commented out by 30-tune-worker-processes.sh on Tue Dec  1 14:01:33 UTC 2020
#worker_processes  1;
worker_processes 8;
...

[Darren-wh]

@thresheek It can get Deployment resources CPU limits and then auto to set worker_processes when starting Nginx?

[thresheek]

@Darren-wh Yes. That is also what I would like you to test. :-)

[Darren-wh]

@thresheek great

[lin1005q]

Add command to a k8s pod, this script(30-tune-worker-processes.sh) does not work.

eg:

command:
- wait-for-it.sh
- server:8080
- --timeout=1
- --strict
- --
- nginx
- -g
- daemon off;
env:
- name: NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE
  value: "1"

[Darren-wh]

@lin1005q which nginx version used？
env:

• name: NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE
  value: "true"

[lin1005q]

This is my Dockerfile.

FROM nginx:alpine
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
RUN apk add --no-cache bash coreutils
ADD dist /usr/share/nginx/html
ADD default-itoa.conf /etc/nginx/conf.d/default.conf
ADD $PWD/wait-for-it.sh /usr/local/bin/wait-for-it.sh
RUN chmod +x /usr/local/bin/wait-for-it.sh

bash-5.1# nginx -V
nginx version: nginx/1.21.1
built by gcc 10.3.1 20210424 (Alpine 10.3.1_git20210424) 
built with OpenSSL 1.1.1k  25 Mar 2021
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-Os -fomit-frame-pointer -g' --with-ld-opt=-Wl,--as-needed,-O1,--sort-common

[Darren-wh]

@lin1005q The Nginx version is ok. Maybe other problems. I think you can check the NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE value has been configured in a pod or you can check the value request and limit about service Whether has been configured. I have been tested the parameter of "NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE" before, It can works.

[lin1005q]

@Darren-wh NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE value is ok.And when I not set command, the nginx work process number is 1(limit 1.0),but when set command,the number is 16(host 16).

@thresheek

[Darren-wh]

@lin1005q What request and limit resources of Kubernetes do you set? If you add
env NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE, you must set the request and limit resources of Kubernetes.

[lin1005q]

Thanks!!! The following are the steps to reproduce.

download wait-for-it.sh

wait-for-it.sh.zip

And unzip it

docker build -t nginx-test .

FROM nginx:alpine
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
RUN apk add --no-cache bash coreutils
ADD $PWD/wait-for-it.sh /usr/local/bin/wait-for-it.sh
RUN chmod +x /usr/local/bin/wait-for-it.sh

kubectl apply -f nginx-test.yaml

replace 192.168.10.6:443 to another port

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx-test
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx-test
    spec:
      containers:
      - command:
        - wait-for-it.sh
        - 192.168.10.6:443
        - --timeout=1
        - --strict
        - --
        - nginx
        - -g
        - daemon off;
        env:
        - name: NGINX_ENTRYPOINT_WORKER_PROCESSES_AUTOTUNE
          value: "1"
        image: nginx-test
        imagePullPolicy: Never
        name: nginx-test
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 1000m
            memory: 500Mi
          requests:
            cpu: 500m
            memory: 100Mi

[root@192-168-50-90 nginx-test]# kubectl exec -it -n default nginx-test-b84867d5-fmmj9 -- ps -ef
PID   USER     TIME  COMMAND
    1 root      0:00 nginx: master process nginx -g daemon off;
   19 nginx     0:00 nginx: worker process
   20 nginx     0:00 nginx: worker process
   21 nginx     0:00 nginx: worker process
   22 nginx     0:00 nginx: worker process
   23 nginx     0:00 nginx: worker process
   24 nginx     0:00 nginx: worker process
   25 nginx     0:00 nginx: worker process
   26 nginx     0:00 nginx: worker process
   27 nginx     0:00 nginx: worker process
   28 nginx     0:00 nginx: worker process
   29 nginx     0:00 nginx: worker process
   30 nginx     0:00 nginx: worker process
   31 nginx     0:00 nginx: worker process
   32 nginx     0:00 nginx: worker process
   33 nginx     0:00 nginx: worker process
   34 nginx     0:00 nginx: worker process
   43 root      0:00 ps -ef

delete command && kubectl apply -f

[root@192-168-50-90 nginx-test]# kubectl exec -it -n default nginx-test-79f49f8847-nxgtv -- ps -ef
PID   USER     TIME  COMMAND
    1 root      0:00 nginx: master process nginx -g daemon off;
   97 nginx     0:00 nginx: worker process
   98 root      0:00 ps -ef