Guidance on how to configure scaling to safeguard against port exhaustion

[cmbankester]

We use NGF for proxying websocket connections to application pods. Our application requires that a large number of websockets get established at one time. Nginx handles this load extremely well, but recently we have been having to scale the data plane replicas up to account for outbound port exhaustion between the nginx agents and the backend pods. I understand that there is support for configuring an HPA as part of the Helm installation, but it appears it can only scale based on memory and cpu (https://docs.nginx.com/nginx-gateway-fabric/reference/api/#gateway.nginx.org%2fv1alpha2.AutoscalingSpec) and not based on connection count or outbound port usage or something like that which would enable us to have the data plane scale before we start seeing the port exhaustion errors in the nginx logs.

I should mention that we have increased the worker_connections parameter to its maximum in order to minimize the number of replicas that we need to spin up in the cluster, so it's perhaps not unexpected that we are seeing port exhaustion. We currently have only a subset of our customers running on the cluster and three data plane replicas seems sufficient to prevent the exhaustion, but in order to move more of our customers over, I need the data plane to be able to scale on its own.

Any advice or suggestions I could try? Maybe if we adjust the worker_connections lower then the memory and/or cpu usage will be more consistent and easier to scale off of?

[sjberman]

The autoscaling spec offers the metrics field, where you can define custom metrics. You should be able to integrate a metrics adapter (I'm assuming Prometheus supports this) with the k8s API so that the HPA can access any custom metrics to make a scaling decision.