Preserve Source IP when using Ingress

Some applications require a Kubernetes service of type LoadBalancer which preserves the source IPs of incoming packets. Example: Ingress controllers. As the Network Load Balancer(NLB) integration is not yet available in Managed Kubernetes, a service is exposed by attaching a public IP to a viable Kubernetes node. This node serves as a load balancer using kube-proxy.

Note: This works fine with services that use externalTrafficPolicy: Cluster, but in this case, the client's source IP is lost.

To preserve the client source IP address, Kubernetes services with externalTrafficPolicy: Local need to be used. This configuration ensures that packets reaching a node are only forwarded to pods that run on the same node, preserving the client source IP. Therefore, the load balancer IP address of the service needs to be attached to the same node running the ingress controller pod.

This can be achieved with different strategies. One approach is to use a DaemonSet to ensure that a pod is running on each node. However, this approach is feasible only in some cases, and if a cluster has a lot of nodes, then, using DaemonSet could lead to a waste of resources.

For an efficient setup, you can schedule pods to be run only on nodes of a specific node pool using NodeSelectors. The node pool needs to have labels that can be used in the node selector. To ensure that the service's load balancer IP is also attached to one of these nodes, annotate the service with cloud.ionos.com/node-selector: key=value, where key and value are the labels of the node pool.

The following example shows how to install the ingress-nginx helm chart as DaemonSet with node selector and to configure the controller service with the required annotation.

1. Create a node pool with a label nodepool=ingress:

   ionosctl k8s nodepool create --cluster-id <cluster-id> \
       --name ingress --node-count 1 --datacenter-id <datacenter-id> --labels nodepool=ingress

2. Create a values.yaml file for later use in the helm command with the following content:

   controller:
     nodeSelector:
       nodepool: ingress
     service:
       annotations:
         cloud.ionos.com/node-selector: nodepool=ingress
     kind: DaemonSet

3. Install ingress-nginx via helm with the following command:

   helm upgrade --install ingress-nginx ingress-nginx \
   --repo https://kubernetes.github.io/ingress-nginx \
   --namespace ingress-nginx --create-namespace -f values.yaml