Managed Kubernetes facilitates the fully automated setup of Kubernetes clusters. Managed Kubernetes also simplifies and carefully supports the automation of Continuous Integration and Continuous Delivery/Continuous Deployment (CI/CD) pipelines for testing and deployment.
IONOS Managed Kubernetes solution offers automatic updates, security fixes, versioning, upgrade provisioning, high availability, geo-redundant control plane, and full cluster administrator level access to Kubernetes API.
Everything related to Managed Kubernetes can be controlled in the DCD via the dedicated Kubernetes Manager. The Manager provides a complete overview of your provisioned Kubernetes clusters and node pools, including their statuses. The Manager allows you to create and manage clusters, create and manage node pools, and download the kubeconfig file.
The control plane manages the worker nodes and the pods in the cluster. In production environments, the control plane usually runs across multiple computers. A cluster usually runs multiple nodes, providing fault tolerance and high availability.
Kube-controller-manager manages controllers that provide functionalities such as deployments, services, etc. For more information, see kube-controller-manager.
The API server is a component of the Kubernetes control plane that exposes the Kubernetes API. The API server is the front end for the Kubernetes control plane. Kube-apiserver is designed to scale horizontally. It scales by deploying more instances. You can run several instances of kube-apiserver and balance traffic between those instances. For more information, see kube-apiserver.
Kube-scheduler distributes pods to nodes. Pods must be created with resource limits so that nodes are not over-commissioned. For more information, see kube-scheduler.
Managed Kubernetes supports regional control planes that provide a distributed and highly-available management infrastructure within a chosen region. It also offers a hidden control plane for both Public and Private Node Pools. Control plane components like kube-apiserver, kube-scheduler, and kube-controller-manager are not visible to the users and cannot be modified directly.
The kube-apiserver can only be interacted with by using its REST API.
The hidden control plane is deployed on Virtual Machines (VMs) running in a geo-redundant cluster in the chosen region. For more information about the control plane components, refer to the Kubernetes documentation. and Kubernetes API.
The Managed Kubernetes clusters have a Calico CNI plugin. Its primary function is to automatically assign IP addresses, set up network interfaces, and establish connectivity between the pods. Calico also allows the use of network policies in the Kubernetes cluster. For more information, see Kubernetes CNI and Network Policies.
Managed Kubernetes does not currently offer an option to choose a different CNI plugin, nor does it support customers that do so on their own.
CNI affects the whole cluster network, so if changes are made to Calico or a different plugin is installed, it can cause cluster-wide issues and failed resources.
The CSI driver runs as a deployment in the control plane to manage volumes for Persistent Volume Claims (PVCs) in the IONOS Cloud and to attach them to nodes.
Note: Network File Storage (NFS) Kubernetes integration is currently available on a request basis. To access this product, please contact your sales representative or IONOS Cloud Support.
Please refer to the Network File Storage (NFS) Kubernetes integration documentation for more information.
IONOS Cloud volumes are represented as Persistent Volume (PV) resources in Kubernetes. The PV reclaim policy determines what happens to the volume when the PV is deleted. The Retain reclaim policy skips deletion of the volume and is meant for manual reclamation of resources. In the case of dynamically provisioned volumes, the CSI driver manages the PV; the user cannot delete the volume even after the PV is deleted.
The PV has resource finalizers that ensure that Cloud resources are deleted. The finalizers are removed by the system after Cloud resources are cleaned up, so removing them prematurely is likely to leave resources behind.
This may be desired when a dynamically provisioned volume is left over or if an external volume should be exposed to Kubernetes workload.
Warning: Do not import your Managed Kubernetes node's root volumes. They are fully managed outside the Kubernetes cluster, and importing them will cause conflicts that may lead to service disruptions and data loss.
Dynamically provisioned PVs are created by the CSI driver, which populates the resource ownership annotations and information gathered from the IONOS Cloud API. For statically managed PVs, this data must be provided by the user. For statically managed PVs this data must be provided by the user.
The following fields should be modified according to the volume that is imported:
spec.capacity.storage: Should contain the size of the volume with suffix G (Gigabyte).
spec.csi.volumeHandle: Volume path in the IONOS Cloud API. Omit the leading slash (/)
spec.nodeAffinity.required.nodeSelectorTerms.matchExpressions.values: Must contain the Virtual Data Center ID from the Volume path.
Creating this PV will allow it to be used in Pod by referencing it in a PVC's spec.volumeName.
Note: Be aware that the imported volume will only be deleted if it is Bound.
A cluster autoscaler is a tool that automatically adjusts the size of the Kubernetes cluster when one of the following conditions is true:
There are pods that failed to run in the cluster due to insufficient resources.
There are nodes in the cluster that have been underutilized for an extended period of time, and their pods can be placed on other existing nodes.
For more information, see Cluster Autoscaler and its FAQ.
The cluster autoscaler increases the node pool if pods cannot be scheduled due to a lack of resources. In addition, adding a node (from the node pool that needs to be increased) should provide a remedy for the situation. If there are no node pools that provide enough nodes to schedule a pod, the autoscaler will not enlarge.
The cluster autoscaler reduces the node pool if a node is not fully utilized for an extended period of time. A node is underutilized when it has a light load, and all of its pods can be moved to other nodes.
Yes, only node pools with active autoscaling are managed by the autoscaler.
No, the autoscaler cannot increase the number of nodes in the node pool above the maximum specified by the user or decrease it below the specified minimum. In addition, the quota for a specific contract cannot be exceeded using the autoscaler. The autoscaler cannot reduce the number of nodes in the node pool to 0.
Yes, it is possible to enable and configure encryption of secret data. For more information, see Encrypting Confidential Data at Rest.
All components installed in the cluster are updated. This includes the K8s control plane itself, CSI, CCM, Calico, and CoreDNS . With cluster maintenance, several components that are visible to customers are updated and reset to our values. For example, changes to CoreDNS are not permanent and will be removed at the next maintenance. It is currently not possible to set your own DNS records in the CoreDNS configuration, but this will be possible later.
Managed components that are regularly updated:
The maintenance time window is limited to four hours for Public and Private Node Pools. If all of the nodes are not rebuilt within this time, the remaining nodes will be replaced at the next scheduled maintenance. To avoid taking more time for your updates, it is recommended to create node pools with no more than 20 nodes.
If old nodes are replaced with new ones during maintenance, the new nodes will subsequently have different or new public IP addresses. You can pre-specify a list of public IP addresses from which entries for new nodes are taken. In such a way, the list of possible host addresses is limited and predictable. For example, to activate them differently through a whitelist.
The Kubernetes cluster control plane and the corresponding node pools can have different versions of Kubernetes. Node pools can use older versions than the control plane, but not vice versa. The difference between the minor versions must not be more than 1.
There is a distinction between patch version updates and minor version updates. You must initiate all version updates. Once initiated, the version updates are performed immediately. However, forced updates will also occur if the version used by you is so old that we can no longer support it. Typically, affected users receive a support notification two weeks before a forced update.
The Kubernetes API is secured with Transport Layer Security (TLS). Traffic between the nodes and the control plane is secured by mutual TLS, which means that both sides check whether they are talking to the expected remote station.
If clusters or node pools are created or modified, the operation may fail, and the cluster or node pool will go into a Failed status. In this case, our team is already informed because we monitor it. However, sometimes it can also be difficult for us to rectify the error since the reason can be a conflict with the client's requirements. For example, if a LAN is specified that does not exist at all or no longer exists, a service update becomes impossible.
If the node is in a NotReady state, and there is not enough RAM or the node runs out of the RAM space, an infinite loop occurs in which an attempt is made to free the RAM. This means the node cannot be used, and the executables must be reloaded from the disk. The node is busy with disk Input/Output (I/O). In such a situation, we recommend doing resource management to prevent such scenarios. For more information, see Requests and limits.
The IONOS Kubernetes currently does not support the usage of your own CAs or your own TLS certificates in the Kubernetes cluster.
You can reserve Public Node Pools in multiple locations in the same cluster. This allows simple geo-redundancy to be configured and implemented. The control plane is geo-reserved (within Germany). There are several replicas running in different locations.
If a node is unavailable, like too many pods are running on it without resource limits, it can be replaced. To do this, you can use the following API endpoint:
A Private Node Pool ensures that the nodes are not connected directly to the internet; hence, the inter-node network traffic stays inside the private network. However, the control plane is still exposed to the internet and can be protected by restricting IP access.
Clusters and node pools turn yellow when a user or an automated maintenance process initiates an action on the resources. This locks the clusters and node pool resources from being updated until the process is finished, and they do not respond during this time.
A NAT is required to enable outbound traffic between the cluster nodes and the control plane. For example, to be able to retrieve container images.
Kubernetes clusters only support public networks only if they are VMs but not LAN networks.
Yes, if your node pool is configured to have a network interface in the same network as the VMs that you want to access, then you can add nodes.
Public Node Pools within a Kubernetes cluster are configured by defining a public dedicated node pool. Networking settings are specified to include public IP addresses for external access.
Private Node Pools within a Kubernetes cluster are configured by ensuring that each node in a pool has a distinct private network, while nodes within the same pool share a common private network.
It is crucial to set up these node pools with a network interface aligned with the network of the intended VMs when adding nodes to Kubernetes clusters.
The Private Cross Connect is required to enable node-to-node communication across all node pools belonging to the same Kubernetes cluster. This ensures that node pools in different VDCs can communicate.
No, the private NAT Gateway is not intended to be used for arbitrary nodes.
The Public Node Pools support the LoadBalancer service type. However, the Private Node Pools currently do not support the LoadBalancer service type.