Terraform Module — Cluster Autoscaler for Kubernetes

In this article, we’ll delve into the technical architecture of a Terraform module engineered to deploy the Cluster Autoscaler on a Kubernetes cluster. The Cluster Autoscaler serves as an indispensable instrument that dynamically adjusts the size of a Kubernetes cluster by scaling the number of nodes up or down in response to the current workload. This optimization of resource allocation enhances the overall efficiency of the cluster.

Overview

The Terraform module establishes the following Kubernetes resources:

• ServiceAccount
• ClusterRole
• Role
• ClusterRoleBinding
• RoleBinding
• Deployment

These resources play a crucial role in deploying and configuring the Cluster Autoscaler within a Kubernetes cluster.

Design

ServiceAccount

The kubernetes_manifest resource for cluster-autoscaler_service_account creates a ServiceAccount in the designated namespace. This ServiceAccount bears an annotation containing the IAM role ARN, enabling it to assume the necessary AWS permissions. The module takes into account the specified namespace, creating the ServiceAccount accordingly.

ClusterRole

The kubernetes_manifest resource for clusterrole_cluster_autoscaler generates a ClusterRole outlining the essential permissions for the Cluster Autoscaler. This encompasses permissions to administer nodes, pods, services, replication controllers, and other resources. This ClusterRole is established to furnish the Cluster Autoscaler with the required permissions.

Role

The kubernetes_manifest resource for role_kube_system_cluster_autoscaler constructs a Role in the specified namespace. This role grants permissions to manage configmaps for the Cluster Autoscaler. The Role is designed to enable the Cluster Autoscaler to handle configmaps.

ClusterRoleBinding

The kubernetes_manifest resource for clusterrolebinding_cluster_autoscaler produces a ClusterRoleBinding that binds the ClusterRole to the ServiceAccount. This endows the Cluster Autoscaler with the necessary permissions at the cluster level. The ClusterRoleBinding is designed to ensure that the Cluster Autoscaler possesses the required permissions to manage the Kubernetes cluster.

RoleBinding

The kubernetes_manifest resource for rolebinding_kube_system_cluster_autoscaler creates a RoleBinding in the designated namespace. This binds the Role to the ServiceAccount, bestowing the Cluster Autoscaler with the necessary permissions within the namespace. The RoleBinding is designed to guarantee that the Cluster Autoscaler has the required permissions to manage the Kubernetes namespace.

Deployment

The kubernetes_manifest resource for deployment_kube_system_cluster_autoscaler generates a Deployment that deploys the Cluster Autoscaler container. The Deployment resource delineates the container image, resource limits and requests, and requisite environment variables. The container is configured with command-line arguments to regulate the Cluster Autoscaler's behavior, such as scale-down delay and scan interval. The Deployment is designed to ensure that the Cluster Autoscaler is deployed and operational within the Kubernetes cluster.

Usage

To utilize this Terraform module, simply reference it in your Terraform configuration and supply the necessary input variables. The module offers a versatile set of input variables that can be effortlessly tailored to suit the needs of your Kubernetes environment.

hcl

module "cluster_autoscaler" {
  source = "path/to/terraform/module"
  # Required input variables
  eks_cluster_name             = "my-eks-cluster"
  permissions_boundary         = "arn:aws:iam::123456789012:policy/PermissionsBoundary"
  tags                         = { "example": "tag" }
  # Optional input variables with default values
  # ...
}

After applying the Terraform configuration, the Cluster Autoscaler will be deployed and configured in the specified Kubernetes cluster.

Input Variables

The module accepts the following input variables

• eks_cluster_name: The EKS cluster where this autoscaler will be provisioned (required).
• permissions_boundary: ARN of the policy that is used to set the permissions boundary for the autoscaling role (required).
• tags: Tags supplied from upstream to merge with generated tags (default: {}).
• cluster_autoscaler_name: The name of the Cluster Autoscaler resources (default: “cluster-autoscaler”).
• cluster_autoscaler_namespace: The namespace where the Cluster Autoscaler will be deployed (default: “kube-system”).
• cluster_iam_role_arn: The ARN of the IAM role to be associated with the ServiceAccount (required).
• cluster_k8s_addon: The label value for the k8s-addon label (default: “cluster-autoscaler.addons.k8s.io”).
• cluster_cpu_limit: The CPU limit for the Cluster Autoscaler container (default: “100”).
• cluster_cpu_request: The CPU request for the Cluster Autoscaler container (default: “100”).
• cluster_memory_limit: The memory limit for the Cluster Autoscaler container (default: “600Mi”).
• cluster_memory_request: The memory request for the Cluster Autoscaler container (default: “600Mi”).
• cluster_docker_image_url: The Docker image URL for the Cluster Autoscaler (default: “cluster-autoscaler:v1.22.2”).
• replicas: The number of replicas for the Cluster Autoscaler (default: “1”).
• http_proxy: HTTP proxy value (default: “SET_HTTP_PROXY”).
• https_proxy: HTTPS proxy value (default: “SET_HTTPS_PROXY”).
• no_proxy: A comma-separated list of domains that should bypass the proxy (default: “SET_NO_PROXY”).
• scale_down_delay_after_add: The delay before scaling down a node after it has been added (default: “10m”).
• scale_down_delay_after_delete: The delay before scaling down a node after it has been deleted (default: “10s”).
• scale_down_delay_after_failure: The delay before scaling down a node after a scale-down failure (default: “3m”).
• scan_interval: The interval between consecutive scans of the cluster state by the Cluster Autoscaler (default: “10s”).

Output Variables

There are no output variables for this module.

Requirements

To use this module, you will need Terraform v1.0.0 or later and Kubernetes provider v2.5.0 or later.

Conclusion

This Terraform module delivers a straightforward and efficient method for deploying the Cluster Autoscaler in a Kubernetes cluster. By adhering to best practices and offering a flexible set of input variables, the module can be seamlessly integrated into various Kubernetes environments to enable autoscaling based on workload. By providing a comprehensive overview of the Kubernetes resources created by the module, users can gain a deeper understanding of how the Cluster Autoscaler is deployed and configured in their Kubernetes environment.

Github repo — https://github.com/apur27/public/tree/master/cluter-autoscaler