Amazon Elastic Container Service (EKS) for Beginners, Part 1

Introduction to EKS

Kubernetes is an open-source container orchestration platform that supports automated deployments and scaling for containerized workloads and services. Kubernetes uses a combination of Pod, Deployment, Service, and Replication objects to create a suitable runtime environment for containerized applications.

The Amazon Elastic Container Service for Kubernetes (EKS) is a managed Kubernetes service offered by Amazon Web Services (AWS). Kubernetes clusters are made of two distinct parts, namely:

• control plane: used to manage the Kubernetes cluster
• worker nodes: host containerized applications

The current implementation of EKS only supports version 1.10.0 of Kubernetes. The service implements a highly available control plane by creating appropriate resources in three availability zones, while the worker nodes and Elastic Compute Cloud (EC2) instances that are used to host containerized applications are managed by the users.

The EKS service automatically updates, patches, and heals the control plane with minimal interruption of traffic to end user applications. The service is tightly integrated with other AWS services, such as Identity Access Management (IAM) for roles and permissions, Elastic Load Balancer (ELB), EC2 for worker nodes, Virtual Private Cloud (VPC) for security, AWS Command Line Interface (CLI), Cloudwatch and Cloudtrail for monitoring and logging.

The Heptio-authenticator-aws tool developed by Heptio is used as a bridge between Kubernetes Role Based Access Control (RBAC) and AWS IAM credentials. This enables EKS to validate all Kubernetes cluster actions performed by the user against their IAM profile. The Webhook mode enables users to use the kubectl CLI to manage the EKS cluster.

Container applications developed to run on native Kubernetes runtimes can, in most cases, be migrated to EKS without any major updates to the application.

The EKS service is currently available in the US West (Oregon) (us-west-2) and US East (N. Virginia) (us-east-1) regions.

Architecture

Amazon EKS service implements high availability by running the Kubernetes control plane in three availability zones. The user provisioned worker nodes are standard EC2 instances managed with an auto scaling group. The control plane and the provisioned worker nodes are in the same Virtual Private Cloud (VPC). The ingress and egress traffic can be managed by creating appropriate security groups and route table rules.

The diagram below illustrates the high-level EKS architecture:

Deploying an EKS Cluster

There are three major steps to provision a EKS cluster:

1. Creating the VPC, internet gateway, route tables, IAM, and security groups to host the Kubernetes cluster
2. Provisioning the Kubernetes Control Plane
3. Provisioning the worker nodes to host the container application workloads

The following sections provide a step-by-step guide to provision a working instance of EKS.

Create the IAM Profiles

• To create the IAM role, go to the Amazon web console, navigate to the Services tab and choose IAM under Security, Identity & compliance
• Choose Roles in the left panel
• Click on Create Role and select EKSunder Choose the service that will use this role, and click Next
• Under the attached permission policy, there should be two policies, AmazonEKSClusterPolicy and AmazonEKSServicePolicy, that are auto attached for EKS
• Click on review, enter the Role name, and click on Create Role

__Note: __You can create specific/custom IAM policies and attach those IAM policies to users. The user must have administrative privileges to use the Amazon EKS APIs.

Create VPC and Subnets

• Create 1 VPC with three Subnets in it
• To create a VPC, go to the Amazon web console, navigate to the Services tab, and choose VPC under Networking and Content Delivery
• Choose Your VPCs in the left side panel
• Click on Create VPC, fill the Name tag and IPv4 CIDR block, and click the Yes, Create button
• Click on Subnets in the left panel
• Repeat the following steps three times and select the same VPC and a different Availability zone for each subnet
  • Click on Create Subnet and fill the Name tag, VPC, Availability Zone and IPv4 CIDR block, then click on the Yes, Create button

__Note: __If you encounter an error when creating the resources in the US-east-1a availability zone, select a different availability zone and repeat the steps.

Associate the Route Table with the Internet Gateway and Subnets

• Create an Internet Gateway (IGW), go to the Amazon web console, navigate to the Services tab and click on VPC under Networking and Content Delivery
• Click on Internet Gateways in the left panel
• Click on Create Internet Gateway, specify the Name tag of Internet Gateway and click on Create
• Choose the Internet Gateway that was created in the previous step, click on Actions, select Attach to VPC in the drop down and choose the VPC created in the previous step.
• Click on Route Tables in the left side panel
• Click on Create Route Table, specify the Name tag of the Route Table, choose the VPC that was created in the previous steps and click on the Yes, Create button
• Once the Route Table is created, select the Route Table that was previously created, then click on the Routes tab in the bottom panel
• Click on the Edit button and enter the Destination field as 0.0.0.0/0, then click on the Target field and it will show the Internet Gateway that we created in previous steps
• Click on the Subnet Associations tab that is right next to the Routes tab in the bottom panel
• Click on Edit button and check mark all the Subnets that were created in the previous section

Create the Security Group

• Create a Security Group (SG), go to the Amazon web console, navigate to the Services tab and choose VPC under Networking and Content Delivery
• Choose Security Groups on the left side panel
• Click on Create Security Group. Specify Name tag, Group name, Description, and select the VPC that was created in the previous section
• The suggested Inbound and Outbound traffic rules for the Security Group are as follows:

Control Plane Security Group

| - | Protocol | Port Range | Source | Destination | | ———— | ———— | ———— | ———— |
| Minimum inbound traffic | TCP | 443 | Worker node security group | - | | Recommended inbound traffic | TCP | 443 | Worker node security group | - | | Minimum outbound traffic | TCP | 10250 | - | Worker node security group | | Recommended outbound traffic | TCP| 1025-65535 | - | Worker node security group |

Worker Node Security Group

| - | Protocol | Port Range | Source |Destination | | ———— | ———— | ———— | ———— | | Minimum inbound traffic (from other worker nodes) | Any protocol you expect your worker nodes to use for inter-worker communication | Any ports you expect your worker nodes to use for inter-worker communication | Worker node security group | - | | Minimum inbound traffic (from control plane) | TCP | 10250 | Control plane security group | - | | Recommended inbound traffic | All, TCP | All, 1025-65535 | Worker node security group, Control plane security group | - | | Minimum outbound traffic | TCP | 443 | - | Control plane security group | | Recommended outbound traffic | All | All | - | 0.0.0.0/0 |

Create the EKS Control Plane

• Create an EKS cluster, go to the Amazon web console, navigate to the Services tab, and click on EKS under Compute.
• Click on Create cluster, specify the Cluster name, select the Kubernetes version, choose the Role ARN, VPC, Subnets, and security group created in the previous sections. Click on Create to create the EKS cluster.
• The EKS cluster is ready for use once the status changes from “Creating” to “Active” in the EKS dashboard.

__Note: __It may take between 10 to 15 minutes to create the cluster.

Configure Kubectl and Kube Configuration

• Download and install kubectl by executing the following commands:

Mac OS

curl -o kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/darwin/amd64/k
ubectl

Linux

curl -o kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/linux/amd64/kubectl

Windows

curl -o kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/windows/amd64/kubectl=.exe

• Run the following command to give execute permission to the kubectl binary:

chmod +x ./kubectl

• Add the kubectl binary to the path.

Mac OS

echo 'export PATH=$HOME/bin:$PATH' >> ~/.bash_profile

Linux

echo 'export PATH=$HOME/bin:$PATH' >> ~/.bashrc

• After installing the kubectl binary, verify it by executing the following command:

kubectl version --short --client 

__Output__:~ Client Version: v1.10.3

Create a Kube Configuration File for the EKS Cluster

• Ensure you have an AWS Access Key ID and AWS Access Key in ~/.aws/credentials

[default]  
aws_access_key_id = <AWS_ACCESS_KEY_ID>  
aws_secret_access_key = <AWS_SECRET_ACCESS_KEY>

• To create the AWS Access Key ID and AWS Access Key, go to the Amazon web console, navigate to the Services tab and choose IAM under Security, Identity & compliance
• Choose Users in the left panel
• Click on your User name, choose the Security credentials tab and click on create access keys

__Note: __Download the Access Key ID and Access Keys as they can be viewed only once.

• The following values are necessary to create / update the kube configuration file for the EKS cluster:

  • <endpoint-url>
  • <base64-encoded-ca-cert>
  • <cluster-name>
  • <role-arn> (Optional value for an IAM role to use with your user)

• To get the values for the EKS cluster, navigate to the Services tab and click on EKS under Compute, click on the EKS cluster that was just created in previous steps:

  • endpoint-url: Use the API server endpoint
  • base64-encoded-ca-cert: Use the contents of the Certificate Authority field
  • cluster-name: Use the EKS cluster name
  • role-arn: Use the Role ARN field

• Create / Update the config file in the ~/.kube directory (Mac OS and Linux) or the %USERPROFILE%/.kube directory (Windows) with the following content:

apiVersion: v1  
clusters:  
- cluster:  
    server: <endpoint-url>  
    certificate-authority-data: <base64-encoded-ca-cert>  
  name: kubernetes  
contexts:  
- context:  
    cluster: kubernetes  
    user: aws  
  name: aws  
current-context: aws  
kind: Config  
preferences: {}  
users:  
- name: aws  
  user:  
    exec:  
      apiVersion: client.authentication.k8s.io/v1alpha1  
      command: heptio-authenticator-aws  
      args:  
        - "token"  
        - "-i"  
        - "<cluster-name>"  
        # - "-r"  
        # - "<role-arn>"

• Save the changes
• Run the following command to test the kube configuration:

kubectl get namespaces

• The output must list the default namespaces in the EKS cluster:

  NAME         STATUS    AGE  
default        Active    5d  
kube-public   Active    5d  
kube-system   Active    5d

Install heptio-authenticator-aws

• Download the heptio-authenticator-aws

Mac OS

curl -o heptio-authenticator-aws https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/darwin/amd64/heptio-authenticator-aws

Linux

curl -o heptio-authenticator-aws https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/linux/amd64/heptio-authenticator-aws

Windows

curl -o heptio-authenticator-aws https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/windows/amd64/heptio-authenticator-aws.exe

• (Do this only for Mac OS and Linux). Execute the following command to give execute permissions to the heptio-authenticator-aws:

chmod +x ./heptio-authenticator-aws

• Add the heptio-authenticator-aws binary to the path

Mac OS

echo 'export PATH=$HOME/bin:$PATH' >> ~/.bash_profile

Linux

echo 'export PATH=$HOME/bin:$PATH' >> ~/.bashrc

• Execute the heptio-authenticator-aws command and confirm the setup is correct:

heptio-authenticator-aws help

Output:~ A tool to authenticate to Kubernetes using AWS IAM credentials

Usage:

  heptio-authenticator-aws [command]

Available Commands:

  help        Help about any command

  init        Pre-generate certificate, private key, and kubeconfig files for the server.

  server      Run a webhook validation server suitable that validates tokens using AWS IAM

  token       Authenticate using AWS IAM and get token for Kubernetes

  verify      Verify a token for debugging purpose

Flags:

  -i, --cluster-id ID     Specify the cluster ID, a unique-per-cluster identifier for your heptio-authenticator-aws installation.

  -c, --config filename   Load configuration from filename

  -h, --help              help for heptio-authenticator-aws

Use "heptio-authenticator-aws [command] --help" for more information about a command.

Create Worker Nodes Using Cloud Formation Templates

The EKS Worker nodes are standard EC2 instances. They connect to the EKS cluster’s control plane through the cluster’s API endpoint. AWS provides a standard CloudFormation template to provision Worker nodes.

• To provision the Worker nodes, go to the Amazon web console, navigate to the Services tab and click on CloudFormation under Management Tools.
• Click on Create Stack, select the Specify an Amazon S3 template URL option under Choose a template, and paste the Worker node template URL Worker, then click Next.

  https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-nodegroup.yaml

• In “Specific Details” enter a name for the Stack
• Enter the following details under the Parameters section:

EKS Cluster:

• ClusterName: Enter the EKS cluster name
• __ClusterControlPlaneSecurityGroup: __Choose the same Security Group that was used to create the EKS cluster.

Worker Node Configuration:

• NodeGroupName: Enter a unique node group name
• __NodeAutoScalingGroupMinSize: __Minimum number of instances to be in the EKS Cluster, use 1
• __NodeAutoScalingGroupMaxSize: __Maximum number of instances to be in the EKS Cluster, use 3
• NodeINstanceType: Select the type of instance depending on the workloads
• __NodeImageId: __For now, there are only two NodeImageId’s that are available, depending on the region:
  • US West (Oregon) (us-west-2) → ami-73a6e20b
  • US East (N. Virginia) (us-east-1) → ami-dea4d5a1
• __KeyName: __Select the keypair you want to use to ssh to the worker node instance

Worker Network Configuration:

• __VpcId: __Enter the EKS cluster’s VPC ID
• Subnets: Select the subnets for the EKS cluster’s control plane and the one for the worker nodes
• Click Next

Options:

• Create tags for the instances with key=value pairs
• Click on Next to review the details and click Create to create the Worker nodes for EKS cluster

Adding the Worker Nodes to the Kubernetes Cluster

• Once the Worker nodes are provisioned, execute the following command on your laptop to download the AWS authenticator’s configuration map file:

  curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-cm.yaml

• Before we execute the aws-auth-cm.yaml file, we need to replace <ARN of instance role (not instance profile)> with an actual value from the outputs of CloudFormation template:

apiVersion: v1  
kind: ConfigMap  
metadata:  
  name: aws-auth  
  namespace: kube-system  
data:  
  mapRoles: |  
    - rolearn: <ARN of instance role (not instance profile)>  
      username: system:node:{{EC2PrivateDNSName}}  
      groups:  
        - system:bootstrappers  
        - system:nodes

__Note: __Only replace the rolearn value, do not modify the rest of the file.

• To get the NodeInstanceRole, go to the Amazon web console, navigate to the Services tab, then click on CloudFormation under Management Tools.
• Click on the Worker node stack, then click on Outputs to see the NodeInstanceRole.
• Once you enter the value in the aws-auth-cm.yaml file, save the change, and execute the following command to add the Worker nodes to the EKS cluster:

kubectl apply -f aws-auth-cm.yaml

• Execute the following command to confirm that the Worker nodes are added to the EKS cluster:

kubectl get nodes

Conclusion

You now have a fully functional AWS EKS cluster with three worker nodes.

Upcoming:

• Deploying Kubernetes Dashboard
• Creating a Storage class for an EKS cluster
• Creating an Ingress router for an EKS cluster
• Deploying a demo application

About Levvel

Levvel helps clients transform their business with strategic consulting and technical execution services. We work with your IT organization, product groups, and innovation teams to design and deliver on your technical priorities.

We help our clients assess, design, implement, and refine their DevOps delivery processes and platforms. Our client engagements significantly reduce the time it takes to provision and deploy new products and consistently improve quality. We focus on quantifiable results while ensuring our clients can sustain these practices over the long-term.

For more information, contact us at hello@levvel.io.