Kubernetes 1.6 in AWS Kube Up and Deploying Your First Public Service

Published in

Dronzebot

7 min readApr 14, 2017

Kubernetes in AWS is sometimes the only way.

I love mashups. Making them is a way to stand on the shoulders of giants and remix. It is fun. So this guide is kindof a mashup because I found that the guides to doing something meaningful that gave you that “ah ha” around an important technology like Kubernetes were missing. So that’s what this is; A cookbook to not only stand up a complete cluster in AWS but also to deploy your first public service.

Make sure to look at the sources at the bottom. They are chocked full of helpful details I left out of this guide. There are also many ways to do this I chose to have it be as simple as possible for the guide.

Things You Will Need

Your “local machine”. For this guide its the developer’s MacBook Pro.
AWS CLI Installed on the local machine you will use to stand up the cluster and use the kubernetes dashboard.
Pretty much wide open permissions on the AWS CLI user for IAM, CloudFormation, S3, EC2, Route53, KeyManager, sorry I don’t have a full list but building a cluster requires a lot of permissions.
Installed kubectl on your local machine (see guide below)
Installed kube-aws on your local machine (see guide below)
A bucket in S3 that cloud formation docs will be rendered to.
Monthly Costs (see estimation guide below)

Install Kubectl on Your Local Environment

Something that is not made clear in the documentation is that you, at some point, will need to install kubectl on a machine that you can run a browser on. You can also stand up the cluster on the same machine if you wish, but you will need to have

$ curl -O https://storage.googleapis.com/kubernetes-release/release/v1.6.1/bin/linux/amd64/kubectl
$ chmod +x kubectl
$ mv kubectl /usr/local/bin/kubectl

Getting Your Cluster Stood Up

1.5 kube-up is to be ignored

One of the confusing things to face right away is that the docs for the latest version of Kubernetes, what you get as top hit when you google “kubernetes in aws” is s bit confusing. It clearly says “kube-up is no longer supported in kubernetes 1.6” but it continues on and doesn’t tell you that the correct thing to do for 1.6 aws is to use the kube-aws cloudformation templates from CoreOs. Don’t try to download and setup kubernetes using the cluster turnup scripts. Go on to the CoreOs Steps or use this guide.

The kube-aws CoreOs Steps are required for 1.6

Once you go to the correct set of instructions things start moving along. kube-aws is a tool that you use to configure how you want your cluster to be configure. It uses that configuration to “render” a set of cloud formation templates that can be used to stand up your cluster. This section is a summary of the CoreOs Steps.

Create a key for the cluster

$ aws kms — region=us-west-2 create-key — description=”kube-aws assets mydomain.k8s.dev”{
 “KeyMetadata”: {
 “Origin”: “AWS_KMS”,
 “KeyId”: “0c7c10ad-5881–4c2e-9df7-a368dc8a6a70”,
 “Description”: “kube-aws assets mydomain.k8s.dev”,
 “Enabled”: true,
 “KeyUsage”: “ENCRYPT_DECRYPT”,
 “KeyState”: “Enabled”,
 “CreationDate”: 1492127610.42,
 “Arn”: “arn:aws:kms:us-west-2:605112536939:key/0c7c10ad-5881–4c2e-9df7-a368dc8a6a70”,
 “AWSAccountId”: “605112536939”
 }
}

Create an S3 Bucket for Rendered Templates

$ aws s3api create-bucket — bucket mydomain.k8s.dev — region us-west-2 — create-bucket-configuration LocationConstraint=us-west-2{
 “Location”: “http://mydomain.k8s.dev.s3.amazonaws.com/"
}

Initialize the Cluster config.yml

This sets up the primary configuration. This will be the gateway fro kubectl to get to your cluster so it ends up being over https and needs DNS. Once the cluster is set up we will create the A record that maps the load balancer to the DNS k8sdev.mydomain.com endpoint.

$ kube-aws init \
 — cluster-name=mydomain-dev-cluster \
 — external-dns-name=k8sdev.mydomain.com \
 — region=us-west-2 \
 — availability-zone=us-west-2a \
 — key-name=mydomain-oregon-dev \
 — kms-key-arn=”arn:aws:kms:us-west-2:605112536939:key/0c7c10ad-5881–4c2e-9df7-a368dc8a6a70"

Render The Stack

Its important to note that credentials are required for kubectl to access your cluster. The easy way is to have kube-aws render those credentials. The better way is to manage your own certificates but doing that is out of scope for this article. A guide like this one, is probably helpful.

$ kube-aws render credentials — generate-ca

We are now ready to render the stack, this generates the templates that will be used to stand up our cluster.

$ kube-aws render stack

Bring Up Your Cluster

Are you ready. This is the part we have been waiting for.

$ kube-aws up --s3-uri s3://mydomain.k8s.dev

There are three stacks that get created. These are screen shots using our domain but the namespace would be whatever you mapped to mydomain.

And three instances from each stack:

As well as a load balancer that will get mapped to your dns:

And we are up and running for about $200/month

Routing DNS Using Route53

The cluster is now inde

Estimating your costs

aws cloudformation estimate-template-cost — template-url https://s3-us-west-2.amazonaws.com/mydomain.k8s.dev/kube-aws/clusters/mydomain-dev-cluster/exported/stacks/control-plane/stack.json
{
 “Url”: “http://calculator.s3.amazonaws.com/calc5.html?key=cloudformation/0a27d1c7-1a80-47f0-b78d-41b07c248711"
}

Control Pane — $94.32

aws cloudformation estimate-template-cost — template-url https://s3-us-west-2.amazonaws.com/mydomain.k8s.dev/kube-aws/clusters/mydomain-dev-cluster/exported/stacks/nodepool1/stack.json — parameters ParameterKey=ControlPlaneStackName,ParameterValue=foobar
{
 “Url”: “http://calculator.s3.amazonaws.com/calc5.html?key=cloudformation/c69d0e39-64a1-4298-a827-7ecc030984dc"
}

Node Pool — $36.51

Using kubectl to Manage Your Cluster

This guide could become giant if we try to make a comprehensive guide to kubectl so we wont do that. Here is a cheatsheat and the overview that can take farther, but we will give you the basics:

The kubeconfig File

This file, and the associated keys, need to be on whatever machine you have kubectl installed on and are planning on managing your cluster.

apiVersion: v1
kind: Config
clusters:
- cluster:
 certificate-authority: credentials/ca.pem
 server: https://k8sdev.mydomain.com
 name: kube-aws-mydomain-dev-cluster-cluster
contexts:
- context:
 cluster: kube-aws-mydomain-dev-cluster-cluster
 namespace: default
 user: kube-aws-mydomain-dev-cluster-admin
 name: kube-aws-mydomain-dev-cluster-context
users:
- name: kube-aws-mydomain-dev-cluster-admin
 user:
 client-certificate: credentials/admin.pem
 client-key: credentials/admin-key.pem
current-context: kube-aws-mydomain-dev-cluster-context

View your running pods

$ kubectl --kubeconfig=kubeconfig get pods --all-namespacesNAMESPACE NAME READY STATUS RESTARTS AGE
kube-system heapster-v1.3.0–76786035-wf0h6 2/2 Running 0 17h
kube-system kube-apiserver-ip-10–0–0–117.us-west-2.compute.internal 1/1 Running 0 18h
kube-system kube-controller-manager-ip-10–0–0–117.us-west-2.compute.internal 1/1 Running 0 18h
kube-system kube-dns-3816048056–239nb 4/4 Running 0 18h
kube-system kube-dns-3816048056–4xphf 4/4 Running 0 17h
kube-system kube-dns-autoscaler-1464605019-xp486 1/1 Running 0 18h
kube-system kube-proxy-ip-10–0–0–117.us-west-2.compute.internal 1/1 Running 0 18h
kube-system kube-proxy-ip-10–0–0–67.us-west-2.compute.internal 1/1 Running 0 17h
kube-system kube-scheduler-ip-10–0–0–117.us-west-2.compute.internal 1/1 Running 0 18h
kube-system kubernetes-dashboard-2396447444–2t6rg 1/1 Running 0 17h
kube-system kubernetes-dashboard-v1.5.1–5gx2j 1/1 Running 0 18h

Using The Dashboard in Our Local Environment

The dashboard is not something that gets exposed to the public endpoint, that’s because it is your command center. Think of it as an extension of kubectl, so it should proxy through to your local machine.

To do this run the proxy:

$ kubectl --kubeconfig=kubeconfig proxyStarting to serve on 127.0.0.1:8001

Once it is running it should proxy to http://localhost:8001/

Deploying Your First Public Service

Ok so the moment of truth. This is why we did it all. We want to make a service available on the internet. This is where the magic happens because behind the scenes our service is going to orchestrate the creation and binding to an AWS ELB as part of the platform service. This will allow us to create a DNS entry and an A record with alias to myservice.mydomain.com that’s cool because now we can manage and deploy internet services only with kubernetes and docker. Kickass.

Alright we did it. Now you will need to wait just a little time for the service to bind to the ELB. When its up you should see a new ELB in aws EC2:

And the service should be fully deployed and mapped to that ELB:

Ok for the moment of truth: lets go to the (this one below is fake so dont really try) endpoint:
http://a13b42f3620d111e7b.us-west-2.elb.amazonaws.com:8080/

And BOOM: