Posts Setting up a Kubernetes cluster on Digital Ocean with SSL
Post
Cancel

Setting up a Kubernetes cluster on Digital Ocean with SSL

It is now possible to create/maintain Kubernetes clusters on Digital Ocean quite easily. In this post I will show you how to create a cluster with a pod that has a public endpoint; this endpoint will be secured with a Let’s Encrypt SSL certificate. Some basic Kubernetes knowledge is assumed so i won’t go into detail explaining what a deployment, service or pod is, or how to work with kubectl. N.B. Kubernetes support on Digital Ocean is still in beta, so it maybe not available to you.

Overview

The application we are going to deploy is called ‘whoami’ which has an endpoint that outputs some information about the request that is sent to it. We will expose this endpoint through https://whoami.example.com (N.B. example.com is a fake domain name and is used for illustration purposes, replace it with domain you actually own)

In the picture below is the end result is shown:

Overview

The parts that are of interest for external access are the ingress, ingress controller and the load balancer.

An ingress is a Kubernetes object defined in our application namespace that defines the rules to which service (or pod) external requests are routed. In this case we define an ingress with the following rule:

“If an incoming request contains ‘whoami.example.com’ in the host header, the request is routed to the whoami service.”

The actual routing is done by an ingress controller which in most cases uses also a load balancer to route incoming requests. The controller we will use is Traefik, which also handles SSL termination. This means also that the traffic between the controller and the service/pod is flowing through HTTP. Traefik also handles the automatic acquiring and renewal of the Let’s Encrypt certificates

Prequisites:

  • You have a Digital Ocean account and created a project in the control panel.
  • You own the domain example.com and you are able to adminster its DNS records through Digital Ocean.
  • You have kubectl installed on your computer.
  • A Digital Ocean API token with read/write access, which is needed later for the configuration for the acquiring of the certificate.

Creating a cluster in Digital Ocean

Through the Digital Ocean control panel create a cluster with 1 node to keep things simple (and cheap):

  • Choose ‘Kubernetes’ in the left navigation bar.
  • Click ‘Create’ and select ‘Clusters’.
  • Select datacenter AMS3.
  • Under ‘Add node pool’ select from the ‘Standard node pool’ the ‘$5 month plan’ and set the number of nodes to 1.
  • For the name type ‘k8s-test’.
  • Click ‘Create cluster’ and a new cluster will be provisioned.

Creating a cluster in DO

  • When the cluster is provisioned go to the cluster control panel and click ‘Download’.
  • Copy this file to your ~/.kube directory and name the file ‘config’. This makes kubectl point to the just created cluster.
  • We need to create a default service account with full rights (needed for the Helm package manager and for the Traefik package be able to deploy its stuff to the cluster, you probably don’t want to do this in production (at least have a dedicated user). )
  • Paste the following to a file and execute ‘kubectl apply -f filename’ to deploy it to the cluster.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: kube-system

Setup Traefik

To install the Traefik ingress controller:

  • Install Helm, this is a package manager for Kubernetes which makes installation of Traefik very easy.
  • Run ‘helm init’ to initialize Helm on the client and on the cluster.
  • Run ‘helm install –name traefik-thingie -f values.yaml stable/traefik –namespace kube-system’.
  • The values.yaml file contain the configuration for Traefik. A more detailed explanation is provided below.
  • Wait until Traefik is deployed. (it creates also a L4 load balancer in Digital Ocean) -In the DO control panel look up the IP address of the load balancer just created and create an A record in the Domain section with the whoami.mydomain.com domain pointing to this IP address.

Configuration

Below is the configuration of the Traefik helm chart which is used in this example. For more information about all options see here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
ssl:
  enabled: true        # Enables SSL
  enforced: true       # Redirects HTTP to HTTPS
acme:
  enabled: true             # Enables Let's Encrypt certificates
  staging: true             # Use Lets Encrypt staging area for this example. For production purposes set this to false
  email: fill.in@your.email # Email address that Let's Encrypt uses to notify about certificate expiry etc.
  challengeType: "dns-01"   
  dnsProvider:              
    name:  digitalocean     # This is why you need your domain to be under Digital Ocean control
    digitalocean:
      DO_AUTH_TOKEN: "INSERT_API_TOKEN_HERE"
  domains:
    enabled: true
    domainsList:
      - main: "example.mydomain.com" # Name of the domain that belongs to this certificate

Save this file as ‘values.yaml’ and use it during installation of Traefik. (see above)

Application

The application is deployed on the cluster using a simple service and deployment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: whoami-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami-container
        image: containous/whoami
----

apiVersion: v1
kind: Service
metadata:
  name: whoami-service
spec:
  ports:
  - name: http
    targetPort: 80
    port: 80
  selector:
    app: whoami

Paste this to a file and use (again) ‘kubectl apply -f filename.yaml’ to deploy it to the cluster.

Ingress

The important part however is the definition of the ingress.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: whoami-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: whoami.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: whoami-service
          servicePort: http

The interesting bits are here:

  • The annotation kubernetes.io/ingress.class should be set to ‘traefik’, so that the Traefik ingress controller is aware of our ingress (remember, an ingress is just a routing rule which the ingress controller should enforce)
  • The ‘host’ field should be set to our domain, and the ‘backend’ should be set to our service defined above.

Paste this to a file and use (again) ‘kubectl apply -f filename.yaml’ to deploy it to the cluster.

Showtime!

If all of the above went well, go to https://whoami.example.com and you should see a page with the content like below (note the X-Forwarded-Port and X-Forwarded-Proto which indicates that the request was coming through https. )

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Hostname: whoami-deployment-84dfcf599c-5s9ll
IP: 127.0.0.1
IP: 10.244.16.3
GET / HTTP/1.1
Host: whoami.example.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.244.16.1
X-Forwarded-Host: whoami.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-thingie-74ff44595f-lflrm
X-Real-Ip: 10.244.16.1

Comments

Very interesting stuff, I found two errors on the process, the first one is in the helm chart command, it should be –name and not -name and the second one is in the application.yaml file, you have 4 hyphens as the separator but it should be three.

All the best,
Eduardo.

Aug 14, 2019 2019-08-14T23:56:03+02:00 by Eduardo  

Hi Maarten,

Thanks a lot for your tutorial, it was quite close to the usecase I am looking for, both using DigitalOcean k8s and DO as DNS.
I am, however, experiencing issues achieving the results you describe. When I have deployed everything and added an Ingress almost identical to yours, nothing seems to change in Traefik. It doesn't seem to change the DNS in DigitalOcean to the IP of the LoadBalancer or in any way recognize the added Ingresses.

The logs of the traefik pod doesn't contain any warnings or errors.

If you have any suggestions as to what I could try next, I am all ears :)

Aug 20, 2019 2019-08-20T08:56:04+02:00 by Emil  

Hi there MAARTEN, a warm greeting and many compliments on how you explained DO Cluster concepts in this wonderful article. Now, let me ask you if are aware of any limitiation about the total number of ports whoami-service could handle theoretically. I am asking this since DO impose a total number of 40 forwarded ports when you setup a Managed Load Balancer applied during the creation if this type of this service through kubectl tool. Hope you can help me to shed a light on this topic. Regards.

Jan 31 2020-01-31T21:36:07+01:00 by Tezuya  
This post is licensed under CC BY 4.0 by the author.