Skip to main content
Version: 3.27 (latest)

Install Typha

Typha sits between the Kubernetes API server and per-node daemons like Felix and confd (running in calico/node). It watches the Kubernetes resources and Calico custom resources used by these daemons, and whenever a resource changes it fans out the update to the daemons. This reduces the number of watches the Kubernetes API server needs to serve and improves scalability of the cluster.

Provision Certificates

We will use mutually authenticated TLS to ensure that calico/node and Typha communicate securely. In this section, we generate a certificate authority (CA) and use it to sign a certificate for Typha.

Create the CA certificate and key

openssl req -x509 -newkey rsa:4096 \
-keyout typhaca.key \
-nodes \
-out typhaca.crt \
-subj "/CN=Calico Typha CA" \
-days 365

Store the CA certificate in a ConfigMap that Typha & calico/node will access.

kubectl create configmap -n kube-system calico-typha-ca --from-file=typhaca.crt

Create the Typha key and certificate signing request (CSR)

openssl req -newkey rsa:4096 \
-keyout typha.key \
-nodes \
-out typha.csr \
-subj "/CN=calico-typha"

The certificate presents the Common Name (CN) as calico-typha. calico/node will be configured to verify this name.

Sign the Typha certificate with the CA

openssl x509 -req -in typha.csr \
-CA typhaca.crt \
-CAkey typhaca.key \
-CAcreateserial \
-out typha.crt \
-days 365

Store the Typha key and certificate in a secret that Typha will access

kubectl create secret generic -n kube-system calico-typha-certs --from-file=typha.key --from-file=typha.crt

Provision RBAC

Create a ServiceAccount that will be used to run Typha.

kubectl create serviceaccount -n kube-system calico-typha

Define a cluster role for Typha with permission to watch Calico datastore objects.

kubectl apply -f - <<EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-typha
rules:
- apiGroups: [""]
resources:
- pods
- namespaces
- serviceaccounts
- endpoints
- services
- nodes
verbs:
- watch
- list
- get
- apiGroups: ["networking.k8s.io"]
resources:
- networkpolicies
verbs:
- watch
- list
- apiGroups: ["crd.projectcalico.org"]
resources:
- globalfelixconfigs
- felixconfigurations
- bgppeers
- globalbgpconfigs
- bgpconfigurations
- ippools
- ipamblocks
- globalnetworkpolicies
- globalnetworksets
- networkpolicies
- clusterinformations
- hostendpoints
- blockaffinities
- networksets
verbs:
- get
- list
- watch
- apiGroups: ["crd.projectcalico.org"]
resources:
#- ippools
#- felixconfigurations
- clusterinformations
verbs:
- get
- create
- update
EOF

Bind the cluster role to the calico-typha ServiceAccount.

kubectl create clusterrolebinding calico-typha --clusterrole=calico-typha --serviceaccount=kube-system:calico-typha

Install Deployment

Since Typha is required by calico/node, and calico/node establishes the pod network, we run Typha as a host networked pod to avoid a chicken-and-egg problem. We run 3 replicas of Typha so that even during a rolling update, a single failure does not make Typha unavailable.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: calico-typha
namespace: kube-system
labels:
k8s-app: calico-typha
spec:
replicas: 3
revisionHistoryLimit: 2
selector:
matchLabels:
k8s-app: calico-typha
template:
metadata:
labels:
k8s-app: calico-typha
annotations:
cluster-autoscaler.kubernetes.io/safe-to-evict: 'true'
spec:
hostNetwork: true
tolerations:
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
serviceAccountName: calico-typha
priorityClassName: system-cluster-critical
containers:
- image: calico/typha:v3.8.0
name: calico-typha
ports:
- containerPort: 5473
name: calico-typha
protocol: TCP
env:
# Disable logging to file and syslog since those don't make sense in Kubernetes.
- name: TYPHA_LOGFILEPATH
value: "none"
- name: TYPHA_LOGSEVERITYSYS
value: "none"
# Monitor the Kubernetes API to find the number of running instances and rebalance
# connections.
- name: TYPHA_CONNECTIONREBALANCINGMODE
value: "kubernetes"
- name: TYPHA_DATASTORETYPE
value: "kubernetes"
- name: TYPHA_HEALTHENABLED
value: "true"
# Location of the CA bundle Typha uses to authenticate calico/node; volume mount
- name: TYPHA_CAFILE
value: /calico-typha-ca/typhaca.crt
# Common name on the calico/node certificate
- name: TYPHA_CLIENTCN
value: calico-node
# Location of the server certificate for Typha; volume mount
- name: TYPHA_SERVERCERTFILE
value: /calico-typha-certs/typha.crt
# Location of the server certificate key for Typha; volume mount
- name: TYPHA_SERVERKEYFILE
value: /calico-typha-certs/typha.key
livenessProbe:
httpGet:
path: /liveness
port: 9098
host: localhost
periodSeconds: 30
initialDelaySeconds: 30
readinessProbe:
httpGet:
path: /readiness
port: 9098
host: localhost
periodSeconds: 10
volumeMounts:
- name: calico-typha-ca
mountPath: "/calico-typha-ca"
readOnly: true
- name: calico-typha-certs
mountPath: "/calico-typha-certs"
readOnly: true
volumes:
- name: calico-typha-ca
configMap:
name: calico-typha-ca
- name: calico-typha-certs
secret:
secretName: calico-typha-certs
EOF

We set TYPHA_CLIENTCN to calico-node which is the common name we will use on the certificate calico/node will use in the next lab.

Verify Typha is up an running with three instances

kubectl get pods -l k8s-app=calico-typha -n kube-system

Result:

NAME                            READY   STATUS    RESTARTS   AGE
calico-typha-66498ddfbd-2pzsr 1/1 Running 0 69s
calico-typha-66498ddfbd-lrtzw 1/1 Running 0 50s
calico-typha-66498ddfbd-scckd 1/1 Running 0 62s

Install Service

calico/node uses a Kubernetes Service to get load-balanced access to Typha.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: calico-typha
namespace: kube-system
labels:
k8s-app: calico-typha
spec:
ports:
- port: 5473
protocol: TCP
targetPort: calico-typha
name: calico-typha
selector:
k8s-app: calico-typha
EOF

Validate that Typha is using TLS.

TYPHA_CLUSTERIP=$(kubectl get svc -n kube-system calico-typha -o jsonpath='{.spec.clusterIP}')
curl https://$TYPHA_CLUSTERIP:5473 -v --cacert typhaca.crt

Result

* Rebuilt URL to: https://10.103.120.116:5473/
* Trying 10.103.120.116...
* TCP_NODELAY set
* Connected to 10.103.120.116 (10.103.120.116) port 5473 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: typhaca.crt
CApath: /etc/ssl/certs
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS alert, Server hello (2):
* error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
* stopped the pause stream!
* Closing connection 0
curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate

This demonstrates that Typha is presenting its TLS certificate and rejecting our connection because we do not present a certificate. In the next lab we will deploy calico/node with a certificate Typha will accept.

Next

Install calico/node