One of the key reasons why Kubernetes (K8s) is so popular is due to its single responsibility design. It largely confines itself to one specific job of scheduling and running container workloads. For rest, it relies on container ecosystem vendors and open specifications to fill the gaps. Take for instance ‘CRI’ (Container Runtime interface) – a plugin interface which enables K8s to work with wide variety of container runtimes, including Docker. Another area is Networking. After you schedule the workloads to run on specific nodes, these workloads would invariably need IP address, traffic routing between them and at times even block traffic for security reasons. K8s could have baked in a networking solution into the scheduling engine but it didn’t. Enter CNI (Container Network Interface) – an open specification for container networking plugins that has been adopted by many projects including K8s. There are over dozen vendors out there who have implemented CNI specification and Calico, an open source project from Tigera, is one of them.
Once you install K8s, the first thing typically you would be required to do is to install a CNI plugin. For instance, if you are using kubeadm the official documentation states –
The network must be deployed before any applications. Also, CoreDNS will not start up before a network is installed. kubeadm only supports Container Network Interface (CNI) based networks…
To install Calico you can use yaml file which creates calico-node daemonset (pod that runs on all the nodes of the K8s cluster). Though if you really want to know the Calico installation underpinnings, it might benefit to do it the hard way.
kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
On other side, if you are using a managed K8s platform like Docker EE, Calico is setup for you by default. Regardless of how you install Calico you should also install Calicoctl – the command line option to interact with Calico resources. Three key components that make Calico work are Felix, BIRD, and etcd. Felix is the primary Calico agent that runs on each machine that hosts endpoints, programming routes and ensure only valid traffic is sent across endpoints. BIRD is a BGP client that distributes routes created by Felix across the datacenter. Calico uses etcd as a distributed data store to ensure network consistency. Once configured, Calico provides you a seamless networking experience. Customizations are typically around IPAM (IP Address Management) and Network Security requirements. Let’s take a quick look at them starting with IPAM.
By default Calico uses 192.168.0.0/16 IP pool to assign IP addresses to pods (sudo ./calicoctl get ippools -o wide). Calico allows to create multiple IP pools and request for an IP from a given IP pool as part of pod deployment. To request IP from a given IP pool you would add a K8s annotation to your pod yaml file as shown below. You can also refer to this link for detailed instruction.
annotations: "cni.projectcalico.org/ipv4pools": "[\"custompool-ipv4-ippool\"]"
Network policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints. NetworkPolicy is a standard K8s resource that uses labels to select pods and define rules which specify what traffic is allowed to the selected pods. Calico extends this further allowing for policy ordering/priority, deny rules, and more flexible match rules. Here’s a simple Calico tutorial on how to apply NetworkPolicy.
Couple of things before we wrap up. Most of public cloud providers have a native CNI plugin but using Calico provides you the desired portability (same reasons for using Docker EE platform). Also most cloud provides don’t allow for BGP (excluding the VPN Gateways & VNET peering) so it’s quite possible that you may use cloud native plugins for routing & IPAM and leverage Calico plugin for extended network policy features (also note Calico does supports IPinIP and VXLAN to overcome BGP shortcomings). You can examine the CNI plugin configuration of your cluster by looking at /etc/cni/net.d/10-calico.conflist.
Hope that gave you a fair idea of where CNI & Calico fit into the K8s ecosystem. Please let me know your thoughts in comments section below.