Kubernetes Networking and Security ( Part - 1)

If you have worked on Public Cloud or for that matter private cloud, the services which has been hosted on depends on the building blocks such as compute, storage, network, security. Hence it is crucial to understand networking for any platform solutions which defines how the services are interconnected or how can it would be accessed internally or externally.

When it comes to Kubernetes, its really important to know various aspects of networking from POD to Cluster.

Kubernetes was created to run distributed systems over a cluster of machines. Distributed systems make networking a central and necessary component of implementation. Understanding the Kubernetes networking model allows you to run, monitor, and troubleshoot applications running on Kubernetes in an optimal way

Understanding networking is crucial to understanding how Kubernetes works

Static Port allocation - In case of a VM hosted with multiple application, different port would be consumed by the application with same IP/DNS name. In Kubernetes, network model optimizes compute resources by sharing machines between applications. This, however, requires that no two applications are using the same networking port. Setting ports manually and hoping that the two ports never cross is not the solution to this problem – you’ll end up playing whack-a-mole all day, and you still won’t be able to get your cluster fully up and running. Another option would be to use dynamic port allocation, although this would introduce its own set of complications. 

Dynamic Port Allocation - Dynamic port allocation would allow every application to take ports as flags, but the API servers would have to know how to insert dynamic port numbers into configuration blocks. Rather than having to deal with ensuring that services know which ports to look for to talk to each other, Kubernetes takes a different approach to networking. 

The following are the 4 main modes of communication within a Kubernetes cluster:

1. Container-to-Container 

2. Pod-to-Pod 

3. Pod-to-Service 

4. Internet-to-Service or External -to-Service

In Kubernetes, network architecture provides the foundation for understanding how containers, pods and services within Kubernetes communicate with each other.

The Kubernetes network architecture simplifies below requirement:

• Every pod gets its own IP Address and has its own network namespace This means you do not need to explicitly create links between Pods and you almost never need to deal with mapping container ports to host ports. It’s important to note that the pods and IP addresses are ephemeral, meaning that if a pod goes down or has to restart for any reason, it will be assigned a new IP address

• Containers within a pod share the pod IP address and can communicate freely with each other

• Pods can communicate with all other pods in the cluster using pod IP addresses (without NAT)

Now each node has its own way of communicating with the other nodes on the cluster. From this point, the communication between pods is similar to the way in which pods on the same node communicate with one another.

• Pod-To-Service Networking

• Internet-to-Service or External -to-Service ( NAT)

• Egress routes traffic from the node to an external connection. 

• Ingress enables external clients to communicate with Kubernetes Services.

• Isolation (restricting what each pod can communicate with) is defined using network policies

• Agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node

Kubernetes clusters require to allocate non-overlapping IP addresses for Pods, Services and Nodes, from a range of available addresses configured in the following components:

• The network plugin is configured to assign IP addresses to Pods.

• The kube-apiserver is configured to assign IP addresses to Services.

• The kubelet or the cloud-controller-manager is configured to assign IP addresses to Nodes.

Pods can be treated much like VMs or hosts (they all have unique IP addresses), and the containers within pods very much like processes running within a VM or host (they run in the same network namespace and share an IP address).

Kubernetes built in network support, kubenet, can provide some basic network connectivity. However, it is more common to use third party network implementations which plug into Kubernetes using the CNI (Container Network Interface) API. The network model is implemented by the container runtime on each node. The most common container runtimes use Container Network Interface (CNI) plugins to manage their network and security capabilities.

There are lots of different kinds of CNI plugins, but the two main ones are:

• Network plugins, which are responsible for connecting pod to the network

• IPAM (IP Address Management) plugins, which are responsible for allocating pod IP addresses.
  

Note:- A server or a pod can have multiple IP addresses on its interfaces, but only the IP addresses in node.status.addresses or pod.status.ips are considered for implementing the Kubernetes network model and defining the type of the cluster

Kubernetes Services

In Kubernetes, a Service is a method for exposing a network application that is running as one or more Pods in your cluster..

Within the cluster the network service is usually represented as virtual IP address, and kube-proxy load balances connections to the virtual IP across the group of pods backing the service. The virtual IP is discoverable through Kubernetes DNS. The DNS name and virtual IP address remain constant for the life time of the service, even though the pods backing the service may be created or destroyed, and the number of pods backing the service may change over time

Although each Pod has a unique IP address, those IPs are not exposed outside the cluster without a Service. Services allow your applications to receive traffic. Services can be exposed in different ways by specifying a type in the spec of the Service:

• ClusterIP (default) - Exposes the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster. This is the default that is used if you don't explicitly specify a type for a Service. You can expose the Service to the public internet using an Ingress or a Gateway

• NodePort - Exposes the Service on the same port of each selected Node in the cluster using NAT. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>. Superset of ClusterIP. Exposes the Service on each Node's IP at a static port (the NodePort). To make the node port available, Kubernetes sets up a cluster IP address, the same as if you had requested a Service of type: ClusterIP.

• LoadBalancer - Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP to the Service. Superset of NodePort. Kubernetes does not directly offer a load balancing component; you must provide one, or you can integrate your Kubernetes cluster with a cloud provider.

• ExternalName - Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up. This type requires v1.7 or higher of kube-dns, or CoreDNS version 0.0.8 or higher.

Kubernetes DNS

Each Kubernetes cluster provides a DNS service. Every pod and every service is discoverable through the Kubernetes DNS service.

For example:

• Service: my-svc.my-namespace.svc.cluster-domain.example

• Pod: pod-ip-address.my-namespace.pod.cluster-domain.example

• Pod created by a deployment exposed as a service: pod-ip-address.deployment-name.my-namespace.svc. cluster-domain.example .

The DNS service is implemented as Kubernetes Service that maps to one or more DNS server pods (usually CoreDNS), that are scheduled just like any other pod. Pods in the cluster are configured to use the DNS service, with a DNS search list that includes the pod’s own namespace and the cluster’s default domain.

This means that if there is a service named foo in Kubernetes namespace bar , then pods in the same namespace can access the service as foo , and pods in other namespaces can access the service as foo.bar