Kubernetes
See Terminology for a reference
Kubernetes (K8s) is an open-source orchestration system for automating deployment, scaling, and management of containerized applications.
[TOC]
Architecture
(Image taken from https://x-team.com/blog/introduction-kubernetes-architecture/)
Everything must be in a namespace, a default namespace is created when you create a cluster.
Default objects
Master Node (one is always present in a cluster)
API Server - Allows interaction with kubernetes API, it's the frontend for the control plane.
kube-scheduler - Assigns pods to nodes at runtime, and checks resources, quality of service, policies, and specifications before scheduling
Controller Manager - Runs controllers (background threads that run tasks in a cluster). It's compiled into a single binary and has a number of roles:
Node Controller - Responsible for worker states.
Replication Controller - Responsible for maintining correct number of pods for replicated controllers.
Endpoint Controller - Responsible for joining services and pods togethers
Service Account Contoller and Token Controller - Responisble for handling access management to API Server.
etcd
Database used by Kubernetes to store all cluster data (job scheduling info, pod details, stage information, etc.).
kubectl
Commandline interface used to interact with the master node. It can be prunounced as Kube Cuttle. Kubectl has a config file which has server information and authentication information to access the API server.
Nodes
A Node requires the following to be running:
Supervisord - Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems
Container tooling (e.g. Docker)
kubelet
The main service on a node, regularly taking in new or modified pod specifications (primarily through the kube-apiserver) and ensuring that pods and their containers are healthy and running in the desired state. This component also reports to the master on the health of the host where it is running. It executed pod containers via the container engine and mounts pod volume and secrets. Kubelet must be running on all nodes, and if it isn't working correctly, you're going to have issues.
It takes a set of Podspecs (Yaml files provided by API Server) that describe a pod.
Kubelet only manages containers that were created by the API server, not any container running on the node.
kube-proxy
A proxy service that runs on each worker node to deal with individual host subnetting and expose services to the external world. It performs request forwarding to the correct pods/containers across the various isolated networks in a cluster. kube-proxy must be running in each node.
It can do simple network stream or round-robin forwarding across a set of backends.
Services definged against the API server: kube-proxy watches the API server for the addition and removal of services.
For each new service, kube-proxy opens a randomly chosen port on the local node.
Connections made to the chosen port are proxied to one of the corresponding back-end pods.
Modes of kube-proxy:
User space mode (most common).
Iptables mode.
Ipvs mode.
Pods
A pod is one or more containers that should be controlled as a single application. It encapsulates application containers, storage resources, a unique network ID and other configuration on how to run the containers.
Pods are ephemeral/disposable, they never self-heal and are not restarted by the scheduler, never create them by themselves in production. Instead use higher level constructs like controllers.
States
Pending - Accepted by Kubernetes system but a container hasn't been created yet.
Running - A pod has been scheduled on a node, all containers have been created, and at least one container is in the running state.
Succeeded - All containers in the pod have exited with a status of 0 (successful execution and will not be restarted)
Failed - All containers have exited and at least one has failed (returned non-zero exit status)
CrashLoopBackOff - A container fails to start and Kubernetes keeps trying to restart it
Controllers
In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state. They help with application reliability, scaling, and load balancing.
Controller Types
ReplicaSet
Ensures that the specified number of replicas for a pod are running at all times.
Deployment
Most applications are packaged as Deployments. Deployments provide declarative updates for pods and ReplicaSets. Used to describe desired state in a YAML file and the Deployment controller will align the actual state to match. Can be used to create new ReplicaSets or replace them with new ones. A Deployment manages the ReplicaSet, and the ReplicaSet manages the Pod. This architecture allows deployments to support a rollback mechanism.
A new ReplicaSet is created each time a new Deployment config is deployed but the old ReplicaSet is still kept to allow for easier rollbacks.
Use-cases
Pod management - Running a ReplicaSet allows us to deploy a number of pods and check their status as a single unit.
Scaling a ReplicaSet scales out the pods and allows for a deployment to handle more traffic.
Pause and Resume (traffic will still get passed to existing replica set)
Used with larger changesets.
Pause, make changes, and resume deployment.
DaemonSet
Ensures all nodes run a copy of a specific pod. As nodes are added or removed from the cluster, a DaemonSet will add or remove the required pods.
Use-cases
Run a single log aggregator.
Monitoring agent.
Job
Supervisor process for pods carrying out batch jobs. Used to run individual processes that need to run once and complete successfully.
Use-cases
Cron job to run nightly report or database backup.
Service
Allows the comunication between one set of deployments with another. When a service is created it is assigned a unique IP address that never changes throughout the lifetime of the service. Pods are then configured to talk to the service and can rely on the service IP on any requests that might be sent to the pod. Services are an important concept because they allow one set of pods to communicate with another set of pods in an easy way.
Example
Frontend deployment needs to talk to backend deployment (which consists of multiple pods), a backend service can provide a single IP for the frontend pod to talk with.
Kinds of Services
Internal - IP is only reachable within the cluster
External - Endpoint available through node IP (known as NodePort)
Load balancer - Exposes application to the internet with a load balancer (available with a cloud provider)
Labels and Selectors
Covered in the terminology page.
Namespaces
Allows teams to access resources, with accountability.
Great way to divide cluster resources between users.
Names for resources must be unique within a namespace.
A default namespace is created when kubernetes is launched.
Newer applications install their resources in a different namespace so they don't interfere with an existing cluster.
Maintenance
Features are backward-compatible and APIs are versioned.
Host can be turned off/on during maintenance.
Logging and Monitoring
Application Monitoring
Built-in TCP/HTTP/container-execution health checks.
Node health check
Failures are monitored by node controller.
Kubernetes status
Can be monitored through addons like metrics-server or the Prometheus Operator.
Limitations
As of 1.16, see official Limitations page.
No more than 5000 nodes
No more than 150000 total pods
No more than 300000 total containers
No more than 100 pods per node
References
Last updated
