TL;DR. Kubernetes is the de facto operating system for container workloads in production. It handles scheduling, scaling, self-healing, and networking across any cloud or bare metal. The Kubernetes picture means understanding not just the basics but the operational reality: YAML sprawl, cost leakage, and the growing ecosystem of tools that make it livable.
What Kubernetes Actually Does
Kubernetes is a cluster management system. You describe desired state; Kubernetes reconciles reality toward it continuously. That reconciliation loop is the core mechanic everything else is built on.
Concretely, Kubernetes handles:
- Scheduling — placing containers on nodes with enough CPU/memory
- Self-healing — restarting crashed containers, replacing failed nodes
- Service discovery — giving every pod a DNS name and stable endpoint
- Rolling deployments — updating without downtime, rolling back on failure
- Secret management — injecting env vars and mounted secrets at runtime
- Autoscaling — Horizontal Pod Autoscaler (HPA) and Vertical Pod Autoscaler (VPA) adjust replicas and resource requests based on metrics
What it doesn't handle without add-ons: persistent storage (CSI drivers), ingress routing (Nginx, Traefik, Gateway API), service mesh (Istio, Cilium), and observability (Prometheus, OpenTelemetry). You assemble the platform; Kubernetes is the base.
Core Architecture: The Parts You Must Understand
A cluster has a control plane and worker nodes.
Control plane components:
kube-apiserver— the single source of truth; everything talks to itetcd— distributed key-value store for cluster statekube-scheduler— places pods on nodes based on resource availability and constraintskube-controller-manager— runs reconciliation loops (Deployment controller, ReplicaSet controller, etc.)
Worker node components:
kubelet— the node agent; reads PodSpec, manages containers via containerdkube-proxy— manages iptables/ipvs rules for Service routing- Container runtime — containerd (standard), CRI-O (RHEL-based distros)
Core objects, in order of importance:
| Object | What it does |
|---|---|
| Pod | Smallest deployable unit; one or more containers sharing network/storage |
| Deployment | Manages ReplicaSets; declarative rollouts |
| Service | Stable network endpoint (ClusterIP, NodePort, LoadBalancer) |
| ConfigMap / Secret | Inject configuration without rebuilding images |
| PersistentVolumeClaim | Request block or file storage from a StorageClass |
| Ingress / HTTPRoute | Route external HTTP traffic to Services |
| HorizontalPodAutoscaler | Scale replicas based on CPU, memory, or custom metrics |
Getting Started: Local to Production Path
Step 1 — Run locally
# Docker Desktop ships a single-node cluster
# Or use kind for a real multi-node setup
kind create cluster --config kind-config.yaml
# Verify
kubectl cluster-info
kubectl get nodes
Step 2 — Write a minimal Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
spec:
replicas: 3
selector:
matchLabels:
app: api
template:
metadata:
labels:
app: api
spec:
containers:
- name: api
image: myrepo/api:v1.2.0
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "512Mi"
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
Always set resources.requests — without them, the scheduler has no signal and your pods will land on overloaded nodes.
Step 3 — Move to a managed cluster
- EKS (AWS) — best ecosystem integration, most operational knobs
- GKE (GCP) — autopilot mode removes node management entirely; best default for new teams
- AKS (Azure) — strong if you're already on Azure
- DigitalOcean Kubernetes — cheapest managed option for small workloads
For the Kubernetes path, GKE Autopilot or EKS Fargate let you skip node pool management entirely until you need GPU nodes or custom kernel configs.
Kubernetes vs. Alternatives in 2026
| Option | When it wins | When it loses |
|---|---|---|
| Kubernetes (self-managed) | Full control, on-prem, GPU clusters | Highest ops burden |
| Managed K8s (GKE/EKS/AKS) | Production without infrastructure team | Vendor lock on control plane |
| Cloud Run / App Runner | Single-container stateless apps | No pod-level control |
| Fly.io | Small teams, geographic distribution | Limited ecosystem |
| Nomad | HashiCorp shops, non-container workloads | Smaller community |
| ECS (Fargate) | AWS-only shops, minimal YAML | Not portable |
The honest answer: if you have fewer than five services and a team of one, Cloud Run or Fly.io will ship faster. Kubernetes pays off at scale — when you have dozens of services, need fine-grained resource isolation, or run GPU workloads.
Production Patterns That Actually Matter
Namespace isolation. Don't run everything in default. Separate by team or environment:
kubectl create namespace payments
kubectl create namespace ml-inference
RBAC from day one. Every service account should have least-privilege access. Audit with:
kubectl auth can-i create pods --as system:serviceaccount:payments:api-sa
Pod Disruption Budgets. Prevent cluster upgrades and node drains from taking down your whole service:
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: api-pdb
spec:
minAvailable: 2
selector:
matchLabels:
app: api
Network Policies. By default, all pods can talk to all pods. Lock it down:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-all
namespace: payments
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
GitOps with ArgoCD or Flux. Store all manifests in Git. The cluster reconciles toward the repo. No manual kubectl apply in production — it's untraceable and doesn't survive a cluster rebuild.
AI and ML Workloads on Kubernetes
Kubernetes has become the standard runtime for ML inference and training jobs. The patterns differ from stateless APIs:
GPU node pools. Request GPU resources explicitly:
resources:
limits:
nvidia.com/gpu: 1
Use node selectors or node affinity to target GPU pools. Spot/preemptible nodes cut training costs significantly; use Job checkpointing so a preemption doesn't waste hours.
KubeFlow — the Kubernetes-native ML platform. Pipelines, notebooks, model serving (KServe), hyperparameter tuning (Katib). Heavy, but the ecosystem is mature.
KEDA (Kubernetes Event-Driven Autoscaling) — scales pods based on queue depth, Kafka lag, or HTTP request rate. Critical for inference services with bursty traffic; HPA's CPU-based scaling is too slow for these patterns.
For end-to-end ML platform guidance, the MLOps Complete Guide 2026 covers the pipeline stack that sits on top of these primitives. If you're deploying trained models specifically, see AI Model Deployment 2026 for serving patterns and SLA considerations.
Cost and Operational Reality
Kubernetes clusters have a floor cost: control plane fees (managed clusters charge per cluster/hour), at least two nodes for availability, and load balancers for each external Service. A "cheap" production cluster starts around $150-300/month before your actual workloads.
Common cost leakage patterns:
- Over-provisioned resource requests — pods claim 4 CPUs but use 0.3; nodes fill up on paper, forcing more nodes
- Forgotten namespaces with idle deployments
- LoadBalancer Services for internal services (use ClusterIP + Ingress instead)
- PersistentVolumes that outlive their pods (set
reclaimPolicy: Deletefor ephemeral data)
Tools to control costs:
- Goldilocks — analyzes actual resource usage and recommends request/limit values
- Kubecost — allocates spend per namespace, label, team
- Karpenter (AWS) / GKE Node Auto-provisioner — bin-packing aware node provisioning; replaces cluster-autoscaler with smarter logic
Observability minimum viable stack:
- Prometheus + Alertmanager for metrics
- Loki or CloudWatch/Cloud Logging for logs
- OpenTelemetry for traces (instrument once, route to Jaeger, Tempo, or Datadog)
- Grafana dashboards for unified view
Without observability, you're flying blind. A missing readiness probe causes silent traffic drops; without metrics you'll spend hours in the dark.
YAML Management and Developer Experience
Raw YAML at scale is miserable. Teams adopt:
- Helm — package manager for Kubernetes; parameterized charts. Standard for third-party software (cert-manager, Prometheus, ArgoCD). Avoid writing complex Helm charts for your own services — the templating language is painful.
- Kustomize — overlay-based patching. Built into
kubectl. Good for environment-specific config (staging vs. production replica counts, image tags). - Timoni — CUE-based alternative to Helm; type-safe, composable. Growing adoption.
- Crossplane — provision cloud infrastructure (RDS, S3, GCS) as Kubernetes objects. Bridges K8s and infrastructure-as-code.
The practical path: use Kustomize for your own services, Helm for upstream charts, and ArgoCD to wire it together.
Next Steps
If you're building AI-native applications that run on this infrastructure, the AI Model Deployment 2026 guide covers the serving layer specifically. For the ML pipeline that feeds models into your cluster, start with MLOps Complete Guide 2026. If you're an indie builder evaluating whether Kubernetes is the right substrate for your AI product, How to Start AI SaaS 2026 has the infrastructure decision framework at early scale.
The Kubernetes skill set means you can read cluster state accurately, debug scheduling and networking failures, control costs, and run production-grade workloads without surprises. Get there by running a real cluster — not just tutorials. Break things on a staging cluster before production shows you how wrong your assumptions were.