learninfra · Linux · Networking · Kubernetes · System Design · AI Infrastructure · Exam blueprints · Drills

Pod Networking & CoreDNS (CKA)

How a packet gets from one pod to another: CNI, kube-proxy and cluster DNS.

An interactive Kubernetes lesson: 22 steps, about 35 minutes, on a live simulation in your browser.

You have just built this cluster: kubeadm init on cp-1, kubeadm join on two workers. Every node reports NotReady, and the two test pods you create, web-a and web-b, stay Pending.

Nothing is broken. kubeadm installs the control plane, kube-proxy and CoreDNS, and deliberately leaves one thing out: the pod network. Each kubelet reports NetworkPluginNotReady until a network plugin is installed on its node, and the scheduler will not place ordinary pods on a node in that state.

What you will learn

  1. A cluster with no pod network

    • kubeadm is done; nodes are NotReady: kubeadm gives you a cluster without a pod network. Until a CNI plugin is installed, nodes are NotReady and only hostNetwork pods run.
    • Go around the scheduler: CNI is a call the container runtime makes for every pod: set up this pod's network and give it an IP. A plugin on that node has to answer.
    • Install a network plugin: Kubernetes defines what the pod network must do. A CNI plugin, one per cluster, does it on every node.

  2. The pod network model

    • Every pod gets its own IP: One pod CIDR for the cluster, one slice per node, one IP per pod.
    • Same node: a virtual cable: A pod's eth0 is one end of a veth pair into the node. Same-node traffic crosses the node's bridge and never leaves the machine.
    • Across nodes: no NAT: The pod network rule: every pod reaches every other pod by IP, on any node, with no NAT. The address the sender has is the address the receiver sees.

  3. ClusterIP is a rule, not a device

    • A ClusterIP lives nowhere: A ClusterIP is not a device. It is a destination-NAT rule kube-proxy installs on every node: ClusterIP:port becomes one ready pod IP.
    • Break it: no kube-proxy
    • iptables, IPVS, nftables: The proxy mode changes how the rules are stored in the kernel, never what they mean.

  4. Endpoints and Service types

    • Endpoints: the list behind the rules: Selector + readiness → EndpointSlice → kube-proxy rules. A Service forwards to the addresses in that list and nothing else.
    • NodePort: every node answers: Pod to pod: no NAT. In through a NodePort: the entry node rewrites the source, unless externalTrafficPolicy is Local.
    • LoadBalancer: one more wrapper: LoadBalancer = NodePort + an external balancer that some controller provisions. No controller, no external IP; the NodePort still works.

  5. CoreDNS in depth

    • The cluster's DNS server: Cluster DNS is CoreDNS pods behind the kube-dns Service at 10.96.0.10. Every pod's resolv.conf points there.
    • search domains and ndots:5: ndots:5 means: fewer than five dots, walk the search list first. Short Service names work because of it; external names pay three wasted queries for it.
    • The Corefile: The Corefile is a list of plugins. kubernetes answers cluster.local; forward passes everything else upstream.
    • Headless: DNS returns the pods: Normal Service: name → ClusterIP, and kube-proxy picks a pod. Headless Service: name → pod IPs, and the client picks.
    • Break it: a typo in the Corefile

  6. Testing layer by layer

    • Three layers, three tests: CNI carries packets between pod IPs. kube-proxy turns a ClusterIP into a pod IP. CoreDNS turns a name into a ClusterIP. Test in that order.
    • Drill: test DNS from a throwaway pod
    • Drill: find the Corefile

  7. Recap & playground

    • Cheat sheet
    • Playground