learninfra · Linux · Networking · Kubernetes · System Design · AI Infrastructure · Exam blueprints · Drills

Cilium: Networking Without kube-proxy

Replace kube-proxy's rule lists with eBPF programs: identity-based policy, Hubble flow logs, encap choices and day-two operations.

An interactive Kubernetes lesson: 25 steps, about 35 minutes, on a live simulation in your browser.

The shop's frontend, web, calls the backend api for every checkout. Two Deployments, two replicas each, one ClusterIP Service in front of api. The stream is green: every request returns 200.

kube-proxy is the per-node agent that makes a Service work. On each node it writes packet-filter rules that translate the Service's virtual IP to a real pod IP. Your app dials one stable address, and the rules do the rest.

What you will learn

  1. kube-proxy at scale

    • The shop at rush hour: A ClusterIP is a promise nodes keep with per-node rule lists: twice the endpoints means twice the rules, and every packet walks them in order.
    • Endpoints churn every minute: With kube-proxy, an endpoint change is a rule rewrite on every node. Churn in pods means churn in rules.

  2. Services without kube-proxy

    • Cilium arrives by Helm: Cilium moves Service translation into the kernel: the same ClusterIP, answered by a hash lookup instead of a rule walk.
    • Who translates the ClusterIP?: A ClusterIP needs a translator on the path. Here that is kube-proxy; on Cilium it is an eBPF program.
    • Replacing kube-proxy in values

  3. Identity, not IP

    • An IP dies with its pod: A pod IP is a lease, not a name: it dies with the pod, and the replacement gets a new one.
    • The ledger pod is replaced: A Service owns a label query, re-run continuously: new pod, same labels, same name, no update needed anywhere.
    • Policy follows labels, not IPs: Cilium identity derives from labels: the agent hands each pod a stable number, and policy matches the number, not the address.
    • L7 rules in Cilium YAML: L3 and L4 ask who may connect; L7 asks what they may ask for, on the same connection.
    • The shop opens a second region: ClusterMesh shares services, not clusters: peer once, and the same name and the same labels work on both sides.

  4. Hubble sees every drop

    • Hubble sees every drop: A drop has a reason; read the reason before touching the rule.
    • Read the drop
    • Drill: only the drops
    • Break it: deny all on api

  5. Overlay or native

    • Overlay or native routing: Overlay works everywhere and taxes every packet; native costs nothing per packet and needs a fabric that knows pod routes.
    • Do the MTU math: Pod MTU is underlay - overhead: with VXLAN on a 1500 network, pods get 1450.

  6. Encrypt the wire

    • Transparent encryption: Transparent encryption seals node-to-node traffic at the agent layer. Apps and policies behave exactly as before.
    • What Hubble sees now: Encryption is opaque to observers, including yours: Hubble keeps who and verdict, and loses the inside.

  7. Tetragon and day-two ops

    • Tetragon enforces, Falco alerts: Falco reports the behaviour; Tetragon stops it: same syscalls observed, different response.
    • Break it: the agent goes down
    • Upgrade order, MTU blackhole: Upgrade the brain before the hands: operator first, then agents one node at a time.
    • One fixed address out: Egress policy gives outside traffic one stable source address. Pick it for allowlists, audits and billing.
    • Drill: is Cilium healthy?

  8. Recap & playground

    • Cheat sheet
    • Playground