Eight tracks

Each track is a question the kernel already answers and almost nobody reads the answer to. Prose on one side, a running kernel on the other, and every claim a program you can run against it.

• Processes

  2 lessons

  What exactly does the child get?

  fork and what is copied, shared and reset; exec and what survives it; wait and who reaps; zombies and orphans; process groups, sessions and the controlling terminal.

  • What exactly does the child get?9 min
  • Who reaps, and what is left if nobody does8 min

• Descriptors

  3 lessons

  Which of the three levels is this bug in?

  The fd table, the open file description and the inode, drawn apart. dup2 and redirection, the offset two processes share after fork, O_APPEND against lseek and write, pipe capacity and the deadlock that follows from not draining one, and leaks as you can see them.

  • Which of the three levels is this bug in?11 min
  • The cursor your child moves9 min
  • The buffer is 64K, and that is the bug10 min

• Signals

  2 lessons

  When is this delivered, and what does it interrupt?

  Dispositions and masks, what cannot be caught, delivery at syscall boundaries, EINTR and the restart that does not happen, SIGCHLD, SIGPIPE, and why kill -9 is not a solution but an admission.

  • Sent is not delivered9 min
  • The restart that does not happen8 min

• PID 1 and lifecycle

  1 lesson

  Why does your container take ten seconds to stop?

  What init owes the system: reaping, forwarding SIGTERM, exec in an entrypoint so the shell gets out of the way, graceful shutdown and the grace period that runs out.

  • What pid 1 owes the machine9 min

• Identity and permission

  2 lessons

  Who does the kernel think you are?

  Real and effective uid and gid, supplementary groups, the mode bits and the one function that checks them, setuid and the confused deputy, dropping privilege in the right order, and capabilities instead of root.

  • The three triples are checked exclusively8 min
  • A program running as root on behalf of somebody who is not11 min

• /proc and observability

  lands in

  What is this process actually doing?

  Reading state rather than guessing it. strace and what it costs, D state and why you cannot kill it, and fd and memory forensics on a process you did not start.

  /forensics

• Namespaces and cgroups

  2 lessons

  What is a container, before any container tooling exists?

  PID, mount, net, user and UTS namespaces as views a process holds, and cgroup v2 as a tree: memory.max and the OOM killer's score, cpu.max and throttling that looks like a slow disk.

  • A namespace is a view, not a boundary10 min
  • Memory kills, CPU throttles, pids refuses9 min

• Reading what was generated

  lands in

  What will the kernel do with this?

  The track the rest exists for. You are handed the entrypoint script, the systemd unit, the subprocess call or the privilege-dropping helper that a model wrote, and asked what the kernel will do with it. Then you run it and read the trace.

  /audit

12 lessons so far. Every claim in one is a program you can run against the kernel beside it.