systemd & Services
A server is a set of programs that must start in order, stay up and leave a log. systemd is the program that does all three.
An interactive Linux lesson: 24 steps, about 32 minutes, on a live simulation in your browser.
web01 is going to serve an online shop. It booted early this morning and you have only just logged in, yet an SSH server was already waiting for you and a scheduler is already running jobs. Nobody typed those commands. Something started them.
Ask for process number 1, then for the whole tree. The kernel starts exactly one program at boot, and every other process descends from it. On Ubuntu, /sbin/init is systemd, which is the name pstree prints at the root.
What you will learn
What pid 1 is for
- Who started all this?: pid 1 is systemd. It starts every service, watches each one for as long as it lives, and keeps its output.
- Units: the things systemd manages
- A unit is a text file: /usr/lib/systemd/system belongs to packages. /etc/systemd/system belongs to you, and wins.
Reading a service
- A web server arrives with its unit
- systemctl status, line by line: status reads top to bottom: which file, enabled at boot or not, running right now or not, which processes, and what it last said.
- stop and start
- reload is not restart: reload: the same process re-reads its config. restart: the process is replaced, and every open connection goes with it.
Now versus next boot
- Enabled is not running: start and stop are about now. enable and disable are about the next boot. Neither pair touches the other.
- enable --now does both
- Drill: off now, off after reboot
A unit of your own
- Write a unit for your own program: [Unit] says what it is and what it comes after. [Service] says what to run, as whom, and when to restart it. [Install] says which target enable hooks it into.
- The file exists. Does the unit?: systemd runs from its in-memory copy of the unit files. After you write or edit one: daemon-reload, then start or restart.
- Check what you built
- Kill it and see who notices: systemd is the parent of every service, so it knows when one dies. Restart= decides whether a death is final.
When a service will not start
- Break it: a restart that never comes back
- The journal has the reason: status tells you that a unit failed. journalctl -u tells you why, in the program's own words. Read up from the last line to the first one the program wrote.
- Break it: someone else has the port
- Find who holds the port: A failed start is a symptom with many causes. The program's own line in the journal is the diagnosis: a bad config, a missing file, a port that is taken.
- Drill: one failed unit
Dependencies, logs and targets
- Requires: start this one first: Requires= says what must be started with this unit. After= says in which order. Dependencies live in unit files, so nobody has to remember them.
- Ask the journal better questions: journalctl is one log for the whole machine plus filters: -u which unit, -p how bad, --since when, -b this boot, -f keep watching.
- Targets: what "booted" means: A target is a named group of units. Booting is reaching the default target, and enable is adding a unit to one.
Recap & playground
- Cheat sheet
- Playground