Whether you like it or not, systemd runs almost every mainstream Linux server, and once you know how to write units it's a very capable process manager. Most of what I deploy (queue workers, small APIs, backup jobs) ends up as a service or a timer, so this is the reference I wish I'd had when I started.
We'll cover the three unit types you'll use most: services, timers and sockets, plus how to override packaged units, read logs and sandbox a service.
Where units live
/usr/lib/systemd/system/(or/lib/systemd/system/): units installed by packages. Don't edit these; updates overwrite them./etc/systemd/system/: your own units and overrides. These take priority.~/.config/systemd/user/: per-user units, managed withsystemctl --user.
After adding or changing a unit file, always reload:
sudo systemctl daemon-reload
Writing a service
Here's a realistic unit for a Laravel queue worker, saved as /etc/systemd/system/myapp-queue.service:
[Unit] Description=My App (Laravel queue worker) After=network-online.target Wants=network-online.target [Service] Type=simple User=www-data WorkingDirectory=/var/www/myapp ExecStart=/usr/bin/php artisan queue:work --sleep=3 --tries=3 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
What the important lines do:
- After= / Wants=: ordering and soft dependency.
Afteronly controls order;Wantsactually pulls the other unit in. You usually want both together. - Type=simple: the process in
ExecStart*is* the service and stays in the foreground. UseType=oneshotfor scripts that run and exit, andType=notifyfor daemons that tell systemd when they're ready. - Restart=on-failure: restart if it crashes or exits non-zero, but not after a clean
systemctl stop.RestartSecstops a crash loop from hammering the CPU. - WantedBy=multi-user.target: what
systemctl enablehooks into, i.e. "start at normal boot".
Start it now and at every boot:
sudo systemctl enable --now myapp-queue.service systemctl status myapp-queue.service
Before deploying a new unit, check it for typos:
systemd-analyze verify /etc/systemd/system/myapp-queue.service
Environment and secrets
Don't hard-code credentials in ExecStart. Use an environment file readable only by root:
[Service] EnvironmentFile=/etc/myapp/env
For anything more sensitive, LoadCredential= passes a file to the service through a private directory ($CREDENTIALS_DIRECTORY) instead of environment variables, which can leak into process listings and crash dumps.
Reading logs with journalctl
Anything a service prints to stdout or stderr ends up in the journal:
journalctl -u myapp-queue -f # follow live journalctl -u myapp-queue --since "1 hour ago" journalctl -u myapp-queue -p err -b # errors since last boot
That alone is a good reason to run long-lived processes under systemd rather than nohup or a screen session.
Timers: a better cron
A timer is a unit that starts another unit on a schedule. It takes two files, but you get logging, dependency handling and catch-up after downtime for free. First the job, /etc/systemd/system/backup.service:
[Unit] Description=Nightly backup [Service] Type=oneshot ExecStart=/usr/local/bin/backup.sh /srv/data /mnt/backup
Then the schedule, /etc/systemd/system/backup.timer:
[Unit] Description=Run backup.service nightly [Timer] OnCalendar=*-*-* 02:30:00 RandomizedDelaySec=15m Persistent=true [Install] WantedBy=timers.target
Enable the timer, not the service:
sudo systemctl enable --now backup.timer systemctl list-timers
- Persistent=true means that if the machine was off at 02:30, the job runs at the next boot. Cron would simply skip it.
- RandomizedDelaySec spreads the start time, which helps when lots of machines hit the same backup server.
OnCalendar syntax takes a little getting used to, and systemd-analyze will tell you exactly what an expression means:
$ systemd-analyze calendar 'Mon..Fri *-*-* 09:00'
Original form: Mon..Fri *-*-* 09:00
Normalized form: Mon..Fri *-*-* 09:00:00
Next elapse: Fri 2026-10-09 09:00:00 AEDT
Shorthands like daily, weekly and hourly work too. For "every 15 minutes after the last run finished", use OnUnitActiveSec=15min instead of a calendar expression.
The backup script itself should fail loudly so the timer's status shows the problem. I cover that in Writing Robust Bash Scripts.
Socket activation
With socket activation, systemd opens the listening socket itself and only starts the service when a connection arrives. That's useful for rarely used services, and it means the port is open from early boot. A minimal example, echo.socket:
[Unit] Description=Echo socket [Socket] ListenStream=127.0.0.1:9999 Accept=yes [Install] WantedBy=sockets.target
With Accept=yes, systemd starts one instance of a template service per connection. The template is named with an @, here echo@.service:
[Unit] Description=Echo service per connection [Service] ExecStart=/usr/bin/cat StandardInput=socket
Enable it with sudo systemctl enable --now echo.socket and test with nc 127.0.0.1 9999. Real daemons (sshd on some distros, CUPS, the Podman API socket) use Accept=no, where one service instance receives the listening socket.
Template units show up elsewhere too: wg-quick@wg0.service in my WireGuard guide is a template where wg0 is the instance name, available inside the unit as %i.
Overriding packaged units
To change a unit installed by a package, use a drop-in rather than copying the whole file:
sudo systemctl edit nginx.service
That opens an editor for /etc/systemd/system/nginx.service.d/override.conf, where you only write the lines you want to change:
[Service] Restart=always LimitNOFILE=65535
To *replace* a list-type setting like ExecStart, clear it first with an empty ExecStart= line. systemctl cat nginx.service shows the final merged result.
Sandboxing a service
systemd can restrict what a service can touch with a few lines. These are a good starting point for most web workers:
[Service] NoNewPrivileges=yes ProtectSystem=strict ProtectHome=yes PrivateTmp=yes ReadWritePaths=/var/www/myapp/storage
ProtectSystem=strict makes the whole filesystem read-only for the service except the paths you list in ReadWritePaths. To see how exposed a unit is and which options would help, run:
systemd-analyze security myapp-queue.service
Add options one at a time and restart in between. Too much sandboxing at once makes it hard to tell which setting broke the app. Container runtimes do similar isolation; if you run your apps in containers, Podman's Quadlet generates systemd units for you (see Podman Rootless Containers).
Conclusion
Most of systemd day-to-day comes down to a handful of commands: write a unit in /etc/systemd/system, daemon-reload, enable --now, read journalctl -u, and use systemctl edit for overrides. Timers alone are worth learning: you get logs, catch-up runs and clear status output that cron never gave you.
If you need a server set up to run your web app or background jobs reliably, that's part of what I do for clients. Have a look at my work.
Comments (0)
No comments yet. Be the first to leave one!
Leave a Comment