Everyone agrees backups matter and most people's backups do not work. The difference is almost always that nobody has tried restoring one.
What to back up
- Databases. A dump, not the data directory — copying Postgres files while it is running produces a file that may not restore.
- User-uploaded files. Unrecoverable if lost.
- Configuration. Compose files, Caddyfile,
.env.
Not the operating system. Rebuilding a server from these three takes an hour and a full image backup is large, slow and rarely what you actually need.
The rule that matters
Off the server. A backup on the same machine protects against deleting a table. It does nothing about the provider losing the instance, the disk failing, or an attacker with root deleting everything including your backups.
Object storage from any provider is cents a month for this volume.
restic, which handles the awkward parts
sudo apt install restic -y
export RESTIC_REPOSITORY="s3:https://s3.example.com/my-backups"
export RESTIC_PASSWORD="a-long-passphrase-you-store-elsewhere"
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
restic init
restic encrypts before upload, deduplicates, and does incremental snapshots. Store that passphrase somewhere other than the server. Without it the backups are unrecoverable, which is the point of the encryption and also the trap.
A nightly script
nano ~/backup.sh
#!/usr/bin/env bash
set -euo pipefail
export RESTIC_REPOSITORY="s3:https://s3.example.com/my-backups"
export RESTIC_PASSWORD_FILE="/home/deploy/.restic-pass"
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
STAGING=/tmp/backup-staging
mkdir -p "$STAGING"
# Database dumps
docker compose -f /home/deploy/apps/myapp/compose.yml exec -T db \
pg_dump -U app app | gzip > "$STAGING/myapp.sql.gz"
# Back up dumps, uploads and config together
restic backup "$STAGING" /home/deploy/apps /home/deploy/uploads \
--exclude-caches
# Keep a sensible history
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune
rm -rf "$STAGING"
chmod +x ~/backup.sh
chmod 600 ~/.restic-pass
Schedule it:
crontab -e
30 2 * * * /home/deploy/backup.sh >> /home/deploy/backup.log 2>&1
The restore test — do this today
This is the chapter's actual point. A backup you have never restored is a hope.
# what snapshots exist
restic snapshots
# restore the most recent to a scratch directory
restic restore latest --target /tmp/restore-test
# look at what came back
ls -la /tmp/restore-test
zcat /tmp/restore-test/tmp/backup-staging/myapp.sql.gz | head -40
Then go further: load that dump into a throwaway database and confirm the tables and row counts look right. Ten minutes now, against discovering during an outage that you have been backing up an empty file for six months.
Put a reminder in your calendar to repeat it quarterly. Backup jobs fail silently — credentials expire, paths change, disks fill — and the failure mode is always that nobody noticed.