Homelab Backups Explained: 3-2-1, Restic, Borg and Restore Tests

A homelab backup plan that survives disk failure, ransomware and your own mistakes keeps three copies of your data on two kinds of media with one copy offsite, the 3-2-1 rule a 2012 US-CERT paper set out. Use restic or Borg for encrypted, deduplicated snapshots, push one copy to a remote that cannot delete history, and prove it all with scheduled restore tests.

This guide goes one level deeper than our post on the home NAS, ZFS, RAID and snapshots, which covers the storage layer. Here we cover the backup layer on top of it. Every flag, default and number below comes from the restic and BorgBackup documentation (restic 0.19.1 and Borg 1.4.5 were the latest GitHub releases when we checked), the rclone, systemd, Docker and S3 Object Lock docs, and government guidance from CISA, NIST and the UK NCSC. Commercial backup products appear only as categories.

Key takeaways

• The US-CERT rule: three copies of any important file (one primary, two backups), on two media types, one offsite. It is a floor, not a full design.
• RAID keeps a pool online when a disk dies; snapshots let you undo changes. Both live on the same machine, so neither survives theft, fire or an attacker with root.
• Restic and Borg split files with content defined chunking, store each unique chunk once and encrypt on the client. Restic aims for 1 MiB chunks; Borg's default targets about 2 MiB.
• Removing old snapshots takes two steps: forget then prune in restic, prune then compact in Borg. Space is freed only by the second.
• Ransomware resistance comes from what the client cannot change: an append-only remote, Object Lock storage, and one disk unplugged in a drawer. Run pruning from a separate machine.
• Monitor exit codes, repository checks and the age of the newest snapshot, and restore something on a schedule. Keep the repository key somewhere other than the server it protects.

What is the 3-2-1 backup rule, and where does it come from?

What the US-CERT paper actually says

The most cited government source for 3-2-1 is "Data Backup Options," a short paper by Paul Ruggiero and Matthew A. Heckathorn, produced by Carnegie Mellon University for US-CERT in 2012 and still hosted on cisa.gov. It opens by telling everyone from home users to security officers to back up the data on desktops, laptops, servers and mobile devices, and says one backup file may not be enough. Then it states the rule:

The paper's further reading cites the American Society of Media Photographers' backup overview and Peter Krogh's book on digital asset management for photographers, which fits the usual account that 3-2-1 came out of photography workflows before government guidance spread it. We confirmed the paper's citations, but the photographers' page did not respond when we tried it, so treat the origin story as the paper's attribution.

The paper also compares targets (external drives, optical media, flash, tape, remote servers, cloud) and its security advice still holds: for sensitive data, weigh the risks of cloud storage, encrypt, and keep media physically secure.

What 3-2-1-1-0 and similar variants add

You will see longer versions of the rule in vendor material and forum posts. The most common is 3-2-1-1-0:

• The extra 1 is one copy that is offline, air gapped or immutable, meaning nothing on the network can change or delete it before a set date.
• The 0 is zero errors when you verify or test restore the backups.

Neither is in the 2012 paper; they are industry extensions. The ideas are in current government guidance, though. CISA's #StopRansomware Guide (September 2023 edition) says to "maintain offline, encrypted backups of critical data" and to test their availability and integrity regularly, because many ransomware variants find reachable backups and delete or encrypt them. The NCSC's small organisations guide (April 2026) says a backup device should not stay connected when not in use, and that you should know how to restore and check the backup holds your important data.

So a modern 3-2-1 for a homelab is three copies, two media, one offsite, at least one that nothing on your network can alter, and a restore that proves it works.

Where 3-2-1 sits in formal contingency planning

NIST SP 800-34 Rev. 1, the Contingency Planning Guide for Federal Information Systems (May 2010, updated November 2010), puts backups inside a contingency plan with recovery priorities and regular exercises. NIST IR 8374 (February 2022), the Ransomware Risk Management profile, maps ransomware defense onto the Cybersecurity Framework's identify, protect, detect, respond and recover functions. Two ideas carry over to a homelab:

• Recovery objectives. Decide how much data you can afford to lose (recovery point objective, which sets how often you back up) and how long you can be down (recovery time objective, which decides whether a slow remote is enough or you need a local copy).
• Testing is part of the plan. A backup you have never restored is untested.

Why are RAID and snapshots not backups?

RAID, ZFS mirrors and RAIDZ protect availability: the array keeps serving data when a disk fails. Snapshots protect against change: you can roll back a file or a dataset to an earlier point. Both live inside the same machine, on the same pool, under the same root account. That leaves a long list of failures they do not cover.

Only copies that live somewhere else, under credentials the source cannot use to delete them, cover the bad days. Keep RAID for uptime, snapshots for quick undo, and a real backup for the rest.

Where snapshots still help a backup

Snapshots make a good backup source. A ZFS or Btrfs snapshot is a frozen view of a dataset, so backing up from it never captures a file half written. On ZFS, snapshots appear under the dataset's hidden .zfs/snapshot directory, so you can point restic or Borg there. For databases, a filesystem snapshot is only crash consistent; a database dump is safer, as the walkthrough shows.

How do deduplicating backup tools work?

Restic and Borg are the two open source deduplicating backup tools most homelabs choose. Both are command line programs. Restic is written in Go and ships as a single binary; Borg is written in Python with C extensions. Both use the same broad design: split data into chunks, identify each chunk by a cryptographic hash, store each unique chunk once in a repository, and encrypt and authenticate everything on the client.

Content defined chunking, step by step

If you split a file into fixed 1 MiB blocks, inserting a single byte near the start shifts every block boundary after it, and every block looks new. Content defined chunking avoids that. The tool slides a small window over the data, computes a rolling hash at every byte position, and cuts a chunk wherever the hash matches a pattern. Because the cut points depend on the content, an insertion only changes the chunks around it. Everything after the next cut point lines up again.

The restic design document describes its version precisely:

• Files are split into variable length blobs at offsets chosen by a 64 byte sliding window using Rabin fingerprints.
• An irreducible polynomial is chosen at random when the repository is created and saved in the repository config, "so that watermark attacks are much harder."
• Files smaller than 512 KiB are not split. Blobs are 512 KiB to 8 MiB, aiming for 1 MiB on average.
• For modified files, only modified blobs are saved again, even when bytes are inserted or removed at arbitrary positions.

Borg's internals document describes its default buzhash chunker:

• It cuts when the last HASH_MASK_BITS bits of the rolling buzhash are zero, which gives a target chunk size of 2 to the power of HASH_MASK_BITS bytes.
• Defaults are CHUNK_MIN_EXP = 19 (512 KiB minimum), CHUNK_MAX_EXP = 23 (8 MiB maximum) and HASH_MASK_BITS = 21 (about 2 MiB target).
• The buzhash table is XORed with a random seed generated once per repository. Borg uses buzhash only to choose cut points; deduplication itself uses the chunk ID, a keyed hash of the content.
• Borg also offers a fixed chunker for block devices and VM images where content does not shift.

Worked example 1: what changes in a 10 GiB VM disk

Say you back up a 10 GiB virtual machine disk image every night. Over a day the guest writes about 50 MiB, spread across 100 separate places in the file.

• Plain file copy: the file changed, so the whole 10 GiB is copied again. Thirty nights is roughly 300 GiB.
• Restic, about 1 MiB average blobs: the image is roughly 10,240 blobs. In the worst case each of the 100 scattered writes lands in a different blob, so about 100 blobs, around 100 MiB, are new. Thirty nights is roughly 3 GiB of new data plus metadata, before compression.
• Borg, about 2 MiB target chunks: the image is roughly 5,120 chunks. The same 100 writes touch up to about 100 chunks, around 200 MiB a night, roughly 6 GiB over thirty nights. Writes that land close together share chunks, so real numbers are usually lower.

The exact figures depend on where writes land and how well the data compresses, but the order of magnitude is the point. Deduplication turns nightly full backups into something close to incremental cost, while every snapshot still restores as a complete file system tree.

Repositories: what is actually on disk

A restic repository is a directory, object storage prefix or REST server path holding a config file plus keys, locks, snapshots, index and data directories. Data lives in pack files, each combining blobs with an encrypted header at the end. Index files map blobs to packs and stay under 8 MiB. A stored file's name is the SHA-256 hash of its content, which makes verification simple. Repository format version 2 adds zstandard compression.

A Borg repository is a transactional log. Objects are stored in segment files of about 500 MB (the default max_segment_size is 524288000 bytes) in numbered subdirectories. Segments are append-only at the file level: a delete is recorded by appending a DELETE entry, not by rewriting the segment. Since Borg 1.2.0, compaction (rewriting sparse segments to reclaim space) is a separate command, borg compact. Borg commands either commit completely or not at all, so an interrupted backup leaves the repository at its last good state.

Restic takes a lock file in the repository while it works and treats locks older than 30 minutes as stale, so backups and maintenance from different machines take turns rather than collide.

Encryption at rest

Both tools encrypt on the client, before data leaves the machine. The storage target, whether it is a USB disk, a friend's server or a cloud bucket, only ever sees ciphertext.

Restic encrypts all repository data with AES-256 in counter mode and authenticates it with Poly1305-AES. Your password goes through scrypt, with parameters and salt from the key file, to derive 64 bytes: 32 for AES-256 and 32 for the MAC. Those keys unlock a master key, so you can add passwords with restic key add without re-encrypting data. Encryption is always on; an empty password needs the explicit --insecure-no-password flag (since 0.17.0).

Borg asks for a mode at borg init --encryption, and its rule of thumb is "use repokey with a strong passphrase." In repokey modes the encrypted key sits in the repository config; in keyfile modes it sits in your home directory on the client. Data is encrypted with AES-256-CTR and authenticated with HMAC-SHA-256 or BLAKE2b-256 (the docs suggest BLAKE2b on 64-bit CPUs without SHA extensions). authenticated modes give integrity without secrecy, and the docs say to avoid none. Back up the key with borg key export: losing the client in keyfile mode, or the config in repokey mode, loses the key.

Pruning: how old snapshots go away

Both tools let you describe a retention policy and then delete what falls outside it.

Restic separates the two jobs. restic forget removes snapshot records according to --keep-* options. restic prune then removes data that only those snapshots referenced. restic forget --prune runs both in one go. The keep options are --keep-last, --keep-hourly, --keep-daily, --keep-weekly, --keep-monthly, --keep-yearly, --keep-tag, --keep-within and the --keep-within-hourly through --keep-within-yearly family. By default restic groups snapshots by host and paths before applying the policy, which you can change with --group-by.

Borg uses borg prune with similar options, including --keep-within, --keep-last, --keep-hourly, --keep-daily, --keep-weekly, --keep-monthly, --keep-13weekly, --keep-3monthly and --keep-yearly, and --glob-archives to limit pruning to archives with a given name pattern. The prune docs are blunt: repository disk space is not freed until you run borg compact. Both tools offer a dry run (--dry-run in restic, -n or --dry-run in Borg). Use it every time you change a policy.

Worked example 2: how many snapshots a policy keeps

Take a nightly backup and the policy --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 3, after the backup has run every night for three years.

• Daily: the newest snapshot from each of the last 7 days. That is 7 snapshots.
• Weekly: the newest snapshot from each of the last 4 weeks. The newest of those is tonight's, already kept as a daily, and the one for last week may also fall inside the 7 days. Count about 2 or 3 new ones.
• Monthly: the newest from each of the last 12 months. This month's is already kept, so about 11 new ones.
• Yearly: the newest from each of the last 3 years. This year's is already kept, so about 2 new ones.

The upper bound is 7 + 4 + 12 + 3 = 26 snapshots. In practice you keep about 22 or 23 because the newest snapshot satisfies several rules at once. Restic's docs say these calendar rules count periods that actually have snapshots, so if the server was off for a week, --keep-daily 7 reaches back further than seven calendar days rather than keeping fewer snapshots.

Storage follows unique data, not snapshot count. If 400 GiB of photos and documents gain about 1 GiB of new data a week, the repository after a year is roughly 452 GiB before compression, whether it holds 10 snapshots or 30. Retention decides how far back you can go and how long deleted data lingers.

Restic and Borg side by side

Neither tool is a wrong choice. Restic suits object storage and many backends; Borg suits an SSH reachable box you control. If both your copies use one tool, consider a third that does not depend on its repository format, such as a ZFS replica or a periodic plain export. That is the "two media" idea applied to software.

How do you get an offsite copy?

The offsite copy is the one that saves you from fire, flood and theft. A homelab has three practical options, and many people combine two.

Option 1: a second machine at another site

A small computer with a big disk at a relative's or friend's house, reached over a VPN, is the classic homelab offsite copy. Restic writes to it over SFTP or to a rest-server; Borg writes over SSH to borg serve. The design point is credentials: the backup client can add snapshots and nothing else, the remote holds no credentials for your network, and a separate admin machine runs pruning.

The rest-server README describes the pieces: --append-only allows new backups but prevents deletion and modification of existing ones; --private-repos limits each user to a repository path matching their username; authentication uses an .htpasswd file with bcrypt (htpasswd -B); and --tls turns on HTTPS, with --tls-min-ver 1.3 available. The README warns that without TLS, Basic authentication sends usernames and passwords in clear text on every request. Current versions refuse to start if the .htpasswd file cannot be opened unless you pass --no-auth; versions up to 0.9.7 silently disabled authentication instead, so do not run an old build.

For Borg, the borg serve docs show how to restrict an SSH key with a forced command in authorized_keys, for example command="borg serve --restrict-to-path /path/to/repo",restrict before the key. --restrict-to-repository is stricter and allows only the named repository with no subdirectories.

Option 2: object storage, ideally with Object Lock

Object storage means no second box to maintain. Restic writes to S3 compatible storage and many other services natively, and to anything rclone supports. Borg 1.4 does not write to object storage directly.

The stronger option is a bucket with Object Lock. Amazon's S3 documentation describes Object Lock as a write once, read many (WORM) model. It works only on buckets with S3 Versioning enabled and protects specific object versions. There are two retention modes:

• Governance mode: users cannot overwrite or delete a locked object version or change its lock settings unless they have special permissions, so a privileged role can still remove it.
• Compliance mode: a locked object version cannot be overwritten or deleted by any user, including the root user of the account, and its retention period cannot be shortened.

There are also legal holds, which protect an object version with no expiry until someone removes the hold, and a default retention period you can set on the bucket. Other S3 compatible providers implement Object Lock to varying degrees; check your provider's own documentation for exactly which API calls and modes it supports.

Two cautions. CISA's guide says to "use immutable storage with caution" because misconfiguration can impose significant cost. And restic's prune deletes and rewrites pack files, which locked object versions refuse until retention expires. A common pattern is a retention period a little longer than your prune interval plus a lifecycle rule for noncurrent versions. Test it on a throwaway bucket first.

Option 3: rotated disks

Two or three external disks, rotated so one is always in another building, are cheap and fully offline while away. The NCSC's advice applies: a backup device should not stay connected when not in use. The weak point is human, so put the swap on a calendar and track each disk's newest snapshot.

Encrypt it before it leaves

Restic and Borg encrypt on the client, so a repository copied anywhere is already ciphertext. If you also sync plain files offsite with a sync tool, encrypt those too. rclone's crypt remote wraps another remote and encrypts file contents using XSalsa20 and Poly1305, with optional file name encryption (standard, obfuscate or off). Keep the crypt password and salt in your password manager; rclone cannot recover them.

The trade-off between sync and backup is covered in the comparison table later, but in short: a sync tool copies your mistakes and your ransomware along with your files, unless the destination keeps versions that the source cannot delete.

How do you protect backups from ransomware and from your own mistakes?

Pull, push and who can delete

CISA's guide is clear about the threat: ransomware tries to find and delete or encrypt any backup it can reach. In a homelab, "reach" usually means one of three things: a mounted share, an SSH key or password on the source machine, or cloud credentials on the source machine. The defense is to make sure the credentials stored on the source can only add data.

There are two clean ways to do that:

• Push to an append-only endpoint. The source pushes backups, but the server refuses deletes and overwrites from that client. Restic rest-server --append-only, rclone serve restic --append-only (rclone documents the flag as "disallow deletion of repository data") and borg serve --append-only all work this way.
• Pull from a separate machine. The backup server logs into the source read-only and pulls data. The source holds no backup credentials at all. This works well with ZFS send and receive, as covered in our NAS post.

What append-only actually guarantees

Read the fine print, because each tool means something slightly different.

For restic, forget and prune need full read, write and delete access, so the docs recommend a separate, well-secured client for maintenance. They also describe a subtle attack: a client that can only add snapshots can add empty ones timestamped just after your real ones, so a count based policy such as "keep three weekly" keeps the junk and forgets the real snapshots. The docs' fix is --keep-within on append-only repositories, which keeps everything inside a time window so you can spot the junk. Its limit: forget --keep-within 7d run eight days after the last good snapshot no longer protects you, so alert on snapshot age.

For Borg, append-only applies to the low level segment files. The notes say Borg will never overwrite or delete committed data in that mode, and borg compact silently does nothing. But borg delete and borg prune are still allowed at the archive level; they just do not free space. In append-only mode Borg writes a transactions log with a UTC timestamp per line, and the docs describe how to roll the repository back to an earlier transaction if a compromised client marked archives deleted. You can set the mode on the repository with borg config /path/to/repo append_only 1, or per SSH key with a forced borg serve --append-only command. The docs show the pattern: one key with --append-only for the not always trustworthy backup clients, a different key without it for backup management.

Separate credentials, separate accounts

• Use a dedicated repository password per machine, stored in a root readable file (--password-file or RESTIC_PASSWORD_FILE for restic, BORG_PASSCOMMAND or BORG_PASSPHRASE for Borg), and keep a copy in a password manager that does not depend on the homelab.
• Cloud credentials on the source should be write only where the provider supports it, and the bucket should have Object Lock or versioning with delete protection.
• The admin machine that prunes should not be the machine being backed up, and ideally should not be reachable from it.
• Turn on two step verification for every cloud account that holds backups. The NCSC recommends it for online backups.

Offline media is the last line

An unplugged disk cannot be encrypted over the network. Keep one rotated disk offline, and make its repository independent of your always-on remote: a separate repository with its own password, not a copy of the same files that a corrupted repository could poison. restic copy --from-repo can populate it from the main repository on a schedule you control.

Protecting against your own mistakes

Most homelab restores follow an rm in the wrong directory, a container upgrade that rewrote a database, or a job that quietly stopped including a path. A few habits catch most of these:

• Keep at least 7 daily and several weekly snapshots, so a mistake noticed a week later is recoverable.
• Back up configuration too: compose files, /etc, proxy configs and the backup scripts. CISA recommends keeping infrastructure as code templates in offline backups so you can redeploy quickly.
• Use --one-file-system (restic and Borg both have it) so a mounted share or USB disk is not pulled in, and --exclude-caches for cache directories.
• After changing includes or excludes, compare the newest snapshot's file count with the previous one.

How do you test restores, and what should you monitor?

Levels of restore testing

Not every test needs a full rebuild. Use a ladder and climb it on a schedule.

Restic's check docs explain why level 3 matters: by default check does not verify that the actual pack files are intact. --read-data downloads and checks everything. --read-data-subset=n/t checks one fixed slice, so running 1/5 through 5/5 on five separate days covers the whole repository; a percentage or size argument picks a random subset instead. On object storage, reading data costs egress, so subsets are kinder to the bill.

Borg's check has a repository phase and an archive phase. --verify-data performs cryptographic integrity verification of all archive data, and --max-duration splits a repository check into partial runs.

What to monitor

• Exit codes. Restic's scripting docs list 0 for success, 1 for failure, 3 when backup could not read some source data or forget could not remove a snapshot, 10 when the repository does not exist, 11 when the lock fails, 12 for a wrong password and 130 when the command was cancelled, and say any unknown code must be treated as a failure. Borg returns 0 for success, 1 for a warning (the run finished but you should read the log) and 2 for an error, with specific codes when BORG_EXIT_CODES=modern is set. Treat restic's 3 and Borg's 1 as alerts: they usually mean files were skipped.
• Snapshot age. Alert if the newest snapshot is older than about 1.5 times your schedule. This catches the job that silently stopped running.
• Size and file counts. A jump in new data can mean a stray cache directory or files rewritten by ransomware; a drop in file count can mean a bad exclude.
• Check results and offline disk age, in the same alert channel as the jobs.

Restic prints JSON with --json, and fatal errors end with a JSON message on stderr that includes the exit code, which is easy to feed into monitoring.

Step by step: back up a server and a Docker volume with restic

This walkthrough uses restic because it covers local disks, a remote append-only server and object storage with one tool. A Borg version of the core commands follows. Adjust paths, names and hosts to your own setup.

The layout:

• Source: a Linux server running Docker containers, with an app that stores data in a named volume and a PostgreSQL container.
• Copy 1: a local external disk mounted at /mnt/backup.
• Copy 2: a rest-server in append-only mode at another site, reached over your VPN.
• Admin: a separate laptop or small machine that runs forget, prune and check against both repositories.

Step 1: Install restic and confirm the version

Use your distribution's package or the official release binary, then confirm what you have.

sudo apt install restic        # Debian or Ubuntu; or download a release binary
restic version

Step 2: Prepare the local backup disk

Warning: the commands below erase the target disk. Run the device check first, confirm the serial number and size match the disk you intend to wipe, and do not continue if the device has partitions you recognise.

# Device check: list disks with model, serial, size and current mounts
lsblk -o NAME,MODEL,SERIAL,SIZE,TYPE,MOUNTPOINTS
ls -l /dev/disk/by-id/ | grep -v part

# Set this to the exact by-id path of the external disk you checked above
DISK=/dev/disk/by-id/usb-EXAMPLE_SERIAL-0:0
sudo wipefs --no-act "$DISK"   # dry run: shows signatures that would be removed

Only when the dry run shows the disk you expect:

sudo wipefs --all "$DISK"
sudo mkfs.ext4 -L backup1 "$DISK"
sudo mkdir -p /mnt/backup
echo 'LABEL=backup1 /mnt/backup ext4 defaults,noatime,nofail 0 2' | sudo tee -a /etc/fstab
sudo mount /mnt/backup

Step 3: Create the password file and the local repository

sudo install -d -m 700 /etc/restic
sudo sh -c 'head -c 32 /dev/urandom | base64 > /etc/restic/local.pass'
sudo chmod 600 /etc/restic/local.pass

sudo restic -r /mnt/backup/restic init --password-file /etc/restic/local.pass

Copy the contents of local.pass into your password manager now. Without it, the repository cannot be read. Our post on backup encryption keys covers where to keep keys so they survive the same disaster as the server.

Step 4: Set up the offsite append-only server

On the offsite machine, run rest-server with authentication, TLS, private repositories and append-only mode. The flags below come from the rest-server README.

# On the offsite machine
htpasswd -B -c /srv/restic/.htpasswd homeserver
rest-server --path /srv/restic \
  --append-only --private-repos \
  --tls --tls-cert /etc/rest-server/cert.pem --tls-key /etc/rest-server/key.pem \
  --tls-min-ver 1.3 --listen :8000

On the source server, store the rest-server credentials and a separate repository password, then initialize. With --private-repos, the path must start with the username.

# On the source server
sudo sh -c 'head -c 32 /dev/urandom | base64 > /etc/restic/offsite.pass'
sudo chmod 600 /etc/restic/offsite.pass
export RESTIC_REPOSITORY='rest:https://homeserver:REST_PASSWORD@offsite.vpn.lan:8000/homeserver/main'
sudo -E restic init --password-file /etc/restic/offsite.pass

Keep the REST credentials in a root only environment file rather than in shell history, as shown in Step 7.

If you prefer object storage with Object Lock, the repository string becomes an S3 URL such as s3:https://s3.example-region.example.com/bucket-name/homeserver, with access keys in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Give those keys only the permissions restic needs to write, and run prune from the admin machine with a different key.

Step 5: Back up a Docker volume and a database consistently

Docker's documentation shows the basic pattern for volume backups: start a throwaway container that mounts the volume and writes a tar archive. Combined with restic's --stdin-from-command, the archive streams straight into the repository with no temporary file, and restic records a failure if the command fails.

For an application volume, stop the container first so files are not changing mid-copy:

docker compose -f /opt/app/compose.yaml stop app

sudo restic -r /mnt/backup/restic --password-file /etc/restic/local.pass \
  backup --tag docker --stdin-filename app_data.tar \
  --stdin-from-command -- \
  docker run --rm -v app_data:/data:ro alpine tar -C /data -cf - .

docker compose -f /opt/app/compose.yaml start app

For a database, ask the database for a consistent dump rather than copying its files while it runs:

sudo restic -r /mnt/backup/restic --password-file /etc/restic/local.pass \
  backup --tag db --stdin-filename app.sql \
  --stdin-from-command -- \
  docker exec app-db pg_dump -U app appdb

Restic's docs explain why --stdin-from-command is preferred over piping into restic backup --stdin: restic cannot tell whether piped data is complete, but with --stdin-from-command it sees the command's exit status.

Step 6: Back up the host's configuration and data directories

sudo restic -r /mnt/backup/restic --password-file /etc/restic/local.pass \
  backup --tag host --one-file-system --exclude-caches \
  --exclude /var/lib/docker \
  /etc /opt /home /srv

Excluding /var/lib/docker keeps image layers out of the backup; you back up volumes and dumps explicitly in Step 5 and pull images again on restore.

Step 7: Wrap it in a script and a systemd timer

Put both repositories in one script so a single timer covers them.

sudo tee /etc/restic/offsite.env >/dev/null <<'EOF'
RESTIC_REPOSITORY=rest:https://homeserver:REST_PASSWORD@offsite.vpn.lan:8000/homeserver/main
RESTIC_PASSWORD_FILE=/etc/restic/offsite.pass
EOF
sudo chmod 600 /etc/restic/offsite.env

#!/bin/sh
# /usr/local/sbin/homelab-backup
set -eu
run() {
  restic backup --tag host --one-file-system --exclude-caches \
    --exclude /var/lib/docker /etc /opt /home /srv
  restic backup --tag db --stdin-filename app.sql \
    --stdin-from-command -- docker exec app-db pg_dump -U app appdb
}
# Local copy
RESTIC_REPOSITORY=/mnt/backup/restic RESTIC_PASSWORD_FILE=/etc/restic/local.pass run
# Offsite copy
set -a; . /etc/restic/offsite.env; set +a
run

# /etc/systemd/system/homelab-backup.service
[Unit]
Description=Homelab restic backup
After=network-online.target docker.service
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/homelab-backup
Nice=10
IOSchedulingClass=idle

# /etc/systemd/system/homelab-backup.timer
[Unit]
Description=Nightly homelab backup

[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=15m
Persistent=true

[Install]
WantedBy=timers.target

sudo chmod 700 /usr/local/sbin/homelab-backup
sudo systemctl daemon-reload
sudo systemctl enable --now homelab-backup.timer
systemctl list-timers homelab-backup.timer

Per the systemd.timer man page, Persistent=true stores the last trigger time on disk and runs a missed job as soon as the timer is active again, and RandomizedDelaySec= spreads start times so several machines do not hit one server together. Add an OnFailure= unit that alerts you when the service exits non-zero.

Step 8: Forget and prune from the admin machine

The source server cannot delete anything on the append-only remote, which is the point. Maintenance runs from the admin machine, which talks to a second rest-server instance (or a second user) that is not in append-only mode, or directly to the files over SSH.

# On the admin machine, against the offsite repository
restic -r "$OFFSITE_ADMIN_REPO" snapshots --compact
restic -r "$OFFSITE_ADMIN_REPO" forget --dry-run --keep-within 30d --keep-monthly 12 --keep-yearly 3
restic -r "$OFFSITE_ADMIN_REPO" forget --keep-within 30d --keep-monthly 12 --keep-yearly 3 --prune

Read the snapshot list before every forget. More snapshots than usual, odd hostnames or timestamps in the future are signs that something has been writing junk, and the restic docs explain why --keep-within is the safer policy here.

For the local disk, which the source server can write to directly, a count based policy is fine:

sudo restic -r /mnt/backup/restic --password-file /etc/restic/local.pass \
  forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 3 --prune

Step 9: Check the repositories

restic -r "$OFFSITE_ADMIN_REPO" check --read-data-subset=10%
sudo restic -r /mnt/backup/restic --password-file /etc/restic/local.pass check --read-data

Schedule the subset check weekly on the offsite repository and a full --read-data monthly on the local disk, where reading is free.

Step 10: Run a test restore

Restore into a scratch directory, never on top of live data. Restic's restore docs recommend --dry-run -vv first to see what would happen.

export RESTIC_REPOSITORY=/mnt/backup/restic RESTIC_PASSWORD_FILE=/etc/restic/local.pass
sudo -E restic snapshots --tag host          # note the newest snapshot ID
mkdir -p /tmp/restore-test
sudo -E restic restore SNAPSHOT_ID --target /tmp/restore-test --include /opt/app --dry-run -vv
sudo -E restic restore SNAPSHOT_ID --target /tmp/restore-test --include /opt/app

diff -r /opt/app /tmp/restore-test/opt/app && echo "restore matches live files"

For the Docker volume, restore the tar stream into a new volume and start a test copy of the stack against it. restic dump writes a single file from a snapshot to standard output, the same way the restic docs pipe a SQL dump back into a database.

sudo -E restic snapshots --tag docker        # note the newest snapshot ID
docker volume create app_data_restore
sudo -E restic dump SNAPSHOT_ID app_data.tar \
  | docker run --rm -i -v app_data_restore:/data alpine tar -C /data -xf -

Then point a copy of your compose file at app_data_restore, load the SQL dump into a fresh database container, start the app on a different port and check that it shows the data you expect. Write down how long it took. That number is your real recovery time.

The same core jobs with Borg

If you prefer Borg, the shape is the same. The commands below follow the Borg 1.4 quickstart and usage pages.

# Initialize with the docs' recommended mode, then export the key
borg init --encryption repokey /mnt/backup/borg
borg key export /mnt/backup/borg /root/borg-key-backup.txt

# Back up, then prune and compact
borg create --stats --one-file-system --exclude-caches \
  /mnt/backup/borg::'{hostname}-{now}' /etc /opt /home /srv
borg prune --list --dry-run --keep-daily 7 --keep-weekly 4 --keep-monthly 12 /mnt/backup/borg
borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 12 /mnt/backup/borg
borg compact /mnt/backup/borg

# Verify and test restore
borg check --verify-data /mnt/backup/borg
cd /tmp/restore-test && borg extract /mnt/backup/borg::ARCHIVE_NAME opt/app

For the offsite copy, put a forced command on the backup server so the source's SSH key can only append:

command="borg serve --append-only --restrict-to-repository /srv/borg/homeserver",restrict ssh-ed25519 AAAA... source-backup-key
command="borg serve --restrict-to-repository /srv/borg/homeserver",restrict ssh-ed25519 AAAA... admin-key

Snapshots, file sync, image backup and deduplicating backup compared

Most homelabs end up with three rows of this table: snapshots for fast undo, a deduplicating backup to a local disk, and an offsite copy that is append-only, immutable or offline.

Common mistakes

• Treating RAID or snapshots as the backup. They do not survive a compromised host or a fire.
• Keeping the only repository password on the server it protects. Keep it, or the exported key, in a password manager and on paper.
• Letting the source delete its own backups. If the server's credentials can prune the remote, ransomware with root can too.
• Count based forget on an append-only repository. Use --keep-within there, and read the snapshot list first.
• Copying live database files. Back up a database dump, or stop the container first.
• Never reading the data back. A check without --read-data or --verify-data does not prove the stored bytes are intact.
• Ignoring restic's exit code 3 or Borg's 1. Both usually mean skipped files.
• No alert for "nothing happened". A job that stops being scheduled produces no errors. Alert on snapshot age.
• Wiping the wrong disk. Run the lsblk and /dev/disk/by-id check and a wipefs dry run first.
• Backing up data but not the recipe. Without compose files, configs and restore notes, a full restore takes days.

FAQ

What is the 3-2-1 backup rule?

Keep three copies of important data (the original and two backups), on two different types of media, with one copy offsite. The rule appears in the 2012 US-CERT paper "Data Backup Options," which CISA still hosts. Modern guidance adds that at least one copy should be offline or immutable and that backups must be tested.

Is RAID a backup?

No. RAID keeps storage online when a disk fails, but it copies deletions, corruption from software bugs and ransomware encryption to every disk in the array instantly. A backup is a separate copy that the source machine cannot change.

Restic or Borg: which should I use for a homelab?

Both are solid, encrypted, deduplicating tools. Restic is a single binary with native support for many backends, including object storage and an append-only REST server. Borg is often chosen for SSH targets you control, where its append-only forced commands and per key restrictions are convenient. Pick the one whose remote target fits your offsite plan.

How do I make backups ransomware proof?

Nothing is proof against everything, but three measures cover most cases: an offsite repository the source can only append to, an Object Lock or offline copy that nothing on your network can delete before a set date, and maintenance run from a separate machine. Then test restores so you know the copies are good.

How often should I test restoring from backup?

Check job results on every run, run repository checks weekly, restore a sample of files monthly, restore a full service quarterly, and rehearse a full rebuild once a year. Adjust the cadence to how much the data matters to you.

Do restic and Borg encrypt backups?

Yes. Restic always encrypts with AES-256 in counter mode and authenticates with Poly1305-AES, deriving keys from your password with scrypt. Borg encrypts with AES-256-CTR and authenticates with HMAC-SHA-256 or BLAKE2b in its repokey and keyfile modes, which you choose when you run borg init.

Can I back up Docker volumes with restic?

Yes. Stop the container, start a throwaway container that mounts the volume read only and writes a tar stream, and pass that command to restic backup --stdin-from-command. For databases, back up a dump from the database itself rather than its data files.

What is append-only mode in backups?

A repository mode where the client can add new backups but cannot delete or overwrite existing data. Restic's rest-server and rclone serve restic offer --append-only, and Borg offers borg serve --append-only and an append_only repository setting. Deleting old backups then requires a separate, trusted admin machine.

Where Safeguard fits

Safeguard does not back up data, scan backups for malware, or detect ransomware, and it is not a substitute for anything in this guide. Its fit is indirect and sits on the software side of a homelab. The services you protect with these backups run as container images and small repositories full of open source components, and the backup tools themselves are open source software you install and update. Safeguard can scan a repository, a container image or an SBOM file (for example safeguard scan --dir . in the folder that holds your compose files, or safeguard scan --image <name> for an image you run), match components against vulnerability databases, show which projects contain a given component and version, and, with your approval, open fix pull requests. That keeps the stack you will one day restore from backup current and known.

A restore is also a rebuild: you pull images and dependencies again, and whatever versions resolve that day are what you run. The section below explains how Safeguard helps you keep those open source components on fixed versions, including when upstream has not shipped a fix yet (fork and patch is early access).

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

These demos show Safeguard working on software projects, not on backup repositories or disks. They apply to the compose files, Dockerfiles and small apps whose data you are backing up.

The first demo is a real Safeguard CLI session. After install, safeguard opens the interactive terminal, you ask it to fix the lodash CVE in a repository, and it reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with your approval, prints a summary and pushes nothing. The same ask, approve and test loop fits the repository behind a self hosted app whose volume you back up every night.

Click through it at your own pace, or open it full screen.

A homelab server usually never leaves your home network. The local runner demo shows how to set up the runner on your own machine, approve it with its device code, and then scan from the terminal. A runner on the Docker host, next to the backup jobs, can scan the images and repositories that live there.

Click through it at your own pace, or open it full screen.

If you manage the lab from your phone, the mobile demo is a real Android session: you send work to a computer running the CLI and approve each command on the phone with Allow once, Allow for turn or Deny. Nothing runs on the server without your tap, which is the same principle as keeping your backup admin credentials off the machine being backed up.

Click through it at your own pace, or open it full screen.

To try it on your own repositories, download Safeguard and follow the CLI documentation.

Quick check

Sources

• CISA (US-CERT): Data Backup Options, Ruggiero and Heckathorn, 2012 (PDF)
• CISA: #StopRansomware Guide
• NIST SP 800-34 Rev. 1, Contingency Planning Guide for Federal Information Systems
• NIST IR 8374, Ransomware Risk Management: A Cybersecurity Framework Profile
• NCSC: Small organisations guide, Backing up your data
• restic documentation: References and design
• restic documentation: Preparing a new repository
• restic documentation: Backing up
• restic documentation: Working with repositories
• restic documentation: Restoring from backup
• restic documentation: Removing backup snapshots
• restic documentation: Encryption
• restic documentation: Scripting
• restic rest-server README
• BorgBackup documentation: Quick Start
• BorgBackup documentation: Data structures and file formats
• BorgBackup documentation: Security
• BorgBackup documentation: borg init
• BorgBackup documentation: borg create
• BorgBackup documentation: borg prune
• BorgBackup documentation: borg check
• BorgBackup documentation: borg serve
• BorgBackup documentation: Additional notes (append-only mode)
• BorgBackup documentation: General (return codes)
• rclone documentation: Crypt
• rclone documentation: Amazon S3 and S3 compatible storage
• rclone documentation: rclone serve restic
• Amazon S3 User Guide: Locking objects with Object Lock
• Docker documentation: Volumes (back up, restore, or migrate data volumes)
• systemd.timer man page
• OpenZFS man page: zfsconcepts(7)
• restic releases on GitHub
• BorgBackup releases on GitHub

Last reviewed: 24 October 2026. Backup tool flags and defaults change between releases; check the documentation for the version you run, and dry run any command that deletes snapshots or erases a disk.