Home | Send Feedback

Backup with restic

Published: April 02, 2020  •  linux, windows

There are various ways to backup data on a computer, and there are various tools that help you with this task. In this blog post, we are going to take a look at a command-line backup tool called restic.

restic

restic is an easy, fast, efficient, and secure backup utility. It is free and licensed under the BSD 2-Clause License.
It is actively developed on GitHub: https://github.com/restic/restic

restic is written in Go, and it runs on Linux, Windows, macOS, FreeBSD, and OpenBSD.
You can download the binary from the release page: https://github.com/restic/restic/releases
Or you install the application with a package manager. Visit the installation documentation page to find the installation steps for your operating system.

For this blog post, I worked on Windows 10 with restic 0.9.6. I downloaded the zip file from the release page, unzipped it, and copied the exe file it into an arbitrary folder. On Windows, you could copy the exe into the %SystemRoot%\System32 folder, and then it's available in the path automatically. Or you add the path to your restic folder to the %PATH% environment variable.


restic supports out of the box many different backends:

If your preferred backend is not listed here, you can couple restic with rclone and use all the supported service of this tool. Visit the restic documentation to learn more about rclone support. And visit the rclone homepage to see a list of all supported backends.


restic is not a trivial file copy program. It works a little bit differently. It is based around the concept of repositories and snapshots. When you backup something, it is stored in a snapshot, which is stored in a repository.

The data in such a repository is always encrypted with AES. This is especially important when you back up sensitive data, and you send the backups to a server which is not under your control, for example to one of the cloud providers.

Initialize Repository

Before we can backup something, we have to initialize the repository. For this example, I connected an external hard disk (z:), and there I created an empty folder backup.

Run the following command to initialize the repository.

# restic init --repo z:/backup

restic now asks you for the password. You have to remember this password. If you lose it, you can no longer access the backup!

enter password for new repository:                                  
enter password again:                                               
created restic repository 5db7847a29 at z:/backup                   
                                                                    
Please note that knowledge of your password is required to access the repository. 
Losing your password means that your data is irrecoverably lost.                                                 

Backup

The repository is created, and we can start backing up data. Each time you run a backup, restic creates a snapshot in the repository. To back up one or multiple folders or files, enter this command

# restic backup -r z:/backup c:/work d:/another_work_folder

-r is the short form of --repo and tells restic which repository to use. At the end of the command, you list files and directories you want to backup. In this example I backup two folders, which are located on two different local disks.

restic asks for your password and backs up the data. For demo purposes, I created one text file in each folder so restic had to transfer two files, which it tells me in the summary output.

enter password for repository:
repository 5db7847a opened successfully, password is correct
created new cache in C:\Users\sr\AppData\Local\restic

Files:           2 new,     0 changed,     0 unmodified
Dirs:            2 new,     0 changed,     0 unmodified
Added to the repo: 1.179 KiB

processed 2 files, 20 B in 0:00
snapshot 1ba98702 saved

Rerun the same command and see one of the benefits of restic. It created another snapshot, but it did not copy any files because there were no changes. This saves space on the backup drive, and it saves time and bandwidth when you back up over a network.

enter password for repository:
repository 5db7847a opened successfully, password is correct

Files:           0 new,     0 changed,     2 unmodified
Dirs:            0 new,     0 changed,     2 unmodified
Added to the repo: 0 B

processed 2 files, 20 B in 0:00
snapshot 47a15bab saved

The backup command also supports command-line options to exclude and include files based on patterns. See the official documentation to dig deeper into this topic.


A tool like this is ideal for an automated backup script. The problem is that restic always requires a password to access the repository and to encrypt the data. The secure way would be that you always enter the password manually, which is not an option when you want to install a scheduled job that runs unattended.

There are different ways to specify the password and pass it to restic. You can set the environment variable RESTIC_PASSWORD, or you can store the password in a file and then specify the path to that file with the option --password-file or the environment variable RESTIC_PASSWORD_FILE. The third option is to call a program when a password is needed with the option --password-command or the environment variable RESTIC_PASSWORD_COMMAND.


Different backends require different environment variables. On this page you find all supported environment variables, and this page tells you how to configure the supported backends.

Restore

The restore command restores files and folders. restic does not restore the files into the original path. You have to specify a target location where restic restores the files.

# restic restore 47a15bab -r z:/backup --target c:/restore

Instead of specifying the snapshot ID, you can also use the keyword latest to restore the latest backup.

# restic restore latest -r z:/backup --target c:/restore

Like the backup command, restore supports filter options, if you want to restore just a host or a path or just a couple of files. The following example restore just the file workfile.txt

# restic restore latest -r z:/backup --target c:/restore --include workfile.txt

On supported operating systems, you can mount the repositories with FUSE and then browse them like a regular folder. This is not supported on Windows.

If you want to learn more about the restore command, visit the documentation page.

Manage Snapshots

List

We called the backup command twice, so there are now two snapshots in our backup repository. To list them, we use the snapshots command.

# restic snapshots -r z:/backup
enter password for repository:
repository 5db7847a opened successfully, password is correct
ID        Time                 Host        Tags        Paths
-----------------------------------------------------------------------------
1ba98702  2020-04-01 16:35:38  rog                     D:\another_work_folder
                                                       c:\work

47a15bab  2020-04-01 16:38:55  rog                     D:\another_work_folder
                                                       c:\work
-----------------------------------------------------------------------------
2 snapshots

If you have many snapshots, you can filter them by path and host, and you can group them. See the documentation page for more information.


Diff

The diff command compares two snapshots with each other

# restic diff -r z:/backup 1ba98702 47a15bab                  
enter password for repository:                                
repository 5db7847a opened successfully, password is correct  
comparing snapshot 1ba98702 to 47a15bab:                      
                                                              
                                                              
Files:           0 new,     0 removed,     0 changed          
Dirs:            0 new,     0 removed                         
Others:          0 new,     0 removed                         
Data Blobs:      0 new,     0 removed                         
Tree Blobs:      0 new,     0 removed                         
  Added:   0 B                                                
  Removed: 0 B                                                

Check Integrity

From time to time, you should check the integrity of the repository. The data in the repository is encrypted but the whole repository is just a collection of files and folders, and somebody could overwrite or delete a file. The check command verifies the repository and tells you if everything is okay or not.

# restic check -r z:/backup
repository 5db7847a opened successfully, password is correct
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
no errors were found

Cleanup

To remove a snapshot restic provides two commands forget and prune. The forget command only deletes the snapshot from the index. The snapshot is no longer visible when you run the snapshots command, but the data is still there. To actually remove the data, you have to call the prune command.

restic forget -r z:/backup 1ba98702
restic prune -r z:/backup

Because you often use these two commands in tandem, you can also add the --prune option to the forget command. This command has the same effect as the two commands above

restic forget --prune -r z:/backup 1ba98702

The documentation recommends to run a check after prune.


Deleting snapshots manually is cumbersome, and it's easy to delete the wrong snapshot accidentally. For these reasons, restic provides a way to delete snapshots based on a policy. A policy could be to keep the last 3 snapshots or to keep snapshots of the last 20 days.

The forget command supports various keep-* parameters. For example to delete all but the last snapshot you can execute this command

# restic forget --prune --keep-last 1 -r z:/backup

If you want to keep the snapshots of the last 7 days, you issue this command. If there are multiple snapshots in a day, restic removes all but the last one for that day.

# restic forget --prune --keep-daily 7 -r z:/backup

For example, you backup 6 times a day for 10 days. You now have 60 snapshots. After you run this command, there are 7 snapshots left in the repository. One snapshot for each of the last 7 days.

You can even combine multiple policies.

# restic forget --prune --keep-daily 7 --keep-monthly 12 --keep-yearly 3 -r z:/backup

With this command forget keeps one snapshot for each of the last 7 days, always the last snapshot of a month for the last 12 months and always the last snapshot of a year for the last 3 years.

See the documentation for more example and a more in-depth description of the options.

Rest Server

Backup and restore works, but there is one issue we have to keep in mind. restic always requires a password to access the repository and to encrypt the backup. When you install a scheduled job, you have to store this password somewhere, as mentioned before, you can store it in an environment variable or a file. The problem is that when an attacker gains control over your computer, he could also figure out this password. He then could not only delete the data on your computer but also all the snapshots in the restic repository.

If that is a concern, you should consider installing a restic REST server. This server has a unique feature called append-only mode. When the server runs in this mode, you can backup your data normally but the server refuses to execute the forget command, even when an attacker has access to your restic password. Your backups are secure and can't be deleted, as long as your backup server is secure.


In this section, I'm showing you how to install this REST server on a virtual private server. I rented a Debian 10 VPS from Hetzner, using the smallest configuration for €2.68 / per month with a 20 GB SSD for this example installation.


Install

First, we download the server application from the GitHub release page. The server, like the client, is written in Go and consists of just one binary.

mkdir /opt/restic
mkdir /opt/restic/data/
cd /opt/restic
wget https://github.com/restic/rest-server/releases/download/v0.9.7/rest-server-0.9.7-linux-amd64.gz
gunzip rest-server-0.9.7-linux-amd64.gz
chmod 744 rest-server-0.9.7-linux-amd64.gz

Next, we create a password. This step is optional, and you can run the server without authentication. The password we create here is just for authenticating to the rest server. This password has nothing to do with the repository password. The repository password is still required but will always be set from the backup client.

For creating the password, we need the htpasswd program, which is part of the Apache utils package.

apt install apache2-utils

Then we create a password and encrypt it with bcrypt. Choose an arbitrary username. The password file needs to be located in the data directory. In our installation this is /opt/restic/data

cd /opt/restic/data
htpasswd -B -c .htpasswd backup_user

I don't want to run the rest-server with the root user, so I create a new user here, and change ownership of the program directory.

adduser --system --shell /bin/false --gecos 'Restic Rest Server user' --group --disabled-password restic
chown restic:restic -R /opt/restic

Now we install the service in systemd so that the rest-server will be started when the server starts.

cd /opt/restic
nano rest-server.service

Paste the following code into the file, save and close it.

[Unit]
Description=Rest Server
After=syslog.target
After=network.target

[Service]
Type=simple
User=restic
Group=restic
ExecStart=/opt/restic/rest-server-0.9.7-linux-amd64 --append-only --path /opt/restic/data
Restart=always
RestartSec=5
StartLimitInterval=0

[Install]
WantedBy=multi-user.target

The important flag here is --append-only. This is the flag we talked about before to prevent somebody from executing the forget command.

ln -s /opt/restic/rest-server.service /lib/systemd/system/rest-server.service
systemctl daemon-reload
systemctl start rest-server
systemctl enable rest-server

Check if the service runs properly

systemctl status rest-server

The rest-server runs by default on port 8000. Make sure that you open this port in the firewall, or change the port with the --listen option

/opt/restic/rest-server-0.9.7-linux-amd64 --listen ":55443" --append-only --path /opt/restic/data

Backup

As with any backend, we first have to initialize the repository. Instead of a path, we now have to specify the URL to the rest-server. Because we also configured username and password, we have to pass this information as well to the tool.

restic init -r rest:http://backup_user:password@95.217.184.211:8000/

If that command is successful, you can now backup to your own restic rest-server

restic backup -r rest:http://backup_user:password@95.217.184.211:8000/ c:/work d:/another_work_folder

Now let's see what happens when we try to delete the snapshot.

# restic snapshots -r rest:http://backup_user:password@95.217.184.211:8000/
# restic forget -r rest:http://backup_user:password@95.217.184.211:8000/ 075b5b93
enter password for repository:
repository f03e5278 opened successfully, password is correct
Remove(<snapshot/075b5b931d>) returned error, retrying after 552.330144ms: blob not removed, server response: 403 Forbidden (403)

Although the password is correct, you get a bunch of error messages, thanks to the --append-only flag of the server. Even when an attacker gains access to your restic password, he can't delete the backups.


The question now is what if you want to delete old snapshots because you only have limited disk space on your VPS. The repository the rest-server creates is a regular repository, and you can access it with the restic command-line tool locally.

Open an SSH connection to your VPS, and install the restic command-line tool.

apt install restic

Now you can run the forget command. Enter the correct password, and your snapshot is gone.

restic forget --prune -r /opt/restic/data 075b5b93

TLS

So far we used HTTP for the transfer, this is not a problem for the backup because that is already encrypted on the client-side, but we sent our HTTP username and password in plaintext over the Internet.

We can solve that by installing a TLS certificate. We are going to create a Let's Encrypt certificate, and for that, you need to have a domain name. Add an A record with the IP address of your server. For this example I use the domain backup.rasc.ch

Install certbot and request a TLS certificate

apt-get install certbot
certbot certonly --standalone

When successful, cerbot should have created the key and certificate in the /etc/letsencrypt folder. In my example, the TLS certificate is stored in /etc/letsencrypt/live/backup.rasc.ch/fullchain.pem and the private key is located here /etc/letsencrypt/live/backup.rasc.ch/privkey.pem.


Open the service file.

nano /opt/restic/rest-server.service

Add the required TLS options

ExecStart=/opt/restic/rest-server-0.9.7-linux-amd64 --tls \
          --tls-cert /etc/letsencrypt/live/backup.rasc.ch/fullchain.pem \
          --tls-key /etc/letsencrypt/live/backup.rasc.ch/privkey.pem \
          --append-only --path /opt/restic/data

Restart the rest-server

systemctl daemon-reload
systemctl restart rest-server

With this configuration in place, you now have to use HTTPS from the client-side

restic snapshots -r rest:https://backup_user:password@backup.rasc.ch:8000/

If you want to learn more about the restic rest-server check out the project page:
https://github.com/restic/rest-server