There are many different ways how you can backup your data. My preferred solution is to store backups on Amazon S3. It's cheap and not that complicated to set up. First, you need an Amazon account if you do not already have one. Go to https://aws.amazon.com/s3/ and click Sign Up.
In the following tutorial, we will set up a backup for a self-hosted Gitea server, but you can apply this tutorial to other files you want to backup to S3.
Create Bucket ¶
After signing up, open the S3 web console
Create a new bucket, choose a bucket name, and select the region.
We don't need special properties for this bucket and ensure that you do not grant public access to the bucket.
Lifecycle rule ¶
I usually add a lifecycle rule that automatically moves files after a few days from S3 to Glacier. Storing data on Glacier is much cheaper than S3, but downloading from Glacier costs you more, and the files have to be stored for at least 90 days on Glacier if you delete them before additional fees apply.
Glacier is especially useful for backups because you rarely need to download them.
Also, check the documentation about other storage classes.
Adding a lifecycle rule only makes sense when you backup files that are revisioned. For instance: backup-1.zip, backup-2.zip, backup-3.zip or backup-20180601.tar.gz, backup-20180602.tar.gz. The lifecycle rule never applies if you always overwrite the same backup file.
Click on the bucket name and open the Management tab, then click on Create lifecycle rule.
Enter a rule name
Under Lifecycle rule actions select the following options.
Under Expire current versions of objects enter 95
And under Transition enter 5.
Policy ¶
Next, we need to create a policy. Open the IAM console and go to Policies and create a new policy.
Select the service S3, under Actions select the PutObject action. Under Resources specify the bucket that we created before. Make sure that the object ends with /*
Review the policy, give it a name, and create it.
User ¶
Go to Users and click on Add user. Enter a user name and select Access key.
Click on Next: Permissions. Next, click Attach existing policies directly and search for the policy you created in the previous step. Select the policy.
Click Next: Review and Create user.
The following dialog shows the Access key ID and Secret access key. Take a note or download them (Download .csv).
Install tools ¶
I run all the following commands as a root user. If you are not logged in as root, prepend sudo to the commands or switch to root with sudo -i.
On the VPS, we install s3cmd, a command-line client for Amazon S3.
apt install s3cmd
Upload an arbitrary file to check if everything is set up correctly.
s3cmd --access_key=AKIA.... --secret_key=LDf... put /usr/local/bin/gitea s3://rasc.giteabackup
Next, we install gpg to encrypt our backups. This protects our backups from the eyes of Amazon and everybody else that gains access to our S3 bucket. This is optional; if you don't care about
the security of your backup files, you can skip this.
apt install gpg gpg-agent
We are encrypting the backup with AES, a symmetric encryption algorithm. Here is an example of how you can use gpgto AES encrypt and decrypt a text file.
gpg --cipher-algo AES256 --symmetric --batch --passphrase the_passphrase test.txt
gpg --decrypt --batch --passphrase the_passphrase -o test.txt -d test.txt.gpg
Backup Gitea ¶
Gitea provides the dump command that stores the configuration and the repositories into one zip file. We have to run the command with the git user.
cd /var/lib/gitea/
sudo -H -u git bash -c "/usr/local/bin/gitea -c /etc/gitea/app.ini dump -f backup.zip -w /var/lib/gitea"
With all the pieces in place, we can create a bash script that runs dump, encrypts the file, and uploads it to S3.
cd /var/lib/gitea/
nano gitea-backup
Insert the following code. Insert the correct access_key and secret_key.
#!/bin/sh
now=`date +"%Y%m%dT%H%M"`
cd /var/lib/gitea
sudo -H -u git bash -c "/usr/local/bin/gitea -c /etc/gitea/app.ini dump -f backup_${now}.zip -w /var/lib/gitea"
gpg --cipher-algo AES256 --symmetric --batch --passphrase the_passphrase backup_${now}.zip
s3cmd --access_key=AK.... --secret_key=ax.... put backup_${now}.zip backup_${now}.zip.gpg s3://rasc.giteabackup
rm /var/lib/gitea/backup_${now}.zip
rm /var/lib/gitea/backup_${now}.zip.gpg
Change the permission so we can run it as a script.
chmod 755 gitea-backup
Test the script with ./gitea-backup. Then, visit the Amazon S3 web console and check if the file is stored in the bucket.
Setup systemd timer ¶
Next, we install a scheduler to run this script periodically. We do this with systemd, which has a built-in timer service.
Create a timer file
cd /var/lib/gitea
nano gitea-backup.timer
Add this code. This instructs systemd to run the backup script every day at 5 am.
[Unit]
Description=Run gitea-backup once a day
[Timer]
OnCalendar=*-*-* 05:00:00
RandomizedDelaySec=30
Persistent=true
[Install]
WantedBy=timers.target
Create the corresponding service file
nano gitea-backup.service
[Unit]
Description=gitea-backup
[Service]
WorkingDirectory=/var/lib/gitea
Type=oneshot
ExecStart=/var/lib/gitea/gitea-backup
Then we link the two files into the /lib/systemd/system folder, start the timer and then enable it, so it automatically starts each time the server boots up.
ln -s /var/lib/gitea/gitea-backup.timer /lib/systemd/system/gitea-backup.timer
ln -s /var/lib/gitea/gitea-backup.service /lib/systemd/system/gitea-backup.service
systemctl daemon-reload
systemctl start gitea-backup.timer
systemctl enable gitea-backup.timer
Check that the timer is installed
systemctl list-timers
To test the service run this command
systemctl start gitea-backup