Backup

From Anarchaserver

Workshop

In this workshop, we will create a bash script to run as a cron job for our system backups. We have installed and run restic for one-off backups and we would like to schedule them on a daily basis. The backup takes place between anarchaserver and a remote server, which is the backup repository. But it can be also applied between our PC and a raspberry-pi or any home based file server.

Meeting Sunday 04 April 2021 - 3PM CET Time here: https://bbb.futuretic.fr/b/spi-utv-eo9-lqe

En este taller, vamos a crear un script bash para ejecutar como un trabajo cron para nuestras copias de seguridad del sistema. Hemos instalado y ejecutado restic para copias de seguridad puntuales y nos gustaría programarlas diariamente. La copia de seguridad tiene lugar entre anarchaserver y un servidor remoto, que es el repositorio de la copia de seguridad. Pero también se puede aplicar entre nuestro PC y una raspberry-pi o un servidor de archivos casero.

Cita el Domingo 04 de Abril - 3PM CET time aqui: https://bbb.futuretic.fr/b/spi-utv-eo9-lqe

Current setup

Current setup

We currently make a nightly backup of all containers from our virtual machine (called anoia) to the virtual machine of calafou.org (called ebro).

We use the backup tool called duplicity through a package from the Debian repository.

We wrote a script that performs that backup, which is located at /usr/local/bin/backup3.sh

The backup script runs every night around 4am, because we run it via cron (configured with sudo crontab -e command).

In the cron job we redirect the script output to /var/log/backup.log

The script stops ALL the containers (to have enough RAM for running duplicity and to make sure that the databases and other volatile files inside the containers are not changing during the backup), backs up the containers, then starts the containers again. Obviously, while the containers are stopped and the backup is running, the anarchaserver services are not available. We have many ideas about how to do a backup without service interruption, but we have not implemented them yet.

There are many possible improvements to this system:

  • Change remote: switch from calafou virtual machine to S14
  • Optimise the script: rewrite the script to be a more beautiful program
  • Test backup recovery: try to restore the backup to make sure it is possible
  • Change the backup tool: use borgbackup instead of duplicity
  • Use a backup manager such as backupninja
  • Etc.

We also have a S14 for setting up the tunnel for backup and openvpn, info about back-tunnel here.

Restore a backup

duplicity scp://anoia@ebro.tachanka.org//home/anoia/backup/repository /var/lib/lxc --ssh-options="-oIdentityFile=/var/backups/.ssh/id_binti" --verbosity 6

Useful links

Older notes

STEP 1 check data usage df -h

STEP 2 check access to another virtual machine

STEP 3 start screen session in root

STEP 4 copy data to virtual machine


rsync -ravz --progress --exclude "/proc" --exclude "/sys" --exclude "/dev" -e "ssh -p 8022" / root@188.210.92.35:/var/backups/vm/anarcha


r (recursive) a (archive - for special files) v (verbose) z (compressed)


STEP 5 diagnostics

watch watch -n 180 "du -sh /var/backups/vm/anarcha"

proc/kcore is a special directory which is created each time the computer boots so we have to exclude it from the rsync command

bwm-ng we want to know whether the data is arriving and at which speed, data amount http://linux.die.net/man/1/bwm-ng

apt-get install bwm-ng

Retrieved from "https://alexandria.anarchaserver.org/index.php?title=Backup&oldid=935"