[[!meta title="Backups of the persistent volume"]]

[[!toc  levels=2]]

We want to allow our users to backup the data they have in their persistent
volume.

See the thread on tails-dev:

<https://mailman.boum.org/pipermail/tails-dev/2014-April/005388.html>


Roadmap
=======

- Research which requirements are really needed by our users
- Evaluate existing tools based on these requirements 
- Implement the missing features, integrate into Tails 
- Write end-user documentation
- Write automated tests
- Evaluate UX


Requirements
============

MUST
----

- Allow backup of Persistent folder
- Allow backup on Local storage
- Encrypt backups
- Preserve permissions and ownership
- Understandable error messages
- Require administrator password when hitting a file not readable by amnesia
  (eg. private key of onion services).

SHOULD
------

- Allow backup on a remote storage
  - ownCloud
- Feature a backup history
- Backing up everything in persistence (eg APT)
- Only send the delta from previous backup (i.e incremental update)
- Automated reminder
- Include, exclude lists


Backup techniques
=================

We shouldn't maintain yet another shiny new piece of code to do that. Or we
might hit tons of corner cases to handle, that one initially does not think of.
And once we take it all into account, then we get a large piece of code to
maintain all by ourselves, because it was meant only for Tails' needs.

So, we shouldn't try to reinvent this wheel, and use an existing, proven solution instead.

GNOME Backups (Deja Dup)
------------------------

  - <https://launchpad.net/deja-dup>
  - <https://wiki.gnome.org/Apps/DejaDup>

Pros:

  - Has a simple interface for configuring backups. The nicest we could
    find and with a [clear mission statment](https://wiki.gnome.org/Apps/DejaDup/Mission).
  - Restoration of single files or folders is integrated in Files.
  - Notifications to start backups periodically.
  - Supports different kinds of remote: FTP, SSH, WebDAV, and Windows
    share are displayed in the graphical interface, duplicity supports
    [many more](http://duplicity.nongnu.org/).

Cons:

- Uses duplicity as back end with the following drawbacks:
  - It forces full backups no and then making them unexpectedly
    extremelly slower than regular incremental ones.
  - Restoring a single file is slow.
  - Impossible to navigate the file system, either of the last or past
    increments without doing a full restoration.
- No feedback whatsoever on the progress of the backup when they get started automatically.
- Doesn't know how to ask for root password for backing up files it cannot access.
- If files are mounted or linked in a different place, you can only
  restore from Nautilus the original places backed up by Deja Dup. That's
  a problem for dotfiles which are single files symlinked to $HOME and you
  can't easily get both the Nautilus integration and the full restoration.
- You can only add folders to be backed up or ignored by Deja-Dup (and not single files).
- Fails to restore symlinks.

See [[!tails_ticket 9888]].

Tar + GPG
---------

A simple approach might be to combine tar and GPG:

    tar cjf - . | gpg --cipher-algo AES -c - > /home/amnesia/YYYY-MM-DD-backup.tbz2.gpg

Initial implementation might be easy but that will probably end up being quite a
big piece of custom code.

Duplicity
---------

<http://duplicity.nongnu.org/>

It supports something that's basically "tar | gpg" for the first iteration, and
it also leaves room for future improvements, thanks to its support for
incremental and remote backups. Also, it allows users to restore or inspect
their backups outside of Tails, without having to manually decipher yet another
backup file format.

### Cons

- Duplicity creates tons of messy files on the file system.
- And requires users to do a full backup from time to time when using incremental backups

Loopback LUKS
-------------

### Pros

- One file per backup.

### Cons

- We still need to find another tool to create the device and copy the files.
- Maybe backups done this way would be much bigger than duplicity backups.
- Duplicity supports incremental backups even if they have some limitations.

Obnam
-----

Obnam only supports encrypting to a GnuPG key, so this would require another
layer of encryption (such as a LUKS container file, or something else) if we
want to support encrypting with a passphrase (and I think we should).

grsync + encrypted device
-------------------------

[grsync](http://www.opbyte.it/grsync/) is a GUI for the renown and rock-solid `rsync`, already shipped in Tails.

Grsync is [packaged for debian](https://packages.debian.org/squeeze/grsync).

The documentation for the creation of the encrypted device [[is already written|doc/encryption_and_privacy/encrypted_volumes]].

It has a rock-science interface, basically displaying the complexity of
the command line options in a graphical way.

### How to test?

* create an encrypted device.
* install the grsync package.
* paste [those lines](https://paste.debian-facile.org/view/raw/a7a7fe3c) in a `.grsync` file, then double-click on it.
(grsync ask you first to confirm the profile you want to use. Just click on "Open")
* define the destination (i.e your encrypted device)
* test wildly! and please report your results

### Pros

* not that much things to code
* grsync can be easily preconfigurated, eventually with multiple profiles
* this solution separates backup and encryption work
* allows remote backups

### Features to request

* grsync should check if enough space is available on the destination before running.
  Update: rsync 3.1.0 [introduces](https://rsync.samba.org/ftp/rsync/src/rsync-3.1.0-NEWS) a `--preallocate` option.
  <del>(Tails actually ships rsync 3.0.9</del> Tails 2.6 ships rsync 3.1.1)
* grsync should ask for confirmation when the option "Delete on the destination" is activated
* when user double-click on a `.grsync` file, a window appears to confirm which file to open. This may be confusing.

### Misc

* some files are normally not readable by rsync (for example persistence.conf, apt/*)
  Grsync can bypass that with the option "Run as superuser", we should investigate the consequences of using such an option.
  We still have the possibility to ignore those files: we just have then to add `--exclude="apt"` in the preconfiguration file.
* decide if we activate the option `--delete` by default.
* test restoration (see File → Invert source and destination). Then, at least, check files permissions.
* test backup + restoration with symlinks and hardlinks in the Persistent folder.
* eventually test remote backups.
* see the [thread on tails-dev](https://mailman.boum.org/pipermail/tails-dev/2015-January/007931.html)

rdup
----

<https://github.com/miekg/rdup>

rdup separates the logic of backing up from the actual copying. It
supports filters to compress and encrypt individual files (typically
using gpg or mcrypt) as well as path names and can do both full as
well as incremental backups.

### Pros

* simple and small command line programs
* more sophisticated than tar+gpg and probably addresses many of the
  corner cases that would otherwise have to be handled by increasingly
  complicated scripts
* in Debian Squeeze / Wheezy / testing

### Cons

* still requires a script of some sort to drive it
* probably requires a gui to make it simple to use

borgbackup
----------

Borg is the perfect backup back end. It supports increments, encryption,
data deduplication, local and remote backups, and mounting backups as
FUSE file systems. And it way faster than obnam which advertises similar
properties. But it doesn't have a graphical user interface.

Clone everything
----------------

We could add to *Tails Installer* an option to **Clone everything**
(Tails partition and persistent volume). This would clone the original
USB stick and also backup all the persistent data.

We could encourage people to submit patches for this even if it's not a
perfect solution. The back end could be quite simple and use rsync
between the two persistent volumes.

A possible improvement would be to skip the cloning of the Tails
partition if the destination USB is already a Tails USB stick.

### Pros

* That's a very simple user interface: it's only possible to do full
  backups. If you loose your data, use the backup USB stick. We don't
  have to provide an option to restore.

### Cons

* No option to do incremental backups.
* People need a dedicated USB stick.

Other solutions
---------------

  - `[sbackup](https://tracker.debian.org/pkg/sbackup)`, Simple Backup:
    unmaintained since 2008.

  - [Lucky Backup](https://tracker.debian.org/pkg/luckybackup): seems
    very oldish and not really active.

  - [Back In Time](https://github.com/bit-team/backintime) which has a
    GNOME frontend. It does snapshots with hardlinks to reduce space.
    Can do local and SSH as remote.

User experience
===============

- The user should be prompted about where to save the backup as this will most
  probably end up directly on a storage device and not go through RAM.

- Ask the user which folders of the persistence setup to backup. But maybe
  that's not needed for a first prototype, as we might assume that the info in
  persistence is kept to the minimum.

- The restore step would do the current delete partition steps (if there was
  already a persistence partition found), then most of the current steps to
  create a partition, but instead of asking the user what they want to use their
  new partition for, it would instead ask the user to pick a backup and would
  restore it.  

- There could be a single menu entry called "Backup and restore persistent
  volume". Combining both actions in a single menu entry makes it clear that
  they are directly related, and limits the number of menu entries.

- There should be a graphical way to restore backups. It is important let
  non-techy users restore their backups if necessary.

Implementation
==============

- Avoid running the GUI as root and rather run the GUI as amnesia or a dedicated
  user and execute precise actions through policykit (execute a command with
  pkexec or implement a dbus service)

- Write the code in GTK3 for Tails 1.1 which will be the next major release,
  based on Debian Wheezy.
