Home Fileserver: Backups from ZFS snapshots

Backups are critical to keeping your data protected, so let’s discover how to use ZFS snapshots to perform full and incremental backups.

In the last article on ZFS snapshots, we saw how to create snapshots of a file system. Now we will use those snapshots to create a full backup and subsequent incremental backups.

Performing backups

Obviously we only created a small number of files in the previous ZFS snapshots article, but we can still demonstrate the concept of using snapshots to perform full and incremental backups.

We’ll write our backups to a backup target file system called ‘tank/testback’.

This backup target could exist within the same pool, like in our simple example, but would most likely exist in another pool, either on the same physical machine, or at any location addressable, using iSCSI or ssh with an IP address etc.

Full backup

Now let’s do a full initial backup from the ‘tank/test@1’ snapshot:

# zfs send tank/test@1 | zfs receive tank/testback

Let’s take a look at the file systems to see what’s happened:

# zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
tank                 766G   598G  28.0K  /tank
tank/test           94.6K   598G  26.6K  /tank/test
tank/test@1         23.3K      -  26.6K  -
tank/test@2         21.3K      -  26.0K  -
tank/test@3         21.3K      -  26.0K  -
tank/test@4             0      -  26.6K  -
tank/testback       25.3K   598G  25.3K  /tank/testback
tank/testback@1         0      -  25.3K  -

Well, the command not only created the file system ‘tank/testback’ to contain the files from the backup, but it also created a snapshot called ‘tank/testback@1’. The reason for the snapshot is so that you can get the state of the backups at any point in time.

As we send more incremental backups, new snapshots will be created, enabling you to restore a file system from any snapshot. This is really powerful!

Let’s just take a look at the files in the full backup — it should contain the original files referenced from our initial snapshot ‘tank/test@1’.

# ls -l tank/testback
total 4
-rw-r--r--   1 root     root          15 May 12 14:50 a
-rw-r--r--   1 root     root          15 May 12 14:50 b

# cat /tank/testback/a /tank/testback/b
hello world: a
hello world: b

As we expected. Good 🙂

Incremental backups

Now let’s do an incremental backup, that will only transmit the differences between snapshots ‘tank/test@1’ and ‘tank/test@2’:

# zfs send -i tank/test@1 tank/test@2 | zfs receive tank/testback
cannot receive incremental stream: destination tank/testback has been modified
since most recent snapshot

Oh dear! For some reason, doing the ‘ls’ of the directory, when we inspected the backup contents, has actually modified the file system.

I have no idea how this happens or why, but I have seen this problem, or phenomenon, mentioned elsewhere.

It appears that the solution is to set the backup target file system to be read only, like this:

# zfs set readonly=on tank/testback

Another possibility is to use the ‘-F’ switch with the ‘zfs receive’ command. I don’t know which is the recommended solution, but I will use the switch for now, as I don’t want to make the file system read only, as we have several incremental backups to perform:

# zfs send -i tank/test@1 tank/test@2 | zfs receive -F tank/testback

Let’s just take a look at the files in the full backup — it should contain the original files referenced from our initial snapshot ‘tank/test@2’ — i.e. just file ‘b’:

# ls -l /tank/testback
total 2
-rw-r--r--   1 root     root          15 May 12 14:50 b
# cat /tank/testback/b
hello world: b

Good, as expected 🙂

Now let’s send all the remaining incremental backups:

# zfs send -i tank/test@2 tank/test@3 | zfs receive -F tank/testback
# zfs send -i tank/test@3 tank/test@4 | zfs receive -F tank/testback

# zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
tank                 766G   598G  29.3K  /tank
tank/test           94.6K   598G  26.6K  /tank/test
tank/test@1         23.3K      -  26.6K  -
tank/test@2         21.3K      -  26.0K  -
tank/test@3         21.3K      -  26.0K  -
tank/test@4             0      -  26.6K  -
tank/testback       93.2K   598G  26.6K  /tank/testback
tank/testback@1     22.0K      -  25.3K  -
tank/testback@2     21.3K      -  26.0K  -
tank/testback@3     21.3K      -  26.0K  -
tank/testback@4         0      -  26.6K  -

Here is the final state of the backup target file system after sending all the incremental backups.

As we would expect, it matches the source file system contents:

# cat /tank/testback/b /tank/testback/c
hello world: b
modified
hello world: c

Restore a backup

Now let’s restore all of our four backup target snapshots into four separate file systems, so we can demonstrate how to recover any or all of the data that we snapshotted and backed up:

# zfs send tank/testback@1 | zfs recv tank/fs1
# zfs send tank/testback@2 | zfs recv tank/fs2
# zfs send tank/testback@3 | zfs recv tank/fs3
# zfs send tank/testback@4 | zfs recv tank/fs4

Let’s look at the file systems:

# zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
tank                 766G   598G  33.3K  /tank
tank/fs1            25.3K   598G  25.3K  /tank/fs1
tank/fs1@1              0      -  25.3K  -
tank/fs2            26.0K   598G  26.0K  /tank/fs2
tank/fs2@2              0      -  26.0K  -
tank/fs3            26.0K   598G  26.0K  /tank/fs3
tank/fs3@3              0      -  26.0K  -
tank/fs4            26.6K   598G  26.6K  /tank/fs4
tank/fs4@4              0      -  26.6K  -
tank/test           94.6K   598G  26.6K  /tank/test
tank/test@1         23.3K      -  26.6K  -
tank/test@2         21.3K      -  26.0K  -
tank/test@3         21.3K      -  26.0K  -
tank/test@4             0      -  26.6K  -
tank/testback       93.2K   598G  26.6K  /tank/testback
tank/testback@1     22.0K      -  25.3K  -
tank/testback@2     21.3K      -  26.0K  -
tank/testback@3     21.3K      -  26.0K  -
tank/testback@4         0      -  26.6K  -

Let’s check ‘tank/fs1’ – it should match the state of the original file system when the ‘tank/test@1’ snapshot was taken:

# ls -l /tank/fs1
total 4
-rw-r--r--   1 root     root          15 May 12 14:50 a
-rw-r--r--   1 root     root          15 May 12 14:50 b
# cat /tank/fs1/a /tank/fs1/b
hello world: a
hello world: b

Perfect, now let’s check ‘tank/fs2’ – it should match the state of the original file system when the ‘tank/test@2’ snapshot was taken:

# ls -l /tank/fs2
total 2
-rw-r--r--   1 root     root          15 May 12 14:50 b
# cat /tank/fs2/b
hello world: b

Perfect, now let’s check ‘tank/fs3’ – it should match the state of the original file system when the ‘tank/test@3’ snapshot was taken:

# ls -l /tank/fs3
total 2
-rw-r--r--   1 root     root          24 May 12 17:35 b
# cat /tank/fs3/b
hello world: b
modified

Perfect, now let’s check ‘tank/fs4’ – it should match the state of the original file system when the ‘tank/test@4’ snapshot was taken:

# ls -l /tank/fs4
total 4
-rw-r--r--   1 root     root          24 May 12 17:35 b
-rw-r--r--   1 root     root          15 May 12 18:58 c
# cat /tank/fs4/b /tank/fs4/c
hello world: b
modified
hello world: c

Great!

Conclusion

Hopefully, you’ve now seen the power of snapshots. In future posts, I will show what else can be done with snapshots.

For more ZFS Home Fileserver articles see here: A Home Fileserver using ZFS. Alternatively, see related articles in the following categories: ZFS, Storage, Fileservers, NAS.