pascal jungblut posts

Migrate a FreeBSD server with a ZFS root filesystem

ZFS is one of the best parts of FreeBSD and it is widely used for installations of all size. Since a few years it’s possible to have the complete root fileystem on ZFS. The snapshotting capabilities and the builtin zfs send and zfs recv commands make it easy to transfer a server to another system (given a compatible hardware).

This short guide will show you how to migrate a running FreeBSD installation to another server. It expects that you are somewhat familiar with the shell, FreeBSD and ZFS. If you are having trouble: the man pages for zfs(8) and zpool(8) are excellent.

Preparing the target

First, boot the FreeBSD install cd image on the target machine. Choose shell and start partitioning the disk:

# create a new partitioning scheme
gpart create -s gpt ada0
# add a boot partition
gpart add -b 34 -s 94 -t freebsd-boot ada0
# add the main data partition that will hold the ZFS pool
gpart add -t freebsd-zfs ada0

Repeat the above steps for each hard disk that should be part of the pool. Replace ada0 with your hard disk’s identifier. Now let’s create the pool.

zpool create tank mirror /dev/ada0p2 /dev/ada1p2

This command will create a mirrored pool named tank on two hard disks. Adapt it to your needs.

Sending a snapshot

Now that the pool is available, we can start to transfer it from the source machine. Log into the source machine, create a new snapshot and send it to the target machine:

setenv SNAPSHOT "move-`date +%y-%m-%d`"
# make a new snapshot
zfs snapshot -r tank@$SNAPSHOT
# send the snapshot to the new server
zfs send -vR tank@$SNAPSHOT | ssh <target> zfs recv -F tank

ZFS will now send all snapshots of all filesystems contained in tank to the new machine and print the statistics while the process is running. You can check with zfs iostat 10 on the target to verify that the data is being written.

Bonus: keeping up with changes

Sending the data to the new machine can take some time. If your machine is still actively being used while you send the snapshots, the changes will not make it to the new server. However: it’s easy to make another snapshot and just send a diff. The diff will, of course, be much smaller than the whole snapshot.

zfs snapshot -r tank@$SNAPSHOT-diff1
zfs send -vRi tank@$SNAPSHOT tank@$SNAPSHOT-diff1 | ssh <target> zfs recv -F tank

It may be necessary to repeat the step a few times, depending on the amount of changes and your connection.

Cleaning up

You probably have to make some adaptions on the target before booting into FreeBSD. For me that’s mostly the network settings as my servers have an assigned IP per machine. To do that, mount the freshly copied root filesystem into /mnt and make the changes:

# mount the root filesystem
mount -t zfs tank/root /mnt
# add the bootcode to the MBR
# repeat for all disks
gpart bootcode -b /mnt/boot/pmbr -p /mnt/boot/gptzfsboot -i 1 ada0
# do what you need to do, e.g:
# vi /mnt/etc/rc.conf

After that, clean everything up and reboot.

# umount and export the pool
zfs umount -a
# also set the bootfs, it was not copied with zfs send
zpool set bootfs=tank/root tank
zfs export tank
reboot

An exact copy of the source server should boot.