FreeBSD:ZFS Install

The following is how to install FreeBSD on ZFS root using GPT. This will offer redundancy and scaling of performance (with additional drives) for you to base your system off of while maintaining efficiency of disk usage. This guide is mainly suited for workstations, home file servers, and media center PCs. It is setup for use with FreeBSD 9+ and was directly tested with 9.0-RELEASE version of FreeBSD which includes the new BSDInstall.

Introduction to FreeBSD

I am going to make the assumption that you have at least a basic knowledge of the command line in FreeBSD. If you are new to FreeBSD all hope is not lost. The FreeBSD Handbook is the primary source of documentation produced by the FreeBSD Documentation Project. Aside from this recently there has also been another great article written by Paul Venezia at InfoWorld entitled Why aren't you using FreeBSD?. If you are familiar with linux but new to FreeBSD there is the FreeBSD Quickstart Guide for Linux Users. Last but not least if you are coming over from Windows you can take a look at FreeBSD is NOT Windows.

ZFS became part of FreeBSD on 6th April 2007, while it stayed in the experimental phase until the 15th of September 2009 when they removed the experimental tag and deemed it ready for production use. Then on the 23th of November 2009 zfsloader was merged into -STABLE making ZFS as boot option easier than ever.

geli is an easy to use block device encryption to encrypt disks on a lower level than the filesystem. We will be taking advantage of geli for our swap and temporary directories.

ZFS, geli and other newer features that have been recently added into FreeBSD are not available through the generic sysinstall(8) that is included on all of the default installation media. To remedy this, I am going to show you how to install FreeBSD manually from the Fixit livefs command line.

Obtaining FreeBSD

The first thing you are going to need is a copy of the most current *-dvd1.iso or *-memstick.img FreeBSD image (the other images do not contain the livefs that is needed). FreeBSD may be obtained from torrent, an FTP mirror near you, or from FreeBSD Daily Snapshots.

Booting the Installation Media

At this point you should boot from your installation media. For use in creation of this tutorial the following test image was used inside of VirtualBox.

FreeBSD-10.0-HEAD-20120120-JPSNAP-amd64-amd64-release.iso

When you first boot up your chosen media you will see the normal boot menu, feel free to wait out the allotted time or press enter. After the system is done booting up you will be presented with the 'Welcome' window inside the new BSDInstaller.

Configuring the Installation Environment

Select < Shell > from the options and press enter. You should now be presented with the shell. First thing we need to do is switch /tmp from read-only to writable (used later to store the zpool.cache file) and then load the required kernel modules via kldload(8).

umount /dev/md1

Ignore any errors you may or may not see from executing this command, the /tmp directory will still be unmounted. Remount the tmp directory and load the needed kernel modules.

mdmfs -s 512M md1 /tmp

kldload geom_mirror

kldload opensolaris

kldload zfs

Remote Installation Setup (Optional)

This section takes two things for granted; (a) that you have a personal set of SSH keys, and (b) that they are located on another server on the network, one that is already running an SCP capable SSH server. If you do not have one of these requirements feel free to skip this section or pause to set this up on your own. I feel it is easier to install via SSH due to the large amount of actual typing you are required to do, whereas with SSH you can merely copy/paste into the SSH window.

Connect to the network via DHCP with dhclient(8) (for network interface name use ifconfig(8)), then create our SSH host keys via ssh-keygen(1).

dhclient em0

cd /tmp

ssh-keygen -f /tmp/hostkey -t rsa1 -b 4096 -N

ssh-keygen -f /tmp/hostkey_dsa -t dsa -N

Next we will need to download your personal SSH keys from the network. Replace the username, IP address, and folder/file with the required information. After that is done go ahead and start up the SSH daemon with sshd(8).

scp username@192.168.1.100:~/.ssh/authorized_keys .

/usr/sbin/sshd -o PermitRootLogin=yes -o HostKey=/tmp/hostkey -o HostDSAKey=/tmp/hostkey_dsa -o AuthorizedKeysFile=/tmp/authorized_keys

Connect to the IP address that was assigned to the installation computer using ssh(1) with your personal SSH keys.

ssh -l root 192.168.1.200

Disk Setup

Moving on to the disk setup, first locate all of your disks, we can use dmesg(8) and grep(1) for this. The system I am using for this guide has two disks, your results may vary.

dmesg | grep -e "^ad" -e "^da" -e "^hd"

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: <VBOX HARDDISK 1.0> ATA-6 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 512000MB (1048576000 512 byte sectors: 16H 63S/T 16383C) ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: <VBOX HARDDISK 1.0> ATA-6 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 512000MB (1048576000 512 byte sectors: 16H 63S/T 163

For partition table setup we will be using GUID Partition Table (GPT) via gpart(8). Let's first pull up a list of the current partitions.

gpart show

If there are existing partitions on any disk remove them with the delete command, then destroy the disks partitioning scheme.

gpart delete -i 1 ada0

gpart destroy ada0

Create a new partitioning scheme on each disk.

gpart create -s gpt ada0

gpart create -s gpt ada1

Now we need to create all of the disk partitions, we will be creating four partitions on each disk. The first partitions will contain the boot loader, the second partitions are the swap partitions and finally the third partitions are for the zpool (ZFS Pool). Adjust the size of the zpool partition to fit your disk size needs.

gpart add -s 64K -t freebsd-boot ada0

gpart add -s 4G -t freebsd-swap -l swap0 ada0

gpart add -s 495G -t freebsd-zfs -l zroot0 ada0

Now that we have the base partition setup, replicate this onto the other disk.

gpart add -s 64K -t freebsd-boot ada1

gpart add -s 4G -t freebsd-swap -l swap1 ada1

gpart add -s 495G -t freebsd-zfs -l zroot1 ada1

Install the protected MBR boot code and gptzfsboot loader onto each of the disks. The gptzfsboot loader will give you the ability to load zfsloader from a zpool.

gpart bootcode -b /boot/pmbr -i 1 -p /boot/gptzfsboot ada0

gpart bootcode -b /boot/pmbr -i 1 -p /boot/gptzfsboot ada1

Verify the partition table to make sure all of the disks match.

gpart show

=> 34 1048575933 ada0 GPT (500G) 34 128 1 freebsd-boot (64K) 162 8388608 2 freebsd-swap (4.0G) 8388770 1038090240 3 freebsd-zfs (495G) 1046479010 2096957 - free - (1.0G) => 34 1048575933 ada1 GPT (500G) 34 128 1 freebsd-boot (64K) 162 8388608 2 freebsd-swap (4.0G) 8388770 1038090240 3 freebsd-zfs (495G) 1046479010 2096957 - free - (1.0G)

Take note that I did not use the entire disk, the reason behind this is that when using RAID and replacing problematic disks more than likely your new disk will have a slightly different disk geometry and as such might limit your ability to use it as a replacement disk if you cannot fit all the necessary partitions on the disk.