Home Fileserver: ZFS File Systems

Once you’ve built your ZFS home fileserver / NAS, you’ll want to create your storage pool, create your file systems and share them to various devices around the home, such as laptops, PC’s, Macs, media centres etc.

I will go through all the necessary steps from start to finish, so you can see how to create a full, working file system hierarchy that is practical and useful.

Please notify me of any errors you may find in the comments section below.

Example scenario

Here I’ll describe an example scenario that could be reasonably typical.

Your NAS will serve family members and will consist of home directories and media file systems for music, photos and video, as in the following example:

home
    fred
        photo repository
        video repository
        projects
    wilma
media library
    music
    photos
    video

In this scenario, there are two users called Fred and Wilma.

Fred is a photographer and film maker, and so he manages photo and video repositories, and manages various miscellaneous projects too.

Note that Fred’s home directory file system will contain photo & video data which is irreplaceable, whereas the media file systems will contain data that can be replaced, either by re-copying from original CD’s or DVD’s, or re-exporting data from Fred’s home directory photo & video repositories.

I made this distinct separation to make it simple to understand and prevent mistakes: all media file systems are replaceable, and home directory file systems are not.

After a photo shoot, Fred will transfer all his photos from his digital SLR to the photo repository within his home directory. After editing the photos, he may choose to export selected photos to the photos file system within the media file system, using an export preset from his photo management software to make them available to other family members.

This separation enables different file system properties to be specified to reduce chances of loss for irreplaceable data, as we will see later.

I will also assume you wish to allow video and audio media clients around the house to be able to stream media from the media file systems within the storage pool, but this will be covered in a separate post for brevity here.

List your drives

ZFS allows you to create one big storage pool from all the drives you have in your system. The first step is to identify the drives:

# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0d0 [DEFAULT cyl 20020 alt 2 hd 255 sec 63]
          /pci@0,0/pci-ide@4/ide@0/cmdk@0,0
       1. c1t0d0 [ATA-WDC WD7500AAKS-0-4G30-698.64GB]
          /pci@0,0/pci1043,8239@5/disk@0,0
       2. c1t1d0 [ATA-WDC WD7500AAKS-0-4G30-698.64GB]
          /pci@0,0/pci1043,8239@5/disk@1,0
       3. c2t0d0 [ATA-WDC WD7500AAKS-0-4G30-698.64GB]
          /pci@0,0/pci1043,8239@5,1/disk@0,0
       4. c2t1d0 [ATA-WDC WD7500AAKS-0-4G30-698.64GB]
          /pci@0,0/pci1043,8239@5,1/disk@1,0
       5. c4t0d0 [ATA-WDC WD7500AAKS-0-4G30-698.64GB]
          /pci@0,0/pci1043,8239@5,2/disk@0,0
       6. c4t1d0 [ATA-WDC WD7500AAKS-0-4G30-698.64GB]
          /pci@0,0/pci1043,8239@5,2/disk@1,0
Specify disk (enter its number): ^C
# 

Here, ignoring the first drive, which is the IDE boot drive (drive number 0, id c0d0, marked as pci-ide), six SATA drives are shown (marked pci), whose ids range from c1t0d0 through c4t1d0.

I’ll use all of them to create a storage pool (zpool) in the following section.

Create the storage pool

Here you need to decide the storage pool vdev configuration. This determines the level of redundancy your storage pool will have.

Due to cost of drives, most people will choose RAID-Z1, and those who don’t want to take any chances will choose RAID-Z2. See the post Home Fileserver: A Year in ZFS for a more complete explanation — see the RAID-Z1 and RAID-Z2 headings there.

For a RAID-Z1 single parity setup, which can survive one drive failure like RAID 5, type:

# zpool create tank raidz1 c1t0d0 c1t1d0 c2t0d0 c2t1d0 c4t0d0 c4t1d0

For a RAID-Z2 double parity setup, which can survive two drive failures like RAID 6, type:

# zpool create tank raidz2 c1t0d0 c1t1d0 c2t0d0 c2t1d0 c4t0d0 c4t1d0

If you created a RAID-Z2 vdev for your storage pool, you should see something like this:

# zpool status tank
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz2    ONLINE       0     0     0
            c1t0d0  ONLINE       0     0     0
            c1t1d0  ONLINE       0     0     0
            c2t0d0  ONLINE       0     0     0
            c2t1d0  ONLINE       0     0     0
            c4t0d0  ONLINE       0     0     0
            c4t1d0  ONLINE       0     0     0

errors: No known data errors
# 

Now let’s check the new storage pool:

# zpool list
NAME   SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
tank  4.06T   189K  4.06T     0%  ONLINE  -
#

So, we have 4TB of double parity storage available. But that’s before parity space is deducted.

So, in fact, we ‘only’ have 2.66TB of space for data, as the rest is for parity:

# zfs list  
NAME   USED  AVAIL  REFER  MOUNTPOINT
tank   117K  2.66T  36.0K  /tank
# 

Create the groups

We’ll now create two types of group: (1) groups for the users and (2) a group for media library access.

Create the media group for media pool access

# groupadd media

Create the primary groups for the users

Create the primary group that will be used by Fred. In this example, the fred group wants to use the id 501:

# groupadd -g 501 fred

Create the primary group that will be used by Wilma. In this example, we don’t care which id the OS gives us:

# groupadd wilma

Create the users

We’ll now create the user accounts for Fred and Wilma.

Create user for Fred

Fred wants to be able to log in to the Solaris NAS box with the bash shell.

Create the user ‘fred’, make his primary group ‘fred’, secondary group ‘media’, his shell ‘bash’, and set his password:

# useradd -g fred -u 501 -s /bin/bash -d /export/home/fred -c fred-flintstone -m fred
# usermod -G media fred
# passwd fred

Explanation of useradd parameters used above:
-g fred: adds user to primary group ‘fred’ (which has groupid 501)
-u 501: creates the userid 501 for this user
-s /bin/bash: assigns the default shell to be bash for this user
-d /export/home/fred: defines the home directory
-c fred-flintstone: creates the comments/notes to describe this user as required
-m: creates the home directory for the user
fred: this is the login name for this user

Create user for Wilma

Wilma does not want to log in to the Solaris NAS box, so she can have a simple account setup.

Create the user ‘wilma’, make her primary group ‘wilma’, secondary group ‘media’, and set her password:

# useradd -g wilma wilma
# usermod -G media wilma
# passwd wilma

Create the home directory file systems

Now we’ll create home directory file systems for two users: Fred and Wilma.

Fred is a Mac user, and Wilma is a Windows user.

Fred needs the photo & video repository and projects file systems to be accessible from his Mac.

Wilma needs her home directory file system to be accessible from her Windows PC.

Please note that I have used simplistic NFSv4 ACLs to make things simple here — i.e. all permissions allowed or denied at the owner, group and everyone levels. If you want to know more about this subject, take a look at the ACL chapter in the ZFS Admin Guide.

With the simplistic NFSv4 ACL property lists that I have used here, this will enable files and directories to be created or copied into these file systems, and these ACLs will be propagated via inheritance. See the brief ACL property list check section below, for user fred.

I don’t pretend to have expert understanding of this non-trivial subject yet, but these settings seem to have worked for me so far.

One non-desirable aspect of using this ultra-simplistic ACL is that non-executable files will still have the executable permission set. I probably should have specified ‘passthrough-x’ as the value for the ‘aclinherit’ property of these file systems. If someone more knowledgeable could comment on this point, that would be good.

Create the home directory file system

# zfs create tank/home

Create Fred’s home directory file systems

Create Fred’s home directory and child file systems for photo and video repositories, plus projects:

# zfs create tank/home/fred
# zfs create tank/home/fred/photo
# zfs create tank/home/fred/video
# zfs create tank/home/fred/projects

Note that I didn’t bother here making Fred’s home directory use the default Solaris location, which would be /export/home/fred, although this would be easy to do by issuing ‘zfs set mountpoint=/export/home/fred tank/home/fred’. Or you could have created the home directory at ‘/tank/home/fred’ and then set this when Fred’s user account was created.

Setup Fred’s photo repository file system properties

# zfs set aclinherit=passthrough tank/home/fred/photo
# zfs set aclmode=passthrough tank/home/fred/photo
# chmod A=\
owner@:rwxpdDaARWcCos:fd-----:allow,\
group@:rwxpdDaARWcCos:fd-----:allow,\
everyone@:rwxpdDaARWcCos:fd-----:deny \
/tank/home/fred/photo
# chown fred:fred /tank/home/fred/photo
# zfs set sharesmb=name=photo tank/home/fred/photo

This sets up inheriting of ACLs for files and directories using the ACL property list specified for owner (full permissions) and group (full permissions), and lastly for everyone (no permissions).
Also, the file system will be shared as a CIFS share using the name ‘photo’.

Setup Fred’s video repository file system properties

# zfs set aclinherit=passthrough tank/home/fred/video
# zfs set aclmode=passthrough tank/home/fred/video
# chmod A=\
owner@:rwxpdDaARWcCos:fd-----:allow,\
group@:rwxpdDaARWcCos:fd-----:allow,\
everyone@:rwxpdDaARWcCos:fd-----:deny \
/tank/home/fred/video
# chown fred:fred /tank/home/fred/video
# zfs set sharesmb=name=video tank/home/fred/video

This sets up inheriting of ACLs for files and directories using the ACL property list specified for owner (full permissions) and group (full permissions), and lastly for everyone (no permissions).
Also, the file system will be shared as a CIFS share using the name ‘video’.

Setup Fred’s projects file system properties

# zfs set aclinherit=passthrough tank/home/fred/projects
# zfs set aclmode=passthrough tank/home/fred/projects
# chmod A=\
owner@:rwxpdDaARWcCos:fd-----:allow,\
group@:rwxpdDaARWcCos:fd-----:allow,\
everyone@:rwxpdDaARWcCos:fd-----:deny \
/tank/home/fred/projects
# chown fred:fred /tank/home/fred/projects
# zfs set sharesmb=name=projects tank/home/fred/projects

This sets up inheriting of ACLs for files and directories using the ACL property list specified for owner (full permissions) and group (full permissions), and lastly for everyone (no permissions).
Also, the file system will be shared as a CIFS share using the name ‘projects’.

NFSv4 ACL property list check

As mentioned above, I have used very simplistic NFSv4 ACL property lists here for these file systems, and specified them so that these ACLs will be propagated into new files and directories that are created or copied into these file systems. To briefly show you what I mean, let’s create a couple of test files and directories to illustrate this:

# su - fred
-bash-3.2$ id
uid=501(fred) gid=501(fred)
-bash-3.2$ groups fred
fred media
-bash-3.2$
-bash-3.2$ cd /tank/home/fred/projects
-bash-3.2$
-bash-3.2$ touch testfile1
-bash-3.2$
-bash-3.2$ ls -l testfile1
-rwxrwx---+  1 fred    fred          0 May 10 21:30 testfile1
-bash-3.2$

Using ‘ls -l’ to view the standard Unix-style permissions, we see that ‘user’ and ‘group’ have read/write/execute permissions, and ‘other’ has no permissions. Note the ‘+’ (plus) character following the permissions list, which indicates that a non-trivial ACL is associated with the file.

Now if we use the ‘-V’ option with ls, we can see the ACL in compact format, where each letter represents a different permission:

-bash-3.2$ ls -V testfile1
-rwxrwx---+  1 fred    fred          0 May 10 21:30 testfile1
                 owner@:rwxpdDaARWcCos:------I:allow
                 group@:rwxpdDaARWcCos:------I:allow
              everyone@:rwxpdDaARWcCos:------I:deny
-bash-3.2$ 

These ACL entries were set when the file was created and have been propagated from the containing directory due to the ACL inheritance specified on the /tank/home/fred/projects directory above — see the ‘chmod’ statement above, and look for where ‘fd’ is specified, ‘f’ indicating file inheritance, and ‘d’ indicating directory inheritance.

Compact format ACL entries are perhaps easier to view than in verbose format, due to the fact that they align when viewing lists of these entries, like above.

However, it’s pretty hard to determine the meaning behind compact format ACL entries, so sometimes you may wish instead to view verbose format ACL entries, as in the following:

-bash-3.2$ ls -v testfile1
-rwxrwx---+  1 fred    fred          0 May 10 21:30 testfile1
     0:owner@:read_data/write_data/append_data/read_xattr/write_xattr/execute
         /delete_child/read_attributes/write_attributes/delete/read_acl
         /write_acl/write_owner/synchronize:inherited:allow
     1:group@:read_data/write_data/append_data/read_xattr/write_xattr/execute
         /delete_child/read_attributes/write_attributes/delete/read_acl
         /write_acl/write_owner/synchronize:inherited:allow
     2:everyone@:read_data/write_data/append_data/read_xattr/write_xattr
         /execute/delete_child/read_attributes/write_attributes/delete
         /read_acl/write_acl/write_owner/synchronize:inherited:deny
-bash-3.2$ 

Now we’ll make a test directory and create a file within it to observe the propagation of inherited ACL entries as nested files and directories are created within the file system:

-bash-3.2$ mkdir testdir1
-bash-3.2$              
-bash-3.2$ ls -dV testdir1
drwxrwx---+  2 fred    fred          2 May 10 21:39 testdir1
                 owner@:rwxpdDaARWcCos:fdi---I:allow
                 owner@:rwxpdDaARWcCos:------I:allow
                 group@:rwxpdDaARWcCos:fdi---I:allow
                 group@:rwxpdDaARWcCos:------I:allow
              everyone@:rwxpdDaARWcCos:fdi---I:deny
              everyone@:rwxpdDaARWcCos:------I:deny
-bash-3.2$ 
-bash-3.2$ cd testdir1
-bash-3.2$ touch testfile2
-bash-3.2$ ls -V testfile2
-rwxrwx---+  1 fred    fred          0 May 10 21:40 testfile2
                 owner@:rwxpdDaARWcCos:------I:allow
                 group@:rwxpdDaARWcCos:------I:allow
              everyone@:rwxpdDaARWcCos:------I:deny
-bash-3.2$ 

NFSv4 ACLs are a fairly complex area, and I need to research the subject in more detail to get the most benefit from their power. Later… 🙂

Create Wilma’s home directory file system

# zfs create -o casesensitivity=mixed tank/home/wilma
# chmod A=everyone@:full_set:fd:allow tank/home/wilma
# zfs set sharesmb=name=home_wilma tank/home/wilma

This sets up inheriting of ACLs for files and directories using the ACL property list specified for everyone (full permissions).

Also, the file system will be shared as a CIFS share using the name ‘home_wilma’.

Permissions can now be set properly from an administrator account for Wilma on her Windows PC.
The wilma account must exist on the Windows PC. The PC is assumed to belong to a workgroup called WORKGROUP.

Domain accounts will probably require Active Directory administration, which is outside the scope of this post, but see the post on Active Directory for more details.

For reliable operation from Windows machines, check out the details on idmapping rules to map Windows user accounts to Solaris accounts. See here for more info: Solaris CIFS Administration Guide. See the Identity Mapping Administration chapter.

At some point, once I’ve delved into this subject in more detail, I will write another post. For now, the above should get you going for simple home setups.

Create the media file systems for music, photos and video

Now we’ll create the media file systems for music, photos and video.

Create the top-level media file system

# zfs create tank/media
# zfs set aclinherit=passthrough tank/media
# zfs set aclmode=passthrough tank/media

Create the music media file system and properties

# zfs create tank/media/music
# chmod A=\
owner@:rwxpdDaARWcCos:fd-----:allow,\
group@:rwxpdDaARWcCos:fd-----:allow,\
everyone@:rwxpdDaARWcCos:fd-----:deny \
/tank/media/music
# chown media:media /tank/media/music
# zfs set sharesmb=name=media_music tank/media/music

File system created in the usual way, owner and group set to media:media, and made a CIFS share named ‘media_music’.

Create the photo media file system and properties

# zfs create tank/media/photos
# chmod A=\
owner@:rwxpdDaARWcCos:fd-----:allow,\
group@:rwxpdDaARWcCos:fd-----:allow,\
everyone@:rwxpdDaARWcCos:fd-----:deny \
/tank/media/photos
# chown media:media /tank/media/photos
# zfs set sharesmb=name=media_photos tank/media/photos

File system created in the usual way, owner and group set to media:media, and made a CIFS share named ‘media_photos’.

Create the video media file system and properties

# zfs create tank/media/video
# chmod A=\
owner@:rwxpdDaARWcCos:fd-----:allow,\
group@:rwxpdDaARWcCos:fd-----:allow,\
everyone@:rwxpdDaARWcCos:fd-----:deny \
/tank/media/video
# chown media:media /tank/media/video
# zfs set sharesmb=name=media_video tank/media/video

File system created in the usual way, owner and group set to media:media, and made a CIFS share named ‘media_video’.

Setup CIFS sharing mechanism

First, as we will use CIFS shares to make the file systems available to Mac and Windows machines, do the following:

# smbadm join -w WORKGROUP
Successfully joined workgroup 'WORKGROUP'
#

Edit the /etc/pam.conf file to support creation of an encrypted version of the user’s password for CIFS.
Add the following line to the end of the file:

# vi /etc/pam.conf

other password required pam_smb_passwd.so.1 nowarn

Specify the password for existing local users.

The Solaris CIFS service cannot use the Solaris encrypted version of the local user’s password for authentication. Therefore, you must generate an encrypted version of the local user’s password for the Solaris CIFS service to use. When the SMB PAM module is installed, the passwd command generates such an encrypted version of the password.

# passwd fred
# passwd wilma
# passwd media

Restart SMB server and check shares

Now that your file systems are created and the shares have been specified, you’ll need to enable/restart the CIFS server on Solaris:

# svcadm enable -r smb/server
# svcadm restart network/smb/server:default
# svcs | grep smb
online         15:49:20 svc:/network/smb/server:default

Also, let’s check the shares are defined correctly:

# sharemgr show -vp
default nfs=()
zfs
    zfs/tank/home/fred/photo smb=()
          photo=/tank/home/fred/photo
    zfs/tank/home/fred/projects smb=()
          projects=/tank/home/fred/projects
    zfs/tank/home/fred/video smb=()
          video=/tank/home/fred/video
    zfs/tank/home/wilma smb=()
          home_wilma=/tank/home/wilma
    zfs/tank/media/music smb=()
          media_music=/tank/media/music
    zfs/tank/media/photos smb=()
          media_photos=/tank/media/photos
    zfs/tank/media/video smb=()
          media_video=/tank/media/video
# 

Conclusion

When I get some more time, I’ll post details of how to access these shares from Macintosh and Windows systems. Hopefully, very soon… 🙂

In the meantime, you can take a look here for details written earlier on how to access these shares from a Macintosh computer. For accessing the shares from a Windows system, from the file manager, try the ‘Map network drive’ menuitem, giving the same share names as used above.

For more ZFS Home Fileserver articles see here: A Home Fileserver using ZFS. Alternatively, see related articles in the following categories: ZFS, Storage, Fileservers, NAS.