Home Fileserver: ZFS setup
The next step in setting up your own ZFS home fileserver is to set up your ZFS storage pool and file systems and then share them with other machines. The ZFS commands should work from any operating system where ZFS is available. I have used two machines in this example: a machine running Sun Solaris for the fileserver, and a Macintosh client machine.
Choose your operating system
The first step is to choose which operating system you will install on your machine. Personally, I can recommend Sun Solaris as I have it running well here and it is the original operating system that ZFS has been designed to run on. I believe it runs on FreeBSD 7.0, Linux under FUSE, and Mac OS X with the relevant download from developer.apple.com or here. If you are able to choose, and wish to have the most stable and reliable version of ZFS, I would personally recommend that you choose Sun Solaris as your operating system.
If you choose to install Sun Solaris, then the next hurdle is to find out which version. When I was looking to install Solaris I had to choose among the following:
- Solaris: version 10 (current), 11, 12 etc is a major release of Solaris that occurs once every year or two, and is solid and heavily tested.
- Solaris Express Developer Edition (SXDE): is a release that occurs once every 2 months or so, and is quite well tested and stable.
- Solaris Express Community Edition (SXCE): is a release that occurs once every week or two, has some testing and may or may not be stable in all areas.
- OpenSolaris Developer Preview: is a preview of a next-generation development of Solaris, and is part of a project called Indiana. It includes new technologies like APT for packaging, first seen in Debian Linux, plus many other new features which you can look up if of interest.
Solaris Express Developer Edition and Solaris Express Community Edition are part of the project called Nevada.
My above comments on each version of Solaris are a brief, but hopefully correct, understanding. Even some Sun insiders seem to think having all these choices is a bit confusing.
Anyway, for running Solaris and ZFS on a home fileserver, my opinion is that your best bet is to choose from the SXDE or SXCE editions.
For quicker bug fixes and new features, I have chosen to use SXCE.
Getting and installing Solaris
To get SXCE, go to http://www.opensolaris.org, click on the Download icon at the top right of the page, then select the DVD link under the text ‘Solaris Express CE’.
You’ll have to register, but it’s free. Then you can download the Solaris image.
You can then burn the ISO file using whatever DVD burning software you have — e.g. DiskUtility for the Mac, or Nero for Windows.
Then you can install Solaris by booting the burned DVD. Installation is simple to do, but fairly slow. After selecting your region and language etc and letting the installation get under way, it’s a good time to go and get a coffee and something to eat 
During installation or after booting the new Solaris installation, you can create your user account. You may wish to use the same user and group ids as you use on your other UNIX box, e.g. your Mac. This will simplify permissions hassles later if you are copying over data from your other machine.
Configuration
Once you have a system where ZFS is available, you can get to work setting up your ZFS storage pool.
You’ll need to decide what kind of setup you want (redundancy or no redundancy). Also which disks you will use. I will assume here that you have multiple disks of the same size and that you wish to setup a large storage pool with built-in redundancy. For redundancy, I will assume you want single-parity. Then our choice will be simple. We’ll setup a RAIDZ array, which is kind of equivalent to the old RAID level 5 setup, but with extra ZFS features not available with RAID level 5.
Once your disks are connected within the case, you need to get the ids of your disks, because you’ll need to specify these disk ids when creating the storage pool.
In Solaris, you can type, as root user:
# format
This will give the following output:
# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0d0 (DEFAULT cyl 20007 alt 2 hd 255 sec 63)
/pci@0,0/pci-ide@4/ide@0/cmdk@0,0
1. c1t0d0 (ATA-WDC WD7500AAKS-0-4G30-698.64GB)
/pci@0,0/pci1043,8239@5/disk@0,0
2. c1t1d0 (ATA-WDC WD7500AAKS-0-4G30-698.64GB)
/pci@0,0/pci1043,8239@5/disk@1,0
3. c2t0d0 (ATA-WDC WD7500AAKS-0-4G30-698.64GB)
/pci@0,0/pci1043,8239@5,1/disk@0,0
Specify disk (enter its number): ^C
#
Now hit CTRL-C to break out of it.
I have installed Solaris on disk 0, and we will now use the 750GB SATA disks with numbered 1 to 3 in this list. They are 750GB disks, but only about 692GB are actually usable thanks to the marketing con used by the disk industry to make the disk sizes appear bigger than they actually are.
Now we’ll issue the ZFS command to create a RAIDZ array of these three 750GB drives, which should give around 1.4TB for data (2 x 692GB), and the other 692GB is used by the RAIDZ array for parity data. This parity data gives the array the ability to (1) remain operational in the event that one of the three hard drives fails, and (2) seal-heal any ‘latent defects’ (aka silent errors or bit rot) so the space is not being wasted
We’re going to call this data storage pool ‘tank’, as used in all the Sun ZFS demos and examples. Presumably tank refers to a large storage container like a pool:
zpool create tank RAIDZ1 c1t0d0 c1t1d0 c2t0d0
Now let’s check its status:
# zpool status tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c2t0d0 ONLINE 0 0 0
errors: No known data errors
#
You can see that this pool is called ‘tank’, that it is online, that no scrub has been requested, and the configuration for the ‘tank’ pool is a RAIDZ1 (single-parity RAIDZ vdev), and you can see the id of each disk in the vdev (virtual device), and for each disk in the vdev, you can see that there have been no read, write or checksum errors found so far.
If you’re curious about ZFS you can discover a load of useful stuff here and here, amongst other places.
Now let’s see how much space we have (these figures are from an in-use pool, not a newly created one):
# zpool list tank NAME SIZE USED AVAIL CAP HEALTH ALTROOT tank 2.03T 996G 1.06T 47% ONLINE - # # zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 663G 701G 25.3K /tank #
The first command ‘zpool list tank’ gives raw storage capacity data for the storage pool that includes capacity used to store parity data.
The second command ‘zfs list tank’ gives storage capacity data for the file systems created within the storage pool, that excludes capacity used for parity data — i.e. the figures only consider user data.
So we can see that the pool has around 2TB of capacity (including parity data), and that in the file systems created under the ‘/tank’ mountpoint we have used 663GB and have 701GB available. 663GB + 701GB = 1364GB, or around 1.3TB, so this seems about right, considering that an additional 692GB is used for parity data (~1.3TB + ~0.7TB = ~2TB).
So it’s looking good.
Setting up your file systems
Now we’ll move on to exploring what we can do with storage pools. So that I don’t risk messing up my existing storage pool, I’m going to create a new pool which will use a 4GB USB memory stick (thumbdrive for U.S. readers?).
Once the pool is created, all the ZFS commands will be identical to the ones I would use if I was working with standard hard drives. There will be no redundancy with the USB stick as I will only use one of them here, but that doesn’t matter for this example.
First I’ll plug the 4GB stick into the USB slot and then see what its ‘disk’ device id is:
# format -e < /dev/null
Searching for disks...
The device does not support mode page 3 or page 4,
or the reported geometry info is invalid.
WARNING: Disk geometry is based on capacity data.
The current rpm value 0 is invalid, adjusting it to 3600
done
c4t0d0: configured with capacity of 3.84GB
AVAILABLE DISK SELECTIONS:
0. c0d0
/pci@0,0/pci-ide@4/ide@0/cmdk@0,0
1. c1t0d0
/pci@0,0/pci1043,8239@5/disk@0,0
2. c1t1d0
/pci@0,0/pci1043,8239@5/disk@1,0
3. c2t0d0
/pci@0,0/pci1043,8239@5,1/disk@0,0
4. c3t0100001E8C38A43E00002A0047C465C5d0
/scsi_vhci/disk@g0100001e8c38a43e00002a0047c465c5
5. c4t0d0 < -USBFLASHDRIVE-34CE cyl 1965 alt 2 hd 128 sec 32>
/pci@0,0/pci1043,8239@2,1/storage@a/disk@0,0
Specify disk (enter its number):
#
Now we’ll create a ZFS storage pool called ‘test’ that will use the USB stick to store its data. I had to use the ‘-f’ flag here because the USB stick previously had a UFS file system stored on it, and when I inserted it, Solaris mounted it as ‘/media/USB FLASH DRIVE’, so we’re forcing it to ignore errors here, and do what we want anyway:
# zpool create -f test c4t0d0
Check:
# zpool status test
pool: test
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
test ONLINE 0 0 0
c4t0d0 ONLINE 0 0 0
errors: No known data errors
#
# zpool list test
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
test 3.81G 584K 3.81G 0% ONLINE -
#
# zfs list test
NAME USED AVAIL REFER MOUNTPOINT
test 106K 3.75G 18K /test
#
In this case, as there is no redundancy used (i.e. we’re not using RAIDZ or MIRROR), the figures shown in ‘zpool list test’ and ‘zfs list test’ match, more or less.
Now let’s create a file system for a user called simon:
# zfs create test/home # zfs create test/home/simon # # zfs list NAME USED AVAIL REFER MOUNTPOINT test 186K 3.75G 19K /test test/home 57K 3.75G 21K /test/home test/home/simon 18K 3.75G 18K /test/home/simon #
As these were created as root, now let’s change the owner and group ids for the test/home/simon file system to simon (owner id) and simon (group id):
# cd /test/home # ls -l total 3 drwxr-xr-x 2 root root 2 Mar 8 20:18 simon # # chown simon simon # chgrp simon simon # ls -l total 3 drwxr-xr-x 2 simon simon 2 Mar 8 20:18 simon #
Now let’s create a file in /test/home/simon called ‘readme.txt’ and fill it with some text. As I’m still root user, I’ll change the owner and group ids to simon again:
# cd simon # echo 'This is a test.' > readme.txt # ls -l total 2 -rw-r--r-- 1 root root 16 Mar 8 20:24 readme.txt # chown simon readme.txt # chgrp simon readme.txt # ls -l total 2 -rw-r--r-- 1 simon simon 16 Mar 8 20:24 readme.txt #
Making the storage pool accessible from other machines
The next step here is to enable the file system at /test/home/simon to be accessible to another machine. With ZFS we have three possibilities:- sharing with SMB/CIFS (Samba), NFS or as an iSCSI target. Only volumes are shareable as an iSCSI target and our file system is not a volume so we can only share it with CIFS or NFS. For this example, I will share the file system using CIFS:
# zfs set sharesmb=on test/home/simon # zfs get all test/home/simon NAME PROPERTY VALUE SOURCE test/home/simon type filesystem - test/home/simon creation Sat Mar 8 20:18 2008 - test/home/simon used 19.5K - test/home/simon available 3.75G - test/home/simon referenced 19.5K - test/home/simon compressratio 1.00x - test/home/simon mounted yes - test/home/simon quota none default test/home/simon reservation none default test/home/simon recordsize 128K default test/home/simon mountpoint /test/home/simon default test/home/simon sharenfs off default test/home/simon checksum on default test/home/simon compression off default test/home/simon atime on default test/home/simon devices on default test/home/simon exec on default test/home/simon setuid on default test/home/simon readonly off default test/home/simon zoned off default test/home/simon snapdir hidden default test/home/simon aclmode groupmask default test/home/simon aclinherit secure default test/home/simon canmount on default test/home/simon shareiscsi off default test/home/simon xattr on default test/home/simon copies 1 default test/home/simon version 3 - test/home/simon utf8only off - test/home/simon normalization none - test/home/simon casesensitivity sensitive - test/home/simon vscan off default test/home/simon nbmand off default test/home/simon sharesmb on local test/home/simon refquota none default test/home/simon refreservation none default #
Note that the ’sharesmb’ property has the value ‘on’ now, so it is now shared as a CIFS share. We didn’t specify a name to use as a share, so let’s see which default share name ZFS has assigned for us:
# sharemgr show -vp
default smb=() nfs=()
zfs
zfs/test/home/simon smb=()
test_home_simon=/test/home/simon
#
So we can see here that the share name assigned is ‘test_home_simon’. We could easily have specified our own preferred share name when we set the ’sharesmb’ property to ‘on’ earlier, if we had wanted to.
Now let’s ensure that the Solaris SMB service is running, so that this share will be visible from any connected client machines:
# svcadm enable -r smb/server svcadm: svc:/milestone/network depends on svc:/network/physical, which has multiple instances. # svcs | grep smb online 15:49:20 svc:/network/smb/server:default
I used CIFS to share the file system in this example. When I have made previous experiments using NFS, I have noticed disappointing write speeds from the Mac to the Solaris fileserver and, from what I could find out, it seems that there are some issues with NFS shares of ZFS filesystems, which result in fairly slow write speeds. I think the problem related to NFS requiring some kind of acknowledgement during write operations, and this caused slow performance. But don’t quote me on this, as I was unable to satisfy myself that I had found the definitive answer to this problem, and it may just be that the Mac OS X’s NFS implementation is flawed.
Update 30/03/2008: extra steps required for CIFS setup
I noticed some differences between OpenSolaris Nevada build 82 (SXCE) and build 85 (clean install) regarding CIFS shares.
Existing working CIFS shares from build 82 didn’t work after moving to build 85.
In order to try and see why, I looked up CIFS guide here:
http://docs.sun.com/app/docs/doc/820-2429/smbserver?a=view
where it says:
“The Samba and CIFS services cannot be used simultaneously on a single Solaris system. The Samba service must be disabled in order to run the Solaris CIFS service. For more information, see How to Disable the Samba Service.”
So, disable samba:
# svcs | grep samba maintenance 20:35:42 svc:/network/samba:default # svcadm disable svc:/network/samba # svcs | grep samba #
When I try to access the share from the Mac using autofs (smbfs:), I see the following message in /var/adm/messages:
Mar 27 20:38:48 solarisbox smbd[667]: [ID 653746 daemon.notice] SmbLogon[WORKGROUP\simon]: NO_SUCH_USER
So, something changed?
See here:
http://docs.sun.com/app/docs/doc/820-2429/configureworkgroupmodetask?a=view
# smbadm join -w WORKGROUP Successfully joined workgroup 'WORKGROUP' #
Edit the /etc/pam.conf file to support creation of an encrypted version of the user’s password for CIFS.
Add the following line to the end of the file:
# vi /etc/pam.conf other password required pam_smb_passwd.so.1 nowarn
Specify the password for existing local users.
The Solaris CIFS service cannot use the Solaris encrypted version of the local user’s password for authentication. Therefore, you must generate an encrypted version of the local user’s password for the Solaris CIFS service to use. When the SMB PAM module is installed, the passwd command generates such an encrypted version of the password.
# passwd simon
Now it works again, after reinitialising the client’s autofs (on the Mac for me).
Configuring client machine access to the file system share
Now I’m going to configure my Mac so that it can access the CIFS share and read and write to it. As I’m using Mac OS X 10.5 (Leopard), autofs is available to auto mount specified file systems.
We’ll configure the autofs configuration files in /etc to use the share we created on the ZFS fileserver. Perform the following steps as root user on the Mac, or other OS/machine supporting autofs.
First create a mountpoint directory where our share will reside:
sh-3.2# mkdir /shares sh-3.2# cd /etc sh-3.2# ls -l auto* -rw-r--r-- 1 root wheel 67 Oct 10 06:53 auto_home -rw-r--r-- 1 root wheel 236 Feb 24 15:00 auto_master -rw-r--r-- 1 root wheel 164 Oct 10 06:53 auto_master.org -rw-r--r-- 1 root wheel 319 Mar 1 14:49 auto_smb -rw-r--r-- 1 root wheel 89 Feb 19 15:36 auto_zfs -rw-r--r-- 1 root wheel 1755 Feb 24 14:59 autofs.conf -rw-r--r-- 1 root wheel 1759 Oct 10 06:53 autofs.conf.org sh-3.2# sh-3.2# vi auto_master
Now add the following line to the end of the ‘auto_master’ file:
# simon's additions /shares auto_smb -nobrowse
This specifies that all paths for shares specified in the ‘auto_smb’ file will be relative to the /shares directory. Now let’s create the ‘auto_smb’ file to specify the relative mountpoint for accessing the file system we shared from the fileserver:
# vi auto_smb test_home_simon -fstype=smbfs ://simon:password@fileserver_ip_address/test_home_simon
Save the file.
Now be sure to check that the permissions are correct on this ‘auto_smb’ file, or you may leave passwords visible!
Now that autofs has been configured to access the shared file system from the fileserver using CIFS, we can cause the Mac to remount any specified file system shares:
sh-3.2# automount -vc automount: /net updated automount: /home updated automount: /shares mounted sh-3.2#
Looks good.
Within the Solaris file manager UI, you may need to set the attributes within the ‘Permissions’ and ‘Access List’ tabs of the properties for the /test/home/simon directory. After that, you may need to restart the Solaris machine (or probably just restart relevant services), and possibly the client machine to ensure it gets the new properties for the share.
Now let’s see if we can read the file ‘readme.txt’ that we created on the fileserver:
Macintosh:~ simon$ cd /shares Macintosh:shares simon$ ls -l drwx------ 1 simon wheel 16384 Mar 8 20:24 test_home_simon Macintosh:shares simon$ cd test_home_simon Macintosh:test_home_simon simon$ ls -l total 1 -rwx------ 1 simon wheel 16 Mar 8 20:24 readme.txt Macintosh:test_home_simon simon$ Macintosh:test_home_simon simon$ cat readme.txt This is a test. Macintosh:test_home_simon simon$
Voila, it’s worked. We successfully read the file hosted on the fileserver.
A little tip if you’re using the Mac’s Finder application to view your shares graphically is that you may need to restart it, as it seems to give a special ’share’ icon to CIFS & NFS shares, and the icon seems not to be displayed correctly until you restart the Finder. Restart the Finder by holding down the ‘alt’ key and right-clicking on the Finder icon in the Dock. Then click the ‘Relaunch’ menuitem in the popup menu. Perhaps Apple needs to sync the ‘automount -vc’ with a repaint of the Finder app?
Conclusion
That has given you a simple overview of how to create a ZFS storage pool, how to create a file system within the pool, and how to share the file system with another machine across the network using CIFS. Time for a beer to celebrate! 
Further reading
You may find the following links interesting:
- What is ZFS?
- ZFS: Getting started
- ZFS: Should You?
- ZFS Adds Exciting Twist to Mundane World
- ZFS best practices from the Solaris Internals authors
- Solaris ZFS Administration Guide (HTML)
- ZFS Administration Guide (PDF: updated monthly)
- ‘zfs: discuss’ forum
- ZFS community
- Rajeev’s autofs link
- Greenfly.org autofs link
And these inspiring blogs from some great Sun guys, which I learnt a lot from:
- Tim Foster’s ZFS articles including his ZFS Automatic Snapshot SMF Service
- Constantin Gonzalez of CSI:Munich - How to save the world with ZFS and 12 USB sticks fame — you have to watch this video!
- Tim Thomas’ ZFS, CIFS & NFS sharing articles
And last, but certainly not least, the two Sun guys who created the amazing ZFS:
The more you learn about ZFS, the more you appreciate what a true engineering marvel it is, so I have deep respect for Bill and Jeff, and all the other people that helped make ZFS a reality — congratulations to you all!
And I applaud Sun for encouraging their staff to create their own blogs, as it helps spread the good word about their products and projects, and it is done in a personal style that shows the blog authors’ enthusiasm for the subject.
And having started to know a bit about Solaris via ZFS, I am beginning to see what a great operating system it really is.
Now I’m off to look for a good hosting service that gives me a full root access Solaris account so I can snapshot my running system regularly and zfs send / recv snapshots of it to another geographical location for safety. I’ve seen Joyent.com, and if anyone knows of others, feel free to comment below.
For more ZFS Home Fileserver articles see here: A Home Fileserver using ZFS. Alternatively, see related articles in the following categories: ZFS, Storage, Fileservers.
Popularity: 70% [?]
Note that on Leopard you can also use the Directory Utility (Applications > Utilities) to set the automounts graphically.
Cheers Simon, this is great stuff! I’ve used ZFS a fair amount on Solaris & Nevada, but never played with CIFS shares from ZFS or autofs. Something else to add to my list of things to try.
Cheers z0mbix
Do you have any performance issues if trying to watch .wmv, .mpg,… which are on the ZFS server on the Mac? I do have a lot…
The issue with NFS is that a client requesting a sync ensures that the sync is committed to the server before returning (i.e. it’s semantically correct). Samba doesn’t, so you think you’ve written it successfully but it may fail. It’s more to do with how the client program is written; using ‘tar’ to untarball a lot of small files over NFS may be slower than SMB owing to this issue.
That doesn’t mean that for single file writes (the majority of accesses) NFS is any slower, however. Google for NFS over ZFS to find out more.
I’d be more interested if you could use NFS4 to mount the shares, as they’re supposed to maintain the extended attributes natively. At the moment, a Mac client mounting an NFS share and generating extended attributes will translate them to AppleDouble files, rather than use the remote system’s extended attribute support. NFS4 is supposed to provide that, but the NFS4 stuff is even more alpha than ZFS is on Mac OS X. Maybe when 10.6 comes out …
Thanks for the compliments z0mbix !
@Patrick:
No, I didn’t see any performance problems yet with viewing .mpg or .avi video file types yet from the Mac over the network when using a gigabit switch. Have you looked in the Network tab of the Mac’s ‘Activity Monitor’ app whilst viewing an .mpg file? How much RAM do you have on the fileserver? Assuming you have a fast network (not wireless), this sounds like it could be a resource problem on the fileserver — limited RAM, or some other problem perhaps?
BTW, thanks for quoting a reference to my pages from your page at: http://schlaepfer.nine.ch/twiki/bin/view/Schlaepfer/SelfMadeNas
@Alex Blewitt:
Thanks for the info. Yes, I think I also read somewhere about NFS waiting for the ‘data written OK’ ack before continuing. When I revisit NFS sharing I will take another look into this subject.
When using NFS sharing, I remember also getting very slow writes from the Mac to the ZFS fileserver on the Solaris box when I was using the Mac’s Finder app to drag and drop files to copy them to the fileserver. The speed was (sometimes?) abysmal (3-5MBytes/sec) when copying using the Finder. However, when I used the command line, I got a sustained 25-30MBytes/second, but I forget with which command: rsync or cp. So it seems there’s also some funny business going on with the Mac’s Finder app too.
The trouble with quoting speeds here is that I didn’t keep a copy of all my tests and setups, and my pool configuration was also changing, so to get definitive speed comparisons I would need to do this scientifically, but I don’t have the time right now
Dear Sir:
I like your treatise a lot, however, I would wonder if you would be willing to post some “howtoage” on accessing a ZFS managed and published volume via iSCSI, that would appear to MacOS as either a readable/writable ZFS volume (using the developer ZFS tools for MacOS), or a raw volume to MacOS that could be formatted as HFS+ after being mounted as a raw iSCSI device in need of formatting by MacOS?
Thanks in advance!
I was in a similar boat, setting up a fileserver, on existing hardware. This means I didn’t need to read the HCL!
I tried Nexenta (CD wouldn’t boot) and some Solaris flavours (OS wouldn’t boot) but then trialed FreeBSD, which worked beautifully.
I gotta say this ZFS business is one classy affair. After much meddling an continuous awe, I decided to go for Linux, RAID and LVM. The main reason for this was the adding of disks to increase a pool. I couldn’t believe this was not possible in ZFS when using RAIDZ devices!
With all the whizbang features of ZFS, you’d think this was a minor item to check for on the list.
It pained me to do leave ZFS behind, perhaps we’ll meet up for a fling in the future.
McP.
@Joe: The problem with trying to use the Mac as an iSCSI initiator (client) is that Mac OS X does not include a built-in iSCSI initiator service. There are third party iSCSI initiator offerings available but I didn’t get these to work. Lookup ’small tree iscsi manager’ or ‘globalSAN iSCSI Initiator for OS X’, and there are a couple of others. It seems like the market is still immature in this area.
I got iSCSI working by using 2 Solaris boxes because Solaris supports both the iSCSI target and iSCSI initiator services out of the box. I think if you are persistent you will probably manage to get it working for the Mac using one of the available offerings — but only the globalSAN one was free of charge when I last looked.
You can take a look at my ZFS backups page for info on how to setup Solaris for the iSCSI target side, and you’ll have to consult the docs for whichever Mac iSCSI initiator software you use to get the Mac side working. Good luck and let me know if you get something working!
@McPop: Yes, I understand that Solaris is picky about the hardware due to relative lack of drivers. As you say, I think I heard that FreeBSD has wider hardware support, but I didn’t look much into it.
You are right — it’s a great pity that there’s no support yet for adding or removing disks from a ZFS RAIDZ vdev, although I expect they are working on this, but who knows when it will appear? The way I overcame this issue is to give my pool much more capacity than I currently need. Hopefully, by the time I need more space, they will have added the ‘add disk’ feature to RAIDZ vdevs. Alternatively, even without the ability to add disks to a RAIDZ vdev, you can ‘zfs send/receive’ your pool to another pool, like a large backup pool, and then reconfigure your pool by destroying it and recreating it with the new disk(s), then ‘zfs send/receive’ to your new larger pool. I didn’t try this yet though, but I think that’s possible to do.
Even if I did need more space now, I didn’t do the ‘zfs send/receive’ stuff, I could add new disks to the existing pool by adding a new vdev — i.e. a new RAIDZ or mirror vdev. Another alternative is to create a new pool with the new disks. So, you can see that there are already existing possibilities if you need more storage capacity, even before they add the new functionality to increase vdev capacity.
Oh, and there’s yet another possibility: replace each disk in an existing vdev with a larger capacity drive, one by one. So, for example, you could replace four 500GB drives, one by one, with four 1TB drives to double your pool’s capacity. You can then utilise the old disks for a backup pool or just another storage pool.
Great read.
How could you make a public smb share instead of per-user?
@Ruckus Ron: Thanks! That’s not an area I’ve really explored too much yet, but I believe it would be done using ZFS ACL. Go to opensolaris.org, click on Discussions and look into the storage or ZFS forums. Alternatively, click on Communities and look there under storage or ZFS. I’ll be looking there soon too
And you could look at the ZFS Admin Guide under the ACL section.
What is your transfer rate to and from your solaris machine and what is your CPU usage? I set up a similar system but my processor is a single core Sempron. CPU usage peaks during file transfers and that gives me 20-30 MB/sec to Solaris over SMB and 30-40 MB/sec from Solaris over SMB. If I use compression transfers to Solaris slow down quite a bit.
I did an experiment with NFS and was able to get over 100 MB/sec from Solaris with around 30% CPU usage. Going to Solaris I sometimes get over 30 MB/sec followed shortly by 3 MB/sec so I am going to upgrade the CPU and use SMB until Mac OS X gets a better client or I figure out the work around.
@Joe: I did some experiments with iSCSI and Mac OS X and came to the same conclusion about the iSCSI options. I never was able to have a solution that was totally stable. After the Mac went to sleep or sometime just on its own the Mac would freeze up and need to be hard rebooted. Even if it did work I don’t know if I would be too into iSCSI because you loose a lot of the ZFS benefits like flexible pools. It seems like a virtual drive is going backwards for the benefit of data protection.
@Elvis: Using NFS I was getting quite disappointing speeds of around 25-30 MBytes/sec to and from the fileserver from the Mac Pro. When I switched to CIFS I got around 40+ MBytes/sec. Today I’m getting around 80 MBytes/sec with the same disks at each end, but by using dual Gigabit ethernet at each end and using an IEEE 802.3ad compliant Gigabit switch. See the page on Trunking for more details. However this will cost around $100+ for a switch that can do this. With this setup I’ve reached the limits of the disk read/write speeds on the Mac Pro and to go faster I’d need to get a faster disk like the Samsung Spinpoint F1, or to add an extra disk to the Mac and use a RAID 0 formation. This could potentially yield speeds of upto 150 MBytes/sec, but this is just a wild guess
CPU utilisation was minimal as I’m using a 64 bit dual-core AMD processor. Is the Sempron 64 bit or 32 bit? It seems the simple rule with ZFS is (1) give it loads of RAM (4GB is nice), and (2) use 64 bit processors.
It is a 64bit Sempron. I ordered an Athlon X2 4050e and will try it out and max out the ram. It has been a really fun experience over all. I got interested in the project because I wanted to have a large storage server in the closet for video editing. I will keep playing and see how it goes. I will also try linking the two ethernet ports. Thanks for working most of it out before I even got interested.
@Elvis: I think the Athlon X2 will give you a lot more power than the Sempron — it’s what I use here and I never saw any issue with processor overload. At least 4GB of RAM should give a nice setup. If you are able to, try to use ECC RAM, as it can detect and correct errors before they’re even sent to the disk and it’s only about a 10% price premium. A no brainer as far as I see it. If you’re thinking of video editing then aggregating multiple gigabit ethernet (2+) links will certainly give a boost to your throughput. Glad you’re having fun getting it all working. I had a lot of fun getting it all working here too — sometimes more ‘fun’ than I bargained for
If you get time, let me know how it works out once you get your new gear installed and setup.
If you do link ethernet ports you’ll need an IEEE 802.3ad capable switch. I used the Linksys SRW2008 as it is reasonably priced and allows flexible aggregation possibilities, but its downside is that the web interface requires Internet Explorer, which I used in a virtualised Windows environment on the Mac. With this switch, a limited number of options can be set via telnet access. On hindsight, I should probably have bought an HP ProCurve 1800-8G switch, as it has very good feedback and I think it works with Firefox too, so you don’t need to use Windows.
Then you need to create a Link Aggregation Group (LAG) on the switch for the ethernet ports you wish to aggregate into a single fast link. E.g. On an 8-port switch, choose ports 1 and 2 for your fileserver and create LAG #1 for this and tell the switch that LAG #1 will use ports 1 and 2. Then select LACP (Link Aggregation Control Protocol) for the LAG. You’ll need to repeat this for the other machine where you’ll do your video editing, but this time create LAG #2 for this machine and use ports 3 and 4.
If it’s a Mac, then in the System Preferences Network Panel, click on the gear icon under the network interfaces and select ‘Manage Virtual Interfaces’. There you’ll select ‘Ethernet 1′ and ‘Ethernet 2′ ethernet ports to aggregate, or bond. Then you can use DHCP like normal etc. And if you’re using Parallels Desktop, you’ll probably need to disable the 2 network interfaces it creates as these may interfere with your aggregated link.
If you’re not using a Mac, then hopefully this comment will help someone else.
Have fun!
I am on a mac and I will let you know how it works and what I end up with.
any luck with finding that hosting?
@chukaman: No luck, as I wasn’t prepared to pay Joyent.com $45 a month so I could get root access on Solaris. Reluctantly I’ve decided to go for a standard Linux-based hosting solution. This means no snapshots
I don’t suppose you know how to setup Appletalk networking for the shares on the Solaris box? Using netatalk under linux it is fairly easy.
Sorry Matt, no I don’t.
After Simon very kindly left a comment on my blog referring me to ZFS and his guide here, I’ve downloaded and tried SXCE on vmware. With no current knowledge of Solaris I’ve managed to setup ZFS and get it all shared nicely on my network. Thanks to Simon’s heads-up, I’m going to be migrating my Gentoo fileserver to Solaris on ZFS as soon as I get some new hardware delivered.
Thanks again Simon
Cheers Matt, glad it was useful. Good luck with the build!
I have a P4@2.4GHz and 1GB RAM. I get like 20MB/sec read/write speed. That is due to P4 beeing 32 bit. ZFS is 128 bit and doesnt like 32 bit CPU. With 64 bit dual core CPU you get speeds in excess 100MB/sec.
Hi Kebabbert, ZFS works best with 64-bit processors, so although 32-bit processors work, they are far from optimal.
Hi all. I managed to get iSCSI working under Mac OS X 10.5.3 with the OpenSolaris 2008.05 Release. iSCSI with ZFS without authentication. As a client I used the globalSAN iSCSI Initiator 3.3.0.35 beta.
I also managed to format the iSCSI Volume as a HFS+ Volumen and got Timemachine working. Taking the iMac to sleep also worked perfectly.
@ist: Good to hear. So now you can use your Solaris volume as a destination for your Time Machine backups of your Macintosh.
Marketing Con?
More like “Not Knowing what the SI Prefixes Mean” User Error.
A Gigabyte is a billion bytes, nothing more, nothing less. Get over it.
Well, Lol, you’re a bit late to the party, but sit down and take it easy
As we wouldn’t want the marketing guys to get a bad reputation, how about you suggest they use GiB instead of GB in their HD marketing? Good luck!
When installing brand new out the box disks for the ZFS pool, do you need to format them first? If so, could you tell us how?
thanks
Hi Shaky, with ZFS no formatting is required, so you can create the pool immediately.
OK, got it set up and running … but read/write is ridiculously slow.
I’m using NFS from my Mac .. getting 2MB/s.
Will try CIFS share.
Don’t think it’s the network.
Could be the SATA drivers/chipset? In the BIOS it’s set to AHCI. How can I be sure solaris is using ahci drivers I wonder?
prtconf -D shows:
pci8086,5044, instance #0 (driver name: ahci)
disk, instance #0 (driver name: sd)
disk, instance #1 (driver name: sd)
disk, instance #2 (driver name: sd)
disk, instance #3 (driver name: sd)
what’s that sd driver I wonder?
listing the devices shows that the drives are on:
/devices/pci@0,0/pci8086,5044@1f,2:
disk@0,0 disk@0,0:i,raw disk@0,0:r,raw disk@1,0:f disk@1,0:p disk@2,0:b,raw disk@2,0:l,raw disk@2,0:u,raw disk@5,0:i disk@5,0:r
/devices/pci@0,0/pci8086,5044@1f,2/disk@0,0:
/devices/pci@0,0/pci8086,5044@1f,2/disk@1,0:
/devices/pci@0,0/pci8086,5044@1f,2/disk@2,0:
/devices/pci@0,0/pci8086,5044@1f,2/disk@5,0:
This guy has a similar setup and problem:
http://mail.opensolaris.org/pipermail/zfs-discuss/2008-April/047039.html
He fixed, he thinks, by manually zeroing out the start and ends of the drives … um, any idea how to do that?!
thanks
3am here, been at this since 6pm …. the day before yesterday. It shouldn’t be this hard!
Hi Shaky, sorry to hear of your initial problems. I don’t have time right now to look at this in any depth as it’s late now, but to get you started, perhaps you might want to consider the following:
1. Regarding the slow speeds you’re getting across your network, I presume you are using a 100 Mbps ethernet link — i.e. you’re not using a gigabit switch / router? Also, if you ARE using gigabit ethernet, be sure you are using Category 5e or 6 ethernet cable on BOTH ends of your network connections — i.e. on Mac and ZFS server. To check your network speed settings, as root issue the ‘dladm show-dev’ command from the command line — see the output from my system below, which shows that I have dual gigabit links operational:
# dladm show-dev
LINK STATE SPEED DUPLEX
nge0 up 1000Mb full
nge1 up 1000Mb full
2. I was unhappy with the write speeds using NFS sharing, so I switched to using CIFS sharing, and this did improve write speeds quite considerably. Using gigabit ethernet, category 6 cables and CIFS sharing, I was getting around 40 MBytes/sec sustained transfer speeds with a 3-drive array using a RAIDZ1 vdev to form the storage pool. When I used link aggregation to link 2 ethernet ports at each end, the sustained speed rose to around 80 MBytes/sec.
3. Regarding AHCI, I can’t remember — will look tomorrow. Try turning it off. As I have an NVidia-based MCP chipset, my system is using the nv_sata driver, but I think you say you are using a different motherboard, so I can’t say any more right now.
Hope this helps for now.
Regards,
Simon
Ah, thought I had gigabit, but no:
LINK STATE SPEED DUPLEX
e1000g0 up 100Mb full
Seems my router only 10/100. Wonder if I can plug directly into my Mac. Will it get an IP address? Hmmm. Will keep on trying!
Yeah, I’m using Intel DG33TL motherboard with Intel gigabit nic and ICH9R sata
thanks for this info!
hmm, i see this in the messages log:
Aug 8 03:31:32 opensolaris ahci: [ID 405770 kern.info] NOTICE: ahci0: hba AHCI version = 1.20
Aug 8 03:31:32 opensolaris unix: [ID 954099 kern.info] NOTICE: IRQ21 is being shared by drivers with different interrupt levels.
Aug 8 03:31:32 opensolaris This may result in reduced system performance.
The first time I installed, the bios was set to IDE. I then reinstalled with it set to AHCI. Wonder if the first install did anything to the drives?
Hi Shaky, you might be able to plug the ethernet cable directly into the Mac’s ethernet port with a standard cable, or you might need to use a crossover cable. Then, unless your Mac is running a DHCP server service, you will probably need to manually set the IP address on the Solaris end. I would get a switch though if you want the higher speeds permanently.
If you plan to connect other machines to the server your best best will be to use a gigabit switch. I can recommend a DLink DGS-1008D which is about 70 euros. This is a simple, fast and cheap 8-port unmanaged gigabit ethernet switch — just plug in cables and go, nothing else to do. It also has intelligence built-in that detects which ports are in use, and only uses electricity according to what it needs to power. See more here:
DLink DGS-1008D green ethernet 8-port Gigabit switch
However, if you have 2 gigabit ethernet ports on each machine, you might want to aggregate the links to get increased speed. If so, you’ll need a managed switch, which are about 150 euros. See here for more info: Home Fileserver: Trunking.
Regarding AHCI, did you try turning it off in the BIOS to see if that works better? It’s not an area I know much about right now, but I think it’s disabled in my BIOS currently. Next time I reboot, I’ll take a look at my BIOS settings. There’s an AHCI wikipedia page, which says AHCI allows things like hot-plugging and NCQ.
BTW, did you check your motherboard for compatibility with OpenSolaris by looking at the hardware compatibility list (HCL) that Sun have on their website? It not, it might be worth checking.
Good luck!
Yeah, mobo is on the list.
Finally zeroed the drives. Took about 4 hours. Now booting to svn_95 and noticed the installer boots to 32 bit. Strange.
thanks for all the help!
OK, glad it’s working now. I never saw that issue with this motherboard and configuration. Have fun!
But it’s a 64 bit CPU … .not sure why installing 32bit
hmmmmm!
Ah, just the installer shows 32 bit. The actual installed solaris is 64bit.
Well, zeroing the disks helped a little. Can now get 5MB/s. Still not awesome!
I haven’t changed to IDE from AHCI yet.
I tried the CIFS share method you detail above, but cant seem to get the passwd command to work.
this bit:
# vi /etc/pam.conf
other password required pam_smb_passwd.so.1 nowarn
# passwd simon
I’ve edited the pam.conf file, but the passwd command executes the normal passwd binary i.e. usr/bin/passwd
“When the SMB PAM module is installed, the passwd command generates such an encrypted version of the password”
how do you install the PAM module? Do you need to restart something after editing the pam.conf file?
thanks
Ah ignore that. it does work. adds entry to /var/smb/smbpasswd.
sudo service com.apple.autofsd stop
sudo service com.apple.autofsd start
to restart the autofs and it works! Well I can cd to it. Can’t see in Leopard Finder, even after relaunching it.
Anyway, no faster
If you can ‘cd’ to the share directory, that sounds good. I presume you can also create a file and ‘cat’ it etc from the Mac?
If so, then it might just be a problem that the Mac’s Finder doesn’t seem to be aware of the share, so try to restart the Finder by holding down the ‘alt’ key and right-clicking on the Finder icon in the Dock. Then click the ‘Relaunch’ menuitem in the popup menu.
Hopefully that should do the trick.
An initial access to a share might take a few seconds as autofs does its business. Shares will also timeout after a while, potentially causing short delays when re-accessing a little-used share.
Well, changed to IDE. Reinstalled. Got it all working again. Still slow. Over the network I get about 3MB/s.
trying: dd if=/dev/zero of=delete.me bs=65536
I get 14MB/s on my main drive, 7MB/s on the zfs pool
Sooooo … .. so could be net related. Still 14MB/s is slow right?
In my messages file I see this:
Aug 10 09:10:21 opensolaris pcplusmp: [ID 803547 kern.info] pcplusmp: pci8086,294c (e1000g) instance 0 vector 0×18 ioapic 0xff intin 0xff is bound to cpu 1
Aug 10 09:10:21 opensolaris mac: [ID 469746 kern.info] NOTICE: e1000g0 registered
Aug 10 09:10:21 opensolaris e1000g: [ID 766679 kern.info] Intel(R) PRO/1000 Network Connection, Driver Ver. 5.2.9
Aug 10 09:10:22 opensolaris unix: [ID 954099 kern.info] NOTICE: IRQ21 is being shared by drivers with different interrupt levels.
Aug 10 09:10:22 opensolaris This may result in reduced system performance.
Then, for the drives, I see:
Aug 10 09:10:23 opensolaris npe: [ID 236367 kern.info] PCI Express-device: ide@0, ata0
Aug 10 09:10:23 opensolaris genunix: [ID 936769 kern.info] ata0 is /pci@0,0/pci-ide@1f,2/ide@0
Aug 10 09:10:23 opensolaris unix: [ID 954099 kern.info] NOTICE: IRQ21 is being shared by drivers with different interrupt levels.
Aug 10 09:10:23 opensolaris This may result in reduced system performance.
not sure if that’s the problem.
3 MBytes/sec across your network sounds slow, but if you’re going through a 100 Mbps router you won’t get over about 10 MBytes/sec. Are you going through your router or are you using an ethernet patch cable to link your two machines directly? And what category ethernet cable are you using?
I get this message for each pool disk, and you clearly see that it says sata, pci and NO ide:
Aug 10 17:02:57 solarisbox sata: [ID 663010 kern.info] /pci@0,0/pci1043,8239@5,1 :
Strangely, yours doesn’t mention sata, but does mention ide… doesn’t look right to me. Have you checked anything related to SATA in your BIOS?
As for the messages relating to shared interrupts, I have similar ones in /var/adm/messages so it’s probably OK.
Going through a router. Um, not sure what cables.
Bios has these choices for IDE/RAID/AHCI
http://www.sun.com/bigadmin/hcl/data/systems/details/7045.html
says to configure as AHCI. I did that first. Was just trying out the IDE setting this time.
So, if you want over 10 MBytes/sec, you’ll need to get a gigabit switch. Plug the switch into the 10/100 router. Plug both computers into the switch.
Secondly, check that both computers use at least category 5e ethernet cable (I use category 6, but 5e should suffice).
That will get you a bit further down the road to getting higher speeds across your network.
Then you might need to research how to get a SATA driver running nicely for your mobo’s SATA chipset from users with the same mobo, and as it’s on the OpenSolaris HCL, it should be possible, hopefully. Good luck!
Then I would use CIFS sharing (NFS write performance was not as good as CIFS).
Thanks. I got a gigabit switch and new cables yesterday. Not had chance to set it all up yet. Thanks again.
Got it set up. Checked Gigabit at each end - all good. However didn’t use Cat 5e cable all the way (too short!). Going to try that tonight.
Still, it’s slow. About 2MB/s. Actually slower than it was yesterday - I changed the drive setup again back to AHCI from IDE.
Read tests using some dude’s script show about 60MB/s. Which isn’t too far off what other testers get (70-80MB/s). So it’s just write speed, to a ZFS pool shared over NFS. Actually the NFS doesn’t really matter, the write speed is just slow.
As mentioned above, this guy has same hardware and same problem, but his fix didn’t work for me:
https://www.opensolaris.org/jive/thread.jspa?threadID=57647
Options:
0. Try the gigabit cable, but I dont think this is the problem now.
1. Try Linux, see if it’s the drivers. Could try sofware RAID or my mobo supports hw RAID. Don’t know if Linux would drive it though.
2. Go back to shop, change mobo and maybe processor if needed. Friend has Asus board that worked just fine. Super fast write speeds.
3. Keep fiddling. There must be a way to get the drives to work faster…. it’s a supported mobo and SATA interface.
I have to get it sorted. I can’t concentrate at work .. I’m just spending time reading forums!
OK. So, using Linux. Just LVM, not RAID5, I get 50MB/s via Samba. Could not get NFS to work. Could the NFS share in Finder, but got error -43 when trying to access it. Can’t cd via terminal either. Perm denied.
Couldn’t get the Samba share to display in finder either (using your method above). Could cd via terminal just fine.
Going to try RAID5 on linux next. But I think this means that it’s the opensolaris ata drivers that are causing the slowness.
So, as much as I’d like to use ZFS, I think I’ve had enough! I’ve been at this for almost a week now!
I just want a resilient file store!
But then I re-read your previous posts in this series:
“He says: “I’m using it [ZFS] because I’m fed up with losing data to weird RAID issues with Linux, and I believe that OpenSolaris with ZFS will be substantially more reliable long-term.”
Ah, I really do want ZFS
Yes, I think you really *do* want to use ZFS for all the super advantages it offers.
However, as you saw from your Linux experiment, the hardware seems to be working fine. The issue seems likely to be a driver issue, or possibly a BIOS misconfiguration causing the driver to operate sub-optimally.
Oh, and you did check that you’re now using category 5e cable or greater on all computer ethernet to switch connections now?
Did you find any useful info when you searched on Google for other ZFS users using your motherboard? They must have the answers that will help you either fix the current (driver) issue, or abandon the motherboard and choose a different one that is definitely known to work (like mine if you have to).
If you decide to get my mobo, note that I am not getting very good power consumption figures (120W with 3 drives in the pool, plus an OS boot drive), due to the fact that I have an AMD processor of family 15 (see here for a full explanation: http://breden.org.uk/2008/03/02/home-fileserver-zfs-hardware/#comment-2130)
So it is my feeling that currently some Intel CPU’s will give you lower power consumption figures than AMD processors, as OpenSolaris has better support for CPU frequency scaling on Intel processors than AMD ones, and when the processor is idle (95% of the time probably) this is important. OpenSolaris supports CPU frequency scaling on AMD processors of family 16 plus, but that means the Barcelona processors and these appear to use between 80W and 120W — ouch!
For the sharing problem, for simplicity, ensure the user and group ids match on your Mac and Solaris boxes. If they don’t currently, then try deleting the user from Admin panel, and recreate the user with this line (replace ‘username’ and ‘groupname’ with your preferred name, of course):
On the mac, in Terminal type: id your_user_name
Record the ‘uid’ and ‘gid’ numbers (user id and group id).
As root at Solaris command line, create group for ‘groupname’ and user for ‘username’ (uid and gid to match Mac’s ids):
# groupadd -g gid groupname
# useradd -g gid -u uid -s /bin/bash -d /export/home/username -c username -m username
-g gid: adds user to group gid
-u uid: creates the userid uid for this user
-s /bin/bash: assigns the default shell to be bash for this user
-d /export/home/username: defines the home directory
-c username: creates the comments/notes to describe this user as required
-m: creates the home directory for the user username
And for your shared directory do: (1) ‘chmod 755 shared_directory’ to give your user on the mac read/write access, (2) ‘chown username shared_directory’ and (3) ‘chgrp groupname shared_directory’
Good luck.
Yup, cat 5e all the way now.
Not really any useful info on Google for my mobo and solaris.
Going to try to find an SATA card and try that.
Oh, I just read that on some chipsets, when there are more than 4 SATA ports on the board, only the first four are usable properly. The others are hidden behind some ‘raidgoofyness’.
Mine are plugged in to 1,2,3 … and for some reason 5.
Could try 4!
Slot 4 didnt work.
SATA card wasn’t recognised by my install of Solaris. I used svn_93. It’s a bug fixed in svn_94. Tried the workaround, that didn’t work. Could try it again, but I decided I want my onboard SATA ports to work.
I’ve found a board that works (my friend has it - the Asus M2N-VW HDMI). Means spending about #80 GBP (need new CPU too), rebuilding the parts and reinstalling.
Tried harware raid5 and Fedora 9 linux last night. Didn’t work.
So, my choices are:
1. Spend, wait (can’t get mobo until tomorrow), rebuild (or let the shop do it, but that means lugging it down there)
2. Go with sotware RAID5 on a Fedora 9 linux install.
3. No RAID, just go with ext3 in a big LVM on a Fedora 9 linux install.
I am a little concerned with the lack of resources for troubleshooting OpenSolaris. fedora seems a lot slicker.
Screw it. I spent. Pick up new mobo and cpu tomorrow.
Hopefully your new hardware will work.
Well, sorted, almost.
I got the Asus M2N-VM HDMI. Rebuilt the machine and installed OpenSolaris svn_93. No network. Apparently the nVidia chipset doesn’t work via the Solaris drivers. I tried the nfo 3rd party drivers but couldn’t get those to work. So I popped out and bought an Intel PCI gigabit ethernet card. All good.
BTW - my friend gave me the wrong mobo model name! He had the M2A-VM
Created and shared zfs pool via NFS and CIFS. I get about 20MB/s over NFS and 30MB/s over CIFS. I’m not using my new 5e cable yet though (need the long cable so I can have box in living room plugged into TV)
So I decided to go via CIFS. I couldn’t get the /shares mount to show up in Finder like you, so I mounted in /Network/Servers via the fstab file. Lovely, worked fine.
My user/group was set properly, however, any files copied via Finder to the share were written without any permissions at all on Solaris box.
I noted my zfs pool had aclinherit=restricted whereas yours has ’secure’ I tried changing that, restarting the smb service, but the share just disappeared. Mac log said it was dead.
So .. still trying!
Needed to do this:
chmod 777 /pool
chmod A=everyone@:rwxpdDaARWcCos:fd:allow /pool/fs
Tried svn_95. Now getting 35-40MB/s. Didn’t read all the comments, someone mentioned a problem with svn_93 and CIFS.
Fast enough for me.
Finally.
Many thanks for all your help during my ordeal!
Congratulations on getting it working shaky! Those speeds sound a lot more what I would expect to see.
You certainly had some ‘fun’ getting it all to work
And thanks for reminding me — I was going to write up some stuff on ACLs and never got round to it yet.
Hope your system works nicely now.
Where can I find good info on choosing a CPU for OpenSolaris?
I want a dual core, low wattage, 64 bit CPU.
Should I go Intel or AMD?
Forget price/watt/performance ratios.
Which CPU would perform best in Opensolaris?
If you want a low power CPU for Solaris (and ZFS) then choose a 64 bit processor that supports CPU frequency scaling. From what I’ve read so far, it seems that Solaris has better support for Intel processors in this respect — see: http://breden.org.uk/2008/03/02/home-fileserver-zfs-hardware/#comment-2130
It’s difficult to recommend a specific processor without knowing what kind of workload you plan for your machine.