Another interesting and well written article! I am very impressed by the thoroughness and completeness of your write ups and hope you will continue to explore this subject in your writings as I have not found anyone who has carried the “ZFS/Solaris for beginners” torch with equal depth and breath. If you don’t mind the inquiry, I am curious about your background… Are you a system admin or network engineer or do you do something else to afford your toys?

Regarding Wout’s earlier question about performance of running ZFS on top of ZFS:

I believe you have the superior implementation as you did it. The reason is that you want to have as little administrative overhead traversing the network as possible. Having a single iSCSI volume for the initiator to handle is going to translate in to less client-side overhead and thus less network over head and thus more available bandwidth for the data itself and higher transfer rates.

Regards,

James

-U.S.

VMWare Servers (Linux/Windows Web/DB servers)
|
| VMWare Server (Backup, log, monitoring servers)
iSCSI |
| iSCSI
| |
ZFS iSCSI target———WHAT PROTOCOL?——->> Storage Replication server

OR

VMWare Servers (Linux/Windows Web/DB servers)
|
| VMWare Server (Backup, log, mon, etc)
iSCSI |
| iSCSI
| |
ZFS iSCSI target—WHAT PROTOCOL?—>> <<– Storage Replication server

Like yourself, I also wondered if I could mount and read/write to the backup pool from the backup server, where the disk space is specified as a volume. I didn’t spend much time trying to find out how to do it though, so I can’t give you a definitive answer. I expect it is possible, but you will probably need to search the internet for answers. Try posting to the ZFS discuss forum on the opensolaris site here, and refer here for the example of the setup you refer to: http://www.opensolaris.org/jive/forum.jspa?forumID=80

For the answer to your second question, I think if you find the answer to the first question above, then it will be a simple matter to share it via SMB directly from the backup server. If you don’t manage this though, like you said, you can simply do an SMB share of the backup file system via the fileserver. I would have thought you wouldn’t lose too much in terms of speed.

Glad you were inspired by these articles to build a couple of boxes.

Merry Christmas,
Simon

Using the configuration described above, the “backup” filesystem is created on fileserver so we can use it as if it was local… but can it be mounted or accessed from the backup machine itself? I’d still like to have the option of reading and writing to it from the backup machine.

Also, can I make the backup filesystem available via smb? Would that just be a zfs change on fileserver? Isn’t that less efficient (connecting with smb via fileserver over to backup through iscsi)?? I’d rather be able to use the same filesystem directly on backup and make it available directly via smb while still using it via iscsi from the fileserver. Hope that makes sense.

PS: thanks for the articles — inspired me to build these two boxes!

Greetings to you in Hong Kong, and thanks a lot – I’m glad it has helped you.

Merry Christmas to you too!
Simon

P.S. Wish you have a merry christmas

Thanks for the compliment! Yes, iSCSI seems quite magical so far. It makes it easy to see how streaming data for backups could be easy to a remote offsite location, as iSCSI needs an IP address. Have fun, and you reminded me that I need to tackle that article on incremental backups… have fun!

Simon

I’m fairly new to ZFS, but as I’m in charge of setting up my lab’s fileserver, I figured I’d give it a shot! I think it’s great but always had trouble with the incremental backups until I found this page – iSCSI is amazing! Looking forward to the incremental backups post. Keep up the good work!

-Justin

If I wanted to try using the imported iSCSI disk directly, rather than attaching it to a pool, how would I write to it with ZFS? There seems relatively little documentation about using iSCSI on Solaris with fully explained examples. Either that, or I have not looked well enough, perhaps.

Yes, those transfer speeds are quite decent, I thought. And I haven’t yet looked into (1) setting the MTU size to 9000 for jumbo frames, or (2) using the 2 onboard GbE ports simultaneously to get a 2Gbps full-duplex pipe using trunking/bonding — something else to try out one day Although I’ll probably use the 2 built-in GbE’s on the Mac Pro and the 2 built-in GbE’s on the ZFS fileserver as this is probably where I’d get the most benefit, as the backup machine will only be taking incremental backups once I get that going properly using snapshot ranges combined with use of zfs send/recv.

Good to know about the striping — thanks a lot!

I meant, you have your zvols exported as iSCSI targets, and then you give those as raw disks to ZFS which implements another layer of ZFS on it. So you basically are storing ZFS blocks inside ZFS blocks on the disks…

So I was wondering if it would be faster if you made the individual disks available as iSCSI targets instead of the ZVOL. The iSCSI server would have to do slightly less processing and block sizes would perhaps be better aligned? Plus the client would have more insight in broken disks, which might or might not make a difference.

Although I must say I like those transfer rates

ZFS will stripe at all times, so don’t worry about that.

Simon

Thanks for the overview!

Wout.