I just built my own ZFS NAS with the OpenSolaris b134 developer’s release. (I really want to have Dedup! However I am having difficulty getting VNC server to work. You mentioned that you had set it up and I know how you like to ride the bleeding edge, so I was wondering if you might have some pointers for getting it working on b134. By default it’s installing xvnc but all the guides I have found have been concerning SUNWxvnc, which is not installed by default. Any insights would be greatly appreciated.

-James

Indeed, when ZFS is requested to read file blocks from redundant storage (mirror, RAID-Z1, RAID-Z2 & RAID-Z3 vdevs), if there is a block checksum error, ZFS will not only return the correct data, but it will also heal the problem. The read request may be in the form of either a file read or a scrub. As a user will never normally read all files back on a regular basis, this highlights the importance of performing regular scrubs, as this will be only real way that bit rot is detected and fixed.

During the heal process, for mirrors, the correct data is located on other drives within the mirror vdev, and in the case of RAID-Z1, RAID-Z2 & RAID-Z3 vdevs, the correct data is reconstructed using the available parity data.

Cheers,
Simon

I just realized that ZFS mirrors also self-heal. I was thinking to myself – Why is Simon going with mirrors rather than RAIDZ? Ah, now I know. Which makes it a very good option for booting, rather than RAIDZ, as the speed should be greater. Maybe this comment will spell it out for other people scratching their heads like I was.

http://blogs.sun.com/timc/entry/demonstrating_zfs_self_healing

So it seems likely that the Solaris poster was right when he mentioned that it would be easier using 10GbE to achieve fast speeds, as when 10GbE becomes more mainstream, drivers should be easier to find for mainstream OSes, indeed they will probably be included within the OS itself.

http://www.colfaxdirect.com/store/pc/viewPrd.asp?idproduct=12

See related items (below) for cable prices.

Thanks for the links, I will check them later.

$300 looks to be a good price! Got any links? I will check it out.

Yes, I was looking at Infiniband a year ago as a possible direct-attach method here for video-editing storage:
http://opensolaris.org/jive/message.jspa?messageID=310488

The replier pointed out that:
(1) Although he was an Infiniband enthusiast, he felt that the 10GbE approach would be easier.
(2) 10GbE switches are the most expensive component and I could just eliminate the switch and go 10GbE NIC to 10GbE NIC directly without the need of a switch.

I didn’t go any further yet with the project, but as a year has passed since, I should take a new look at 10GbE NIC prices and compare with Infiniband… hopefully prices have gone down significantly… then again, maybe they haven’t?

Then again, new tech like Fusion-IO has come along — basically fast Flash on a PCIe-connected card as a much faster way to access local storage than SATA II/III devices etc.
See: http://www.fusionio.com/products/
Here’s a vid about it: http://www.youtube.com/watch?v=VS9Zxnx-r0E&feature=player_embedded
Steve Wozniak is onboard, so it looks like it should be very good
It looks like it has fantastic performance for niche storage needs, but it’s a bit pricey for now, and storage size is very limited

Thanks a lot.

Cheers,
Simon

Why not use Infiniband? That’s becoming affordable right now. Just a direct connection between NAS and workstation, will cost you $300 (2 HCAs and a cable).

Very nice blog btw.

For anyone interested, the SuperMicro AOC-USAS-L8i is a great 8-port SATA (or SAS) card under OpenSolaris, but it uses a bracket which you will have to unscrew to use it in a standard tower case — it’s really designed for use in SuperMicro UIO setups. The other good side is that it’s a good price. It uses the LSISAS106E ASIC. When fitted, there is no secure bracket holding it in place, but it seems to work fine.

The counterpart to the AOC-USAS-L8i card is another 8-port SATA/SAS card — the LSISAS3081E-R, which uses a conventional bracket, but costs more.

Now that SSDs are reaching speeds which are fast approaching the SATA II 3Gbps limit — e.g. my OCZ Vertex Turbo SSD boot drives read at around 240MBytes/sec, we should soon expect to see new releases of these SATA cards based on SATA III chipsets which are able to reach 6Gbps per channel. Then all we need is 10GbE ethernet switches and NICs to become mainstream and the speed of transfer will be phenomenal.

I’ve got a stack here of 20 or so video tapes which I’m editing, and so I get bored of waiting for large transfers of huge video files to transfer to long-term NAS storage across a slow 1GbE link at 50 MBytes/sec or so, so 10GbE will be great when it becomes more affordable.

The breakout cable must have cost me 9 euro extra.

Although the card is HW raid, I am using it under solaris as normal SATA controller.

I am under the impression that those DELL cards are LSI-chips based.

Also, I thought about rotating one of the SSDs through 180 degrees longitudinally, so that power & SATA cables are at opposite ends of the mounting bracket, but dismissed this as I didn’t really want cables in odd positions.

In the end I found a spare molex power connector coming from the PSU, and plugged in another cable with a male molex connector on one end and a SATA power connector on the other end. The wires leading to the SATA connector were routed directly to the end of the connector and were not pass-through (i.e. leading to another connector), and so this gave enough clearance. Problem solved.

The LSI LSISAS3081E-R controller you have, and the SuperMicro AOC-USAS-L8i controller I have, both share the same LSISAS1068E ASIC, so should be pretty similar in terms of use with the motherboard’s BIOS and Solaris, I should expect.

Like you, I never had much luck with the JMicron SATA ports.

I presume your graphics card is a PCIe (PCI Express) card? I put my Asus EN6200LE PCIe-based card in the PCIe slot nearest to the CPU, and the SATA controller card in the PCIe slot furthest from the CPU, although I suppose you already tried swapping the cards around? I don’t know if it should make any difference, but you never know.

Also, you could try checking which motherboard BIOS version you are using and also the SATA controller card’s BIOS version.

As the motherboard’s BIOS can support 6 devices from the SATA ports driven by the NVidia chipset (MCP55) and 2 devices from the single IDE connector, I suppose it’s normal the BIOS only shows 8 devices (excluding the 2 possible JMicron devices).

The HBA card (LSISAS3081E-R / AOC-USAS-L8i) has its own POST messages that display the devices connected to the card, independently of the motherboard’s POST messages.

I don’t know how the motherboard’s BIOS handles external USB / eSATA devices.

Cheers,
Simon

??????????

Now i have run into another problem, i have tried contacting Asus but have not got a reply.
When i plug in the controller and try to boot, the motherboard (M2N-SLI Deluxe) is giving me
graphics error beep codes (1 long 2 short). No video is displayed and i think POST is not completed.
Motherboard probably thinks the raid controller is another graphics card and panics….

Have i missed anything? Are there BIOS settings i should check???

Best regards
/Ono