A Home Fileserver using ZFS

For many people who use a computer, knowing where to store growing amounts of data can become tricky.

You start off with one disk, run out of space, buy a bigger one etc. And if you have a camcorder you’ll be generating gigabytes of data for every Mini DV tape you record. Also, you may have a digital video recorder attached to your TV and wish to permanently keep some of the programmes/films you’ve recorded. Now you’re talking hundreds of gigabytes, if not terabytes of storage that are required to handle all this data.

And then there’s the problem of backups… oh boy, this will be a fun project :)
Read the rest of this entry »

Popularity: 100% [?]

Shuttle XS35GT: Installing XBMC 10.0 Live (Dharma)

After you’ve bought your Shuttle XS35GT silent media center / HTPC, you’ll want to install some good software to make the system work well. Personally, I like the XBMC software, and here I’ll describe how to install it on the XS35GT.

In order to reduce costs, use less memory & disk space, and make the system run faster, I have chosen to install Ubuntu Linux with XBMC, rather than use Microsoft Windows.

Also, as the XS35GT contains no DVD or CD reader, we will be using a USB memory stick as a replacement for a CD/DVD boot/install media, so ensure you have one handy. A 1GB or 2GB device should be sufficient. The XS35GT refused to boot off one of my USB memory sticks and so I had to try another one that it would boot from, so if you get this problem just try another stick and you will be OK.
Read the rest of this entry »

Popularity: 6% [?]

Shuttle XS35GT: Installing XBMC Dharma beta 2 Live

After you’ve bought your Shuttle XS35GT silent media center / HTPC, you’ll want to install some good software to make the system work well. Personally, I like the XBMC software, and here I’ll describe how to install it on the XS35GT.

In order to reduce costs, and make the system run faster, I have chosen to install Ubuntu Linux with XBMC, rather than use Microsoft Windows.

Also, as the XS35GT contains no DVD or CD reader, we will be using a USB memory stick as a replacement for a CD/DVD boot/install media, so ensure you have one handy. A 1GB or 2GB device should be sufficient. The XS35GT refused to boot off one of my USB memory sticks and so I had to try another one that it would boot from, so if you get this problem just try another stick and you will be OK.
Read the rest of this entry »

Popularity: 5% [?]

Shuttle XS35GT: The ultimate silent media center HTPC

The long-awaited Shuttle XS35 series of small form factor (SFF) PCs has finally arrived, after a long release delay caused by cooling problems.

For home media center (HTPC) enthusiasts, this series of computers is ideal for making a media center because they are completely silent, due to the fact that the case houses no fans. Instead, the model range use a combination of (1) a massive heat sink and (2) holes in the case to dissipate generated heat.

For the media center enthusiast, half of the model range is of particular interest: the models that include second generation NVidia ION GPU hardware. According to NVidia, this ION hardware provides over 10 times the performance of integrated graphics, and significantly reduces CPU load. This enables the XS35’s integrated dual-core Atom D510 processor to run at between 5% to 10% CPU load when watching 1080p HD video content. That is impressive!
Read the rest of this entry »

Popularity: 6% [?]

Climate change: Climategate

In the last few days the internet has been ablaze with news of leaked documents, emails and computer code from computers at the CRU — The Climatic Research Unit at the University of East Anglia, UK.

In what is being called ‘Climategate’, evidence is beginning to emerge of collusion to exaggerate temperature increases, hide temperature declines, block Freedom of Information requests, block scientists with differing views from being published in magazines & group deletion of emails to prevent data disclosure.

But what exactly happened, and why all the fuss?

Read the rest of this entry »

Popularity: 4% [?]

What happened to global warming?

On 9 OCT 2009, the BBC News site published an article entitled ‘What happened to global warming?‘ which claims that for the last 11 years, since 1998, we have not observed any increase in global temperatures.

As we are bombarded on a daily basis with news telling us that man’s activities are producing carbon dioxide, carbon dioxide is responsible for global warming/climate change, and that unless we drastically reduce carbon dioxide levels then global temperatures will continue rising and we will all be doomed, the title of this article seemed very intriguing.

The article states that although carbon dioxide levels are continuing to increase, temperatures are not increasing. This seems at odds with what we should expect, according to the current consensus amongst many scientists studying climate change: i.e. more carbon dioxide should equate to higher global temperatures.

So I read the article with great interest and then took a look around at recent news from other sources to see what other people and scientists are saying on this interesting topic, and this is what I found.

Read the rest of this entry »

Popularity: 4% [?]

Link list: October 2009

Here are a few selected links from my trawl through the blogosphere and information super highway, aka The Internet for October 2009.

The links here cover subjects including 10GbE, Storage, Sun, Oracle, “The Cloud”, Solaris, Linux, iLife, iMovie, Aperture, Final Cut, video editing, mind mapping tools, photography.

Enjoy!

Read the rest of this entry »

Popularity: 4% [?]

Link list: September 2009

Here are a few selected links from my trawl through the blogosphere and information super highway, aka The Internet for September 2009.

The links here cover subjects including version control systems, git, Storage, Sun, Oracle, Solaris, Linux, Apple, Snow Leopard, ZFS, HP, self-improvement, getting things done / task management, mind-mapping, finding lowest price camera equipment etc.

Enjoy!

Read the rest of this entry »

Popularity: 5% [?]

Home Fileserver: Handling pool errors

Until today I had never encountered even one read, write or checksum error on my ZFS NAS. Today I saw one checksum error coming from the mirrored SSD root boot pool which I have just installed.

Unless you’re using a file system like ZFS (or NetApp / Veritas…$$$), then you’ll almost certainly never even know that errors are occurring within your storage system. Ignorance is bliss, apparently, but I’d rather have information available so I can act on it, determine the root cause of the problem and try to prevent re-occurrence.

Unresolved storage errors can often lead to bigger problems later, so let’s fix the problem right now while we can.
Read the rest of this entry »

Popularity: 7% [?]

Home Fileserver: ZFS boot pool recovery

If you should be unlucky enough to be unable to boot your OpenSolaris NAS one day, then these notes taken from a real restoration test might help you get back up and running again quickly.

After setting up a supposedly robust mirrored ZFS root boot pool here using two SSD drives, I decided to give it a system test by completely destroying the boot pool. I did this in order to understand and verify the process of restoring a boot environment from scratch, and I have documented this process here in case I need it one day, and you’re welcome to use it too if you need it. :)
Read the rest of this entry »

Popularity: 7% [?]

Home Fileserver: Mirrored SSD ZFS root boot

When I started using this ZFS NAS for additional tasks, I realised that my boot environment was becoming more important, and was a weak point in this system.

Initially I wanted to run a master copy of this website locally on this ZFS server, and that entailed installing WordPress and the OpenSolaris AMP package comprising of the Apache HTTP server, MySQL and PHP. Also, I enabled and configured the VNC server. All of this required a fair amount of configuration and I didn’t want to have to do it again in the event of drive failure. Once is enough, so I wanted to discover how to backup and restore my boot environment to protect my investment in time and painstaking configuration work.
Read the rest of this entry »

Popularity: 8% [?]

Breden’s BASIC: The Source Code

After getting a decent version control system setup on this ZFS NAS (git, Subversion or something else ???), I will be feeding the source code of Breden’s BASIC into it.

Then after re-familiarising myself with the code, I plan to make the 6502 assembly language source code available online for 6502 enthusiasts, or anyone else curious to know how we had to program system code back in the 1980s.

Anyone unfamiliar with assembly language for a simple 8-bit processor like the 6502 might be quite surprised (shocked?) how low-level coding was. One little mistake and the whole machine would lock up!
Read the rest of this entry »

Popularity: 9% [?]

Home Fileserver: Media Center

Your ZFS home fileserver / NAS is a great place to store your music, photo and video media, and if you’ve setup your ZFS file systems similar to the way described in Home Fileserver: ZFS File Systems, then it will be quite simple to view this media from a media center.

The nice thing about storing your media on your fileserver is that if your original CDs or DVDs get scratched, you will always have your backup available on your fileserver. The other huge advantage of having your media on your NAS is that you don’t have to go hunting through your DVD library to find the right box, only to find that it never got put back in the right box. And with a network-enabled media center hooked up to your ZFS fileserver / NAS, you can locate any media at the touch of a remote control.

There are many different media centers or HTPC products out there and they vary in openness, price, power and usability.
Read the rest of this entry »

Popularity: 15% [?]

Home Fileserver: OpenSolaris 2009.06

After running SXCE for the last year, due to a failed hardware upgrade, it was time to install OpenSolaris 2009.06.

I was due to leave on a trip the next morning, but foolishly I decided to install a 3.5″ drive into a 5.25″ drive bay with some anti-vibe rubber grommets installed. When re-attaching the IDE cable onto the back of the drive in an awkward place, I broke one of the IDE interface pins and the drive failed to be recognised at POST after rebooting. After several attempts to reboot, it was time to face facts. The IDE boot drive that had served me faithfully since these days over a year ago, had reached the end of the line. Game Over.
Read the rest of this entry »

Popularity: 16% [?]

Home Fileserver: ZFS File Systems

Once you’ve built your ZFS home fileserver / NAS, you’ll want to create your storage pool, create your file systems and share them to various devices around the home, such as laptops, PC’s, Macs, media centres etc.

I will go through all the necessary steps from start to finish, so you can see how to create a full, working file system hierarchy that is practical and useful.
Read the rest of this entry »

Popularity: 26% [?]

Macro magic 3

During a recent trip to the Hillier Gardens in Ampfield, Romsey in Hampshire, England, I took the following photos with a Nikon 105mm VR macro lens. This lens has excellent bokeh properties, due to the use of a nine-blade rounded diaphragm opening, giving pleasing effects in the out-of-focus areas of the photos.

I visited Hillier Gardens when the weather was overcast and, crucially for macro photography, windless. Well, virtually no wind.

I did not use a flash on any of the photos, relying purely on the available light, to give as natural as possible colours, lighting and textures.

When taking these photos I used (1) a tripod, (2) cable shutter release and (3) shutter release delay, in order to provide stability, prevent introduced vibrations from blurring photos, and allow longer exposures to allow greater depth of field using the available natural light.

The only image editing that has been done is to scale the photos to fit this web page, and to overlay a watermark for the copyright message and web link.
Read the rest of this entry »

Popularity: 12% [?]

Home Fileserver: A Year in ZFS

Doesn’t quite have the same ring to it as ‘A Year in Provence’, does it? Oh well, never mind. ;-)

After a year of using Solaris and ZFS for a home fileserver, I thought I would share my experiences here to give an insight into things that worked or did not work.

Also, others have asked me to give a summary of my experiences of using ZFS to highlight strong and weak areas, and to give a critique.
Read the rest of this entry »

Popularity: 23% [?]

Credit Crunches

The term ‘credit crunch’ has been well and truly flogged to death during the recent financial chaos, so it was time for me to dump all this stuff out of my head into this new blog post. As well as cursory explanation of what happened, I’ll chuck in a couple of chuckles as we all need to laugh.

After imprudent lending on a monumental scale to people who hadn’t a chance in hell of ever being able to pay back enormous housing loans, the proverbial has finally hit the fan, as ’sub-prime’ borrowers default on their loans in ever-growing numbers. The ‘credit default swaps’ (CDS) insurance policies underwriting these sub-prime (high risk) housing loans have now been found to have been woefully under-priced, thus the sellers of these insurance policies have had to face enormous pay-outs as the borrowers default on their loan repayments. Worse, as various investment banks ‘invested’ in this sub-prime debt, they infected themselves to the point of no return, as they discovered when the CDS’s were unable to pay out when the sub-prime borrowers defaulted on their loans. The final straw that broke the donkey’s back was high fuel and food price inflation, pushing up household debt to unaffordable levels. Oh, and a falling house price market, meaning that selling the houses would not cover many of the recent loans. It was the ‘perfect storm’ of the financial world.
Read the rest of this entry »

Popularity: 22% [?]

USA preparing for civil unrest?

It seems that the U.S. government seems to think there is some impending calamity ahead, as it will be deploying the army on home soil for the first time.

What problem does the government foresee? Who knows, but if they’re making it public by putting this info on the internet then it’s obviously not that secret. But too secret to tell you outright, it seems. Yes, you’ll have to guess on this one.
Read the rest of this entry »

Popularity: 26% [?]

FAA using ZFS

Quote from eWeek.com:

The embattled FAA, which has suffered a number of embarrassing flight-plan system crashes this year, has upgraded its legacy internal business systems to a new open-systems server and storage infrastructure supplied by Sun Microsystems and an IP network provided by Cisco Systems. If all goes as planned, this architecture may replace critical systems that directly affect all air travelers in the United States.

The most recent example of this happened on Aug. 26, when a corrupt file entered the flight plan system and brought it down for about 90 minutes during a high-traffic period late in the day on the East Coast. This was not an isolated incident, as the FAA’s chief administrator originally had told the media. Similar crashes occurred on Aug. 21 and in June, FAA records show.
Read the rest of this entry »

Popularity: 26% [?]

Link list: September 2008

Here are a few selected links from my trawl through the blogosphere and information super highway, aka The Internet for September 2008. As always, as with any information you read, check for authenticity by doing your own research. Enjoy!
Read the rest of this entry »

Popularity: 23% [?]

Home Fileserver: Active Directory Integration

Matt Harrison wrote to me recently, to notify me about a guide he’d written showing how to integrate a ZFS-based OpenSolaris fileserver with Microsoft Active Directory.

His guide covers all the necessary areas like: configuring Kerberos, synchronizing time/dates, enabling the CIFS server, joining the domain, user and group mapping, ZFS datasets, setting ACLs and auto-sharing home directories.
Read the rest of this entry »

Popularity: 33% [?]

Home Fileserver: RAIDZ expansion

Recently I decided to expand the storage capacity of my ZFS home fileserver’s ZFS storage pool, which was configured to use a single RAIDZ vdev comprising of three 750 GB drives.

I wanted to expand the RAIDZ vdev by adding one extra 750 GB drive.

As you may or may not be aware, it is currently not possible to expand a RAIDZ vdev in ZFS, which is a great pity.

However, it is still possible to achieve this expansion goal, but you have to perform a few tricks. Here is how I did it.
Read the rest of this entry »

Popularity: 39% [?]

Microgeneration

The UK seems to be embarking on a big nuclear power plant building exercise, to replace existing plants about to be decommissioned, to help reduce reliance on oil and gas, to reduce the carbon footprint and to increase availability of power for increasingly power-hungry consumers. These new plants will most likely be the new EPR reactors.

Apart from past horrors like Windscale, Chernobyl, Three Mile Island, constant ‘minor’ leaks, spills, irradiation of power plant workers, radioactive contamination of the water table, the political ‘hot potato’ of the costs and dangers of long-term safe storage of radioactive waste for thousands of years, plus costs of securing the waste from terrorists, a big nuclear power plant rollout plan sounds like a great idea.

But what about renewable energy sources? Are these really viable, or just a crazy idea? They are at least clean and safe, don’t generate waste in operation, and provide employment for installation and maintenance personnel.
Read the rest of this entry »

Popularity: 26% [?]

Two French nuclear spills in one month

July 2008 has been a bad month so far for the French nuclear power industry with two nuclear spills already, and the month is not finished yet.

The first nuclear accident occurred on Monday 7th July 2008 at the Tricastin facility, where 74 kg of liquid uranium was spilled and entered the water table.

Interestingly, or not, at the time of writing this, the original Tricastin nuclear spill story I read on The Daily Telegraph website at this URL France orders tests on all nuclear power stations after leak has been removed (it seems back again, although edited perhaps, as the edit date is inconsistent with the Google Cache version). However, while it lasts, here is the Google Cache version.

As if the first nuclear spill at the Tricastin site at Bollene was not embarassing enough, there was a second nuclear spill at another Areva-controlled nuclear plant at Romans-sur-Isere, south-eastern France on Saturday 19th July 2008, which you can read about here: Second Nuclear Leak In France. Inspectors reportedly found that the pipe had been broken for several years and did not meet safety standards. Tut, tut, but never fear, as French Environment Minister Jean Louis Borloo has called for tougher controls at nuclear power plants, suggesting automatic problem detection and alerting systems be used in future. Now, which companies sell that cool technology, and why aren’t they already using them if they exist? Borloo’s comments will no doubt soothe any fears any potential buyers of French nuclear technology might have.
Read the rest of this entry »

Popularity: 29% [?]

The Great Global Warming Debate

I would be the last person to claim to be an expert on issues related to global warming. However, recent news items that have caught my attention have made me curious. In today’s article American physicists warned not to debate global warming it appears that there is unease regarding the close examination of the figures used in the current global warming algorithms and models.

Viscount Monckton claims that the fundamental figures used in the global warming models were derived from a theoretical model that looks good initially, but that fails to take into account the infinite variations in conditions presented by the real world. Here is his explanation of how the global warming believers’ climate models are verified:

“Since we cannot measure any individual forcing directly in the atmosphere, the models draw upon results of laboratory experiments in passing sunlight through chambers in which atmospheric constituents are artificially varied,” writes Monckton. “Such experiments are, however, of limited value when translated into the real atmosphere, where radiative transfers and non-radiative transports (convection and evaporation up, advection along, subsidence and precipitation down), as well as altitudinal and latitudinal asymmetries, greatly complicate the picture.”

Read the rest of this entry »

Popularity: 26% [?]