I’ve just installed BTRFS on some systems that matter to me. It is still regarded as experimental but Oracle supports it with their kernel so it can’t be too bad – and it’s almost guaranteed that anything other than BTRFS or ZFS will lose data if you run as many systems as I do. Also [...]
The general trend seems to be that cheap hard drives are increasing in capacity faster than much of the data that is commonly stored. Back in 1998 I had a 3G disk in my laptop and about 800M was used for my home directory. Now I have 6.2G used for my home directory (and another [...]
As storage capacities increase the probability of data corruption increases as does the amount of time required for a fsck on a traditional filesystem. Also the capacity of disks is increasing a lot faster than the contiguous IO speed which means that the RAID rebuild time is increasing, for example my first hard disk was [...]
LWN has an interesting article comparing recent developments in the Linux world to the “Unix Wars” that essentially killed every proprietary Unix system . The article is really interesting and I recommend reading it, it’s probably only available to subscribers at the moment but should be generally available in a week or so (I used [...]
One of my clients is currently investigating new hosting arrangements. It’s a bit of a complex process because there are lots of architectural issues relating to things such as the storage and backup of some terabytes of data and some serious computation on the data. Among other options we are considering cheap servers in the [...]
I previously wrote about my first experiences with BTRFS . Since then I’ve been using BTRFS on more systems and have had good results. The main problem I want to address is with the reliability of RAID .
Requirements for a File Server
Now one of my clients has a need for a new fileserver. [...]
There are some really good Network Attached Storage (NAS) devices on the market. NetApp is one company that is known for making good products . The advantage of a NAS is that you have a device with NVRAM for write-back caching, a filesystem that supports all the necessary features for best performance (NetApp developed their [...]
Based on my investigation of RAID reliability  I have determined that BTRFS  is the Linux storage technology that has the best potential to increase data integrity without costing a lot of money. Basically a BTRFS internal RAID-1 should offer equal or greater data protection than RAID-6.
As BTRFS is so important and so [...]
I’ve previously written Some Notes on DRBD  and a post about DRBD Benchmarking .
Previously I had determined that replication protocol C gives the best performance for DRBD, that the batch-time parameters for Ext4 aren’t worth touching for a single IDE disk, that barrier=0 gives a massive performance boost, and that DRBD gives a [...]
Everyone agrees that backups are generally a good thing. But it seems that there is a lot less agreement about how backups should work. Here is a list of 5 principles of backup software that seem to get ignored most of the time:
(1/5) Backups should not be Application Specific
It’s quite reasonable for people [...]