I previously wrote about a failure of a USB flash device in my Internet gateway [1]. I have since had another failure in the same system, so both the original 4G devices are now dead. That’s two dead devices in 10 weeks. It could be that the USB devices that I got for free at an exhibition were just really cheap, I’m sure that they weren’t expecting them to be used in that way. The devices from the same batch which are used for their intended purpose (sneaker-net file sharing) are still working well. But in any case I’m not going to resume this experiment until warmer weather. At this time of year some extra heat dissipation from computer gear in my home is more like a feature and less like a bug.
The second USB device to fail appeared to have it’s failure in the Ext4 journal (the errors were reported at around sector 2000), I didn’t keep a record of the problem with the first device, but from memory I think it was much the same.
Rumor has it that cheap flash storage devices don’t implement wear-levelling to avoid patent infringement. If that rumor is correct then any filesystem that uses a fixed journal in the same way as Ext3/4 is probably unsuitable for any serious use on such devices, while a filesystem based on Copy On Write will probably perform better. In Spring I’ll try using BTRFS on cheap USB flash devices and see if that works better. I have another spare device from the same batch to test so I can eliminate hardware differences. I can’t do enough tests to be a good statistical sample, but if a device lasts from Spring to Autumn using BTRFS with the same use that caused failures with Ext4 in a few weeks then I will consider it a strong indication that BTRFS is better than Ext3/4 for such uses.
For the next 5 months or so I’ll be using a hard drive in my Internet gateway system again.
I guess you can’t necessarily expect a USB flash to implement wear levelling, since they are not intended for general file storage. Typically the flash itself has some reserved sectors to replace failed blocks, but that doesn’t give you much extra life if you are using it for a journal or a swap file.
At least one of the cheap ones I have pulled apart only had a small gate array for the controller, I doubt it had wear levelling logic in there. On the other hand, another freebie 2GB USB flash had a Chinese Chipsbank controller, which does have wear levelling.
The only sure way to check would be to lookup the part number of the controller.
I’ll be interested to hear how you go trialling Btrfs on one of these.
You can use ext4 filesystem without journal. mke2fs and tune2fs can control this feature (has_journal).
neonsignal: Thanks for the information about the devices you disassembled. Maybe mine had just cheap gate arrays. Now I wonder whether I should try a different brand next time or just waste another device. As the devices are cheap I guess I might as well just try one out, I’m already well beyond any cost savings issue and into strictly hobby use.
TK: An ext4 filesystem without a journal is an ext2 filesystem which will have more potential issues for reliability. Also there are other metadata locations that get written a lot such as Inode tables and block counts. Using copy on write should reduce most of those problems but limit it to the superblocks.
etbe: ext4 has support for running without journal. Then all other extra ext4 features are supported, just journal is missing. So it’s still better than ext2. And possible fsck after crash should complete faster. But I don’t recommend running without journal if extra care of filesystem reliability is needed. But it’s better than broken USB stick.
Here is information about ext4 features:
http://kernelnewbies.org/Ext4
I’m experimenting with using a CF card (with IDE adapter) as the only storage on my low volume server, an older laptop sans screen. The CF card costs a bit more than using a USB flash (on the plus side, it has better read performance). An SSD drive would be better value for money, but they seem to be mostly SATA, which defeats the purpose of using an old machine.
Don’t know for sure if the CF card has wear levelling (though I’m hoping). I’ve taken some steps to reduce the writes: turned off ext4 journalling, removed the swap, increased the commit delay to 10 minutes, and moved /tmp and /var/tmp into tmpfs systems. Many of the sector writes that remain are for the log files, but I’d prefer not to have these in RAM. I’ll see how it all goes; the machine does run cooler than it did with a hard drive.
Just an update on using a little 8GiB CF card in an IDE adaptor as the only storage on a low volume server. It’s been running continuously for over nine months now, and has done some 118GiB of drive read transfers and 25GiB of drive write transfers.
Probably only a viable option if someone wanted to resurrect an old machine for an appliance; for anything with a SATA connector, I figure that a real SSD drive would be the way to go.