New Version of Memlockd

I’ve just released a new version of Memlockd, a daemon to lock essential files in RAM to increase the probability of recovering a system that is paging excessively [1].

The new features are:
Using Debian/Wheezy paths for shared objects on i386 and amd64.

Added a new config file option to not log file not found errors so we don’t see i386 errors on amd64 and amd64 errors on i386.

Added a systemd service file which I haven’t yet tested, but I won’t get to test it for a while so for the moment I’ve released it and hope that the person who submitted the file got it right and that my minor change didn’t break it.

Added a run-parts style config directory, default is /etc/memlock.d and now the config file uses a % to chain to another file or directory.

So I fixed all but one of the Debian bugs in time for Wheezy, provided that the systemd stuff works. If someone has time to test it with systemd for me then that would be great!

Another USB Flash Failure

I previously wrote about a failure of a USB flash device in my Internet gateway [1]. I have since had another failure in the same system, so both the original 4G devices are now dead. That’s two dead devices in 10 weeks. It could be that the USB devices that I got for free at an exhibition were just really cheap, I’m sure that they weren’t expecting them to be used in that way. The devices from the same batch which are used for their intended purpose (sneaker-net file sharing) are still working well. But in any case I’m not going to resume this experiment until warmer weather. At this time of year some extra heat dissipation from computer gear in my home is more like a feature and less like a bug.

The second USB device to fail appeared to have it’s failure in the Ext4 journal (the errors were reported at around sector 2000), I didn’t keep a record of the problem with the first device, but from memory I think it was much the same.

Rumor has it that cheap flash storage devices don’t implement wear-levelling to avoid patent infringement. If that rumor is correct then any filesystem that uses a fixed journal in the same way as Ext3/4 is probably unsuitable for any serious use on such devices, while a filesystem based on Copy On Write will probably perform better. In Spring I’ll try using BTRFS on cheap USB flash devices and see if that works better. I have another spare device from the same batch to test so I can eliminate hardware differences. I can’t do enough tests to be a good statistical sample, but if a device lasts from Spring to Autumn using BTRFS with the same use that caused failures with Ext4 in a few weeks then I will consider it a strong indication that BTRFS is better than Ext3/4 for such uses.

For the next 5 months or so I’ll be using a hard drive in my Internet gateway system again.

Flash Storage Update

Last month I wrote about using USB flash storage devices for my firewall and Squid proxy [1]. 4 days ago it failed, the USB device used for the root filesystem stopped accepting write requests. The USB device which is used for /var/spool/squid is still going well after almost 5 months of intensive use while the USB device for the root filesystem failed after 24 days of light use. Both USB devices were of the same model and were obtained at the same time. Presumably one of them was just defective.

I’m now using an old 1G USB device for the root filesystem. When using it on less ancient systems with USB 2.0 there was an obvious speed difference between the 1GB and 4GB devices. But when run on USB 1.2 they can both support the maximum speed of the port so performance probably isn’t any worse. Not that it really matters for the root filesystem, the server is supposed to run without a break for long periods of time so if boot time becomes a performance issue then whatever is causing the reboots will be a much bigger problem.

It’s annoying to have a device fail and the failure rate for USB flash devices running 24*7 is looking rather bad at the moment. But I’m confident that things will run well from now on.

USB Flash Storage

For some years I have had my Internet gateway/firewall system in a cupboard in my bedroom. While I don’t mind some computer noise (I’ve slept near a server for most of the last 22 years) it’s good to have it as quiet as possible so getting rid of the hard drive is desirable.

I considered buying an IDE flash drive, but I’d like to continue my trend of not paying for hardware so I chose to use some USB flash devices that HP was giving away at a seminar (thanks HP – as an aside the P3 system is an old Compaq Desktop system). So I’ve got one 4G USB device for root and one for Squid.

For the past few months I’ve had /var/spool/squid be a USB flash device. I considered using RAID-0 for that filesystem because the computer is a P3 and only has USB 1.2 and thus a maximum theoretical transfer rate of 1.5MB/s and a maximum real-world rate of about 1MB/s. But my ADSL connection doesn’t seem able to sustain much more than 1MB/s and Squid doesn’t write data synchronously so in all my tests the USB speed hasn’t affected HTTP performance.

One issue that has delayed my move to all USB is the difficulty of booting as the P3 system in question doesn’t support booting from USB. I considered creating a boot CD that loads the kernel from the USB device, but that seemed a little painful and also relies on the CD-ROM drive working – which isn’t a great idea for a system that runs 24*7 in a dusty cupboard. I ended up using GRUB on the IDE hard drive to load the kernel and initrd and then mount a USB device as root, this seems to work and the command “hdparm -S6 /dev/sda” in /etc/rc.local makes the hard drive go to sleep once the system is booted.

The only technical parts of the process were putting in the UUIDs of the filesystems in /etc/fstab (because I can’t be sure which USB device will be found first) and creating a new initramfs with modules for USB storage listed in /etc/initramfs-tools/modules so that a USB device could be the root filesystem.

The firewall system is now a bit quieter and based on my last tests of hard drive power use will probably dissipate about 5-7W less heat. The next thing to do is wait and see if it keeps running or falls over. ;)

Cooling a Thinkpad

Late last year I wrote about the way that modern laptops suck [1]. One of the problems that inspired that post was the excessive heat generated by my Thinkpad T61.

There is a partial solution to this, Fool Control explains how the kernel option pcie_aspm=force can be used on kernels from 2.6.38 onwards to solve a heat regression problem [2]. I applied this to my Thinkpad T61 and the result was that on a cool evening (ambient temperature about 24C) the temperature changed from 85C to 66C on the NVidia video card, and for the “virtual devices” it changed from 80C and 78C to 60C and 61C. I’m not sure exactly what each of those measurements refers to, but it seems that the change was somewhere between 17C and 20C.

This changes the system from being almost unbearable to use to being merely annoyingly warm.

I’m not going to make my laptop be my primary computing device again though, the combination of a desktop system with a 27″ monitor and an Android phone is working quite well for me [3]. But I haven’t yet got version control systems working for all my software. Also Wouter suggested using NBD which is something I haven’t got working yet and probably won’t until I can swap on it and therefore have a diskless workstation. Finally I still haven’t got the “Chrome to Phone” browser extension working such that a page I’m viewing at home can be loaded on my phone.

Magic entries for BTRFS and Software RAID

I’ve just discovered that the magic database for the file(1) command in Debian/Unstable has no support for Linux Software RAID and that it’s support for BTRFS is lacking (no reporting of space used, number of devices, or the UUID). Below is my first draft of a change to fix these problems. I would appreciate it if someone with a big-endian system could test these out and let me know how they go, I suspect that I will have to change the “lelong” types to “long” but I’m not sure.

4096 lelong 0xa92b4efc Linux Software RAID
>4100 lelong x version 1.2 (%d)
>4112 belong x UUID=%8x:
>4116 belong x \b%8x:
>4120 belong x \b%8x:
>4124 belong x \b%8x
>4128 string x name=%s
>4168 lelong x level=%d
>4188 lelong x disks=%d

0 lelong 0xa92b4efc Linux Software RAID
>4 lelong x version 1.1 (%d)
>16 belong x UUID=%8x:
>20 belong x \b%8x:
>24 belong x \b%8x:
>28 belong x \b%8x
>32 string x name=%s
>72 lelong x level=%d
>92 lelong x disks=%d

# BTRFS
0x10040 string _BHRfS_M BTRFS Filesystem
>0x1012b string >\0 label "%s",
>0x10090 lelong x sectorsize %d,
>0x10094 lelong x nodesize %d,
>0x10098 lelong x leafsize %d,
>0x10020 belong x UUID=%8x-
>0x10024 beshort x \b%4x-
>0x10026 beshort x \b%4x-
>0x10028 beshort x \b%4x-
>0x1002a beshort x \b%4x
>0x1002c belong x \b%8x,
>0x10078 lequad x %lld/
>0x10070 lequad x \b%lld bytes used,
>0x10088 lequad x %lld devices

How to Start Learning Linux

I was asked for advice on how to start learning Linux. Rather than replying via email I’m writing a blog post for future people who ask such questions and also to get comments from other people which may provide information I missed.

Join a LUG

The best thing to do is to start by joining your local Linux Users Group (LUG). Linux International maintains a list of LUGs that is reasonably comprehensive [1]. Even if there isn’t a LUG near enough for you to attend meetings you can learn a lot from a mailing list of a LUG that’s close to your region. There is usually no great reason not to join the mailing list of a LUG in a different region or country, but a local LUG is that the advice will often be tailored to issues such as the local prices of hardware and the practices of your government.

Also note that Linux International doesn’t list all LUGs, the MLUG group in Melbourne [2] and the BLUG group in Ballarat [3] aren’t listed. Anyone who joins LUG (the group based in Melbourne, Victoria that I’m a member of) will be advised of the smaller groups in the region if they ask on the list.

As an aside it would probably make sense for the main LUV web page [4] to have links to local LUGs and to the LI page of users’ groups and for other LUGs to do the same. It’s pretty common for a Google search to turn up the web site of a LUG that’s near the ideal location but not quite right. Also it would be good if LUV could have a link to the Victorian Linux Users Group in Canada – this should reduce the confusion a bit and they have a link to us [5].

Play with Linux

Get a spare PC (with no important data) and try installing different distributions of Linux on it. Make sure that it never has anything particularly important so you can freely try things out without worrying about the risk of losing data. Part of the learning process usually involves breaking a system so badly that it needs to be reinstalled. Linux can run on really old hardware, an old system with 64M of RAM will do for learning (but 128M will really be preferred and 256M will be even better).

Learn with other Beginners

LUV has a very active beginners group, with a beginners mailing list and special beginners meetings. A group that has such things will be more helpful as you can easily learn from other people who are at a similar level to you. Also you can spend time learning Linux with friends, just spend a weekend with some friends who want to learn Linux and play with things – you can often learn more by trying things than by reading books etc.

Do some Programming

One of the advantages of Linux (and other Free Software OSs) is that it comes with a full range of programming languages for free. You can get a much greater understanding of an OS by writing programs for it and a typical Linux distribution gives you all the tools you need.

Any other Ideas?

Does anyone have any other suggestions? Please leave a comment.

Timing Processes

One thing that happens periodically is that I start a process from an interactive shell, discover that it takes longer than expected, and then want to know how long it took. Basically it’s a retrospective need to have run “time whatever” that I discover after the process has been running for long enough that I don’t want to restart it. My current procedure in such situations is to run ps from another session to discover when it started and then type date to display when it ends.

A quick test with strace showed that bash uses the wait4() system call to determine when a process ends, but passes NULL as the last parameter. If it passed the pointer to a struct rusage then it would have the necessary data.

I think it would be a really good feature for a shell to allow you to type something like “echo $TIME_LAST_CMD” to see how long the last command took. For the common case where you aren’t interested in that data it would only involve an extra parameter to the wait4() system call, a small amount of memory allocated for it, and to store yet another environment variable in it’s list.

A quick Google search didn’t show any way of filing wishlist bugs against Bash and I don’t think that this is a real bug as such so I haven’t filed a bug report. If anyone reads my blog and has some contact with the Bash people then please pass this idea along if you think it’s worthy of being included.

Flash, Apple, and Linux

Steve Jobs has published an interesting article about Flash [1]. He criticises Flash for being proprietary, this seems a little hypocritical coming from Apple (who’s the only competitor for Microsoft in terms of being the most proprietary computer company) but is in fact correct. Steve advocates HTML5 which is a better technical solution to a lot of the things that Flash does. He claims that Apple users aren’t missing out on much, but I think that sites such as Physics Games [2] demonstrate the benefits of Flash.

I think that Apple’s attack on Flash is generally a good thing. HTML5 web sites will work everywhere which will be a good incentive for web designers to fix their sites. I also think that we want to deprecate it, but as it’s unfortunately popular it’s useful to have tools such as GNASH to use Flash based web sites with free software. Microsoft has belatedly tried to compete with flash, but it’s Silverlight system and the free but patent encumbered Linux equivalent Moonlight have very little content to play and will probably disappear soon. As an aside the relentless determination of GNOME people to force the MONO project (including Moonlight) on it’s users convinced me to remove GNOME from all systems that I run.

OS News has a good analysis of the MPEG-LA patents [3] which are designed to prevent anyone making any commercial use of H.264 – which includes putting such videos on sites that contain Google advertising! These patent terms are so horrible that they want to control video streams that were ever encoded with them, so you can’t even transcode a H.264 stream to an open format without potentially having the scum at MPEG-LA going after you. This is worth noting when examining Apple’s actions, they support MPEG patents and therefore seem happy to do anything that reduces the freedom of their customers. Apple’s 1984 commercial has been proven to be a lie, it’s Apple that wants to control our freedom.

Charles Stross makes some good points about the issues related to Apple and Flash [4]. He believes that it’s all part of an Apple push to cloud computing and that Apple wants to own all our data at the back-end while providing a relatively reliable front-end (IE without all the anti-virus nonsense that is needed on the MS-Windows platform. Cloud computing is a good thing and I can’t wait for the Linux support for it to improve, I support a number of relatives who run Linux and it would be a lot easier for me if they could have the primary storage for everything be on the cloud so that I can do central backups of user data and they can use their own data while visiting each other. I think that a network filesystem that is similar in concept to offline-IMAP would be a really good thing, I know that there are some filesystems such as AFS and CODA that are designed for wide area network use with client-side caching but as far as I am aware they aren’t designed for the type of operation that offline/caching IMAP supports.

Matt Brubeck has given a good status report of the work porting Firefox to Android [5]. He notes that the next version of Fennec (mobile Firefox) will have Electrolysis – the Firefox one process per tab feature that was first implemented in Google Chrome [6]. I think that the development of Fennec and the one process per tab feature are both great developments. Matt also says “One of my personal goals is to make Firefox compatible with more mobile sites, and to give web developers the tools and information they need to make their sites work great in mobile Firefox. I’ll write much more about this in future articles“, that sounds great, I look forward to the results of his coding and to reading his blog posts about it!

Marshmallow Challenge for Linux Programmers

Tom Wujec gave an interesting TED talk about training people in team-work and engineering through building the tallest possible structures from 20 pieces of spaghetti, 1 yard of string, and 1 yard of sticky-tape with a time limit of 18 minutes [1]. The project is completed by groups of four people – which is probably about the maximum number of hands that you could have on such a small structure at one time. They have a web site MarshmallowChallenge.com/ which gives some suggestions for conducting a challenge.

One interesting point made in the talk is that kindergarten students tend to do better than most adults.

I think it would be good to have such challenges at events such as Linux conferences. The type of people who attend such conferences tend to enjoy such challenges, and it may lead to some lessons in team-work that can help the software development process. Also we can discover whether Linux programmers are better than the typical kindergarten students. ;)