|
|
I want to use the Ext4 filesystem on Xen DomUs. The reason for this is that the problem of fsck times on ext4 (as described in my previous post about Ext4 [1]) is compounded if you have multiple DomUs running fsck at the same time.
One issue that makes this difficult is the fact that it is very important to be able to mount a DomU filesystem in the Dom0 and it is extremely useful to be able to fsck a DomU filesystem from a Dom0 (for example when you want to resize the root filesystem of the DomU).
I have Dom0 systems running CentOS5, RHEL5, and Debian/Lenny, and I have DomU systems running CentOS5, RHEL4, Debian/Lenny, and Debian/Unstable. So to get Ext4 support on all my Xen servers I need it for Debian/Lenny and RHEL4 (Debian/Unstable has full support for Ext4 and RHEL5 and CentOS5 have been updated to support it [2]).
The Debian kernel team apparently don’t plan to add kernel support for Ext4 in Lenny (they generally don’t do such things) and even backports.debian.org doesn’t have a version of e2fsprogs that supports ext4. So getting Lenny going with Ext4 requires a non-default kernel and a back-port of the utilities. In the past I’ve used CentOS and RHEL kernels to run Debian systems and that has worked reasonably well. I wouldn’t recommend doing so for a Dom0 or a non-virtual install, but for a DomU it works reasonably well and it’s not too difficult to recover from problems. So I have decided to upgrade most of my Lenny virtual machines to a CentOS 5 kernel.
When installing a CentOS 5 kernel to replace a Debian/Lenny kernel you have to use “console=tty0” as a kernel parameter instead of “xencons=tty“, you have to use /dev/xvc0 as the name of the terminal for running a getty (IE xvc0 is a parameter to getty) and you have to edit /etc/rc.local (or some other init script) to run “killall -9 nash-hotplug” as a nash process from the Red Hat initrd goes into an infinite loop. Of course upgrading a CentOS kernel on a Debian system is a little more inconvenient (I upgrade a CentOS DomU and then copy the kernel modules to the Debian DomUs and the vmlinuz and initrd to the Dom0).
The inconvenience of this can be an issue in an environment where multiple people are involved in running the systems, if a sysadmin who lacks skills or confidence takes over they may be afraid to upgrade the kernel to solve security issues. Also “apt-get dist-upgrade” won’t show that a CentOS kernel can be updated, so a little more management effort is required in tracking which machines need to be upgraded.
deb http://www.coker.com.au lenny misc
To backport the e2fsprogs package I first needed to backport util-linux, debhelper, libtool, xz-utils, base-files, and dpkg. This is the most significant and invasive back-port I’ve done. The above apt repository has all the packages for AMD64 and i386 architectures.
For a Debian system after the right kernel is installed and e2fsprogs (and it’s dependencies) are upgraded the command “tune2fs -O flex_bg,uninit_bg /dev/xvda” can be used to enable the ext4 filesystem. At the next reboot the system will prompt for the root password and allow you to manually run “e2fsck -y /dev/xvda” to do the real work of transitioning the filesystem (unlike Red Hat based distributions which do this automatically).
So the state of my Debian systems running this is that the DomUs run the CentOS kernel and my backported utilities while the Dom0 just runs the backported utilities with the Lenny kernel. Thus the Debian Dom0 can’t mount filesystems from the DomUs – which makes things very difficult when there is a problem that needs to be fixed in a DomU, I have to either mount the filesystem from another DomU or boot with “init=/bin/bash“.
For a long time it has been obvious that in all cases anti-piracy technologies discourage purchases and in many cases encourage piracy. I first discovered the significance of this in about 1991 when I attended a public lecture by a senior employee of Borland and a member of the audience claimed that the Borland product he bought didn’t function correctly due to anti-piracy measures. The Borland employee firmly stated that Borland did not use anti-copying technology on any of it’s products, didn’t have any plans to do so, and the problem in question must have been caused by something else. Of all the hostile questions that were asked, this was the only one that caused the speaker to appear agitated so it was obviously an issue that was considered to be important within Borland.
In the late 80’s anti-piracy measures were mostly based around creating floppy disks that couldn’t be easily copied (violations of various aspects of the disk formatting standards). This meant that you couldn’t make a backup copy of the data, so it wasn’t uncommon for people to seek pirate copies of their commercial software for daily use to avoid wearing out their valuable original floppy disks. Then the dongle was invented and people who bought software sometimes sought pirate copies so that they could use their printer and their commercial software without having to change plugs on their PC. But in those cases the benefits to uncrippled software to the users were small.
Now a large part of the battle on copy protection concerns DVDs. If you had a DVD of a recent movie and an MP4 which would you rather watch? Would you prefer to be forced to watch some anti-piracy rubbish for a couple of minutes at the start of the movie (with fast-forward disabled) or would you prefer to just start watching it? Would you prefer to be able to pre-program the sections of the movie that you watch (as some parents desire to skip the sex and/or violence in movies for their teenagers) or would you prefer to be forced to watch the movie straight-through with only a manual fast-forward to skip sections? Would you prefer to have a DVD that can’t be played properly on many (most?) computers because of the CSS encoding or an MP4 that plays on everything from PCs to mobile phones without an issue? Would you rather have 100 movies in the spare space on your laptop hard drive when you travel and 1000 movies on your desktop system or the much smaller number of boxed DVDs that you can store? I think that in most cases a pirate MP4 will give a better experience than a DVD.
So the question is, why pay for a DVD when in most cases you get a lesser experience than you will get from a MP4 file downloaded by bittorrent?
One reason for buying the DVD is to support the film industry. But I doubt that such a profitable industry will get much sympathy in today’s economy. Another reason is the morality, some people consider piracy to be theft (it isn’t – by definition theft requires that for at least a moment the property be completely in the possession of the thief) and therefore avoid it.
One technical reason for buying a DVD is the fact that it may have multiple languages supported, it will have subtitles, it may have an audio track with the creators giving a commentary, and it may have extra scenes that were cut from the main release. I believe that work on adding subtitles to the video file formats is a work in progress, so it’s only a matter of time before the DVD rips include all this extra data.
Really the content creators should focus on making a product that meets the needs of users and that they want to pay for. Pirating books is technically possible, but almost no-one does it. Some successful authors such as Charles Stross freely publish significant parts of their work and Cory Doctorow freely publishes all his work in electronic form. Books just work well, they meet the needs of users and people want to buy them. Sure they can sell them second hand, lend them to other people, and it’s technically possible to pirate them, but they remain profitable. On my documents blog I have a page of links to free short stories that I liked [1] and a page of links to free books [2]. It seems to me that creators of other copyright content should consider how they can be of service to their customers.
We are all familiar with corporations and misguided individuals who get whiny about the supposed losses due to piracy. Bruce Everiss has unfortunately joined this trend and demanded the disconnection of Internet users based on unproven accusations of game piracy [3]. I don’t know whether the game buying experience sucks as badly as the DVD buying experience, but based on the reports of locked-down consoles that have to be cracked before they run Linux I expect that the modern game industry is doing at least as badly as the movie industry. They need to provide things that users want!
One thing to note is that a Windows or console game player who uses pirate games will probably buy some games at some future time, while someone like me who uses free software both by principle and because it gives a better user experience will probably never pay for a game (I haven’t got time to play all the free games so I probably wouldn’t even buy a Linux game).
BidRivals.com is an interesting new auction site, their business model is that you buy “bids” for $0.80 each. If you want to bid on an item it costs you $0.80, then if you win the auction you pay for it. Every bid increases the price by 2 cents. So if you see an auction with a current price of $2.00 that apparently means 100 bids have been placed – IE $80 have been spent.
Currently a Canon EOS 50D Digital SLR camera + a lens is on auction, it has a “buy it now” price of $2,499 (the same price that Harvey Norman advertises) and the bidding is at $163 and climbing – so the auction site has apparently made almost $6,500 in revenue and the product has not been sold yet! The auction also has no fixed end time, it seems that about 10 or 15 seconds are added to the clock every time someone bids, and the autobidder will kick in when 4 seconds remain. So there will probably be an autobid every 6-11 seconds – somewhere between 300 and 600 bids per hour at a cost of $0.80 each – every item that is running with autobids will generate something between $240 and $480 in revenue without being sold!
Now if you don’t win the auction (or give up trying to win) then you can buy it for the difference between the cost of your bids and the product price. In the case of the camera in question if you had made 500 bids (which would be quite easy with the autobid feature) then that would be $400 worth of bids, and given a choice between losing that $400 and using it as a down-payment on the regular retail price I think that most people would choose to buy the product. Of course if you buy it from Harvey Norman you save the delivery fee and probably get more options if you want to return it – I’ve never had to return something to Harvey Norman but I’m assuming that it would be a lot easier than returning an item to an online store!
I don’t consider this to be a real auction site. I believe that a real auction has genuine bids of a value that is determined by the bidder, the auctioneer (or auction software) may decline bids that are too low in value or which have too small an increment. Bidding in an auction generally costs the bidder nothing – the only exception I’ve personally seen is auctions which have a printed catalogue in which case you pay for the catelogue, a fixed fee which is small when compared to the auction prices. It seems to me that a significant portion of the revenue (possibly the majority of the revenue) of BidRivals is from the bidding fees, and the other significant portion of the revenue would be comprised of profits made from auction customers who opt to buy the item at it’s list price to avoid wasting the money that they have put in to bids. The actual prices of the items are small by comparison.
Note that I am not accusing BidRivals of doing anything illegal (such as running a gambling system), I am merely stating that I don’t believe that they offer a good deal for customers. While they aren’t strictly a gambling site, it seems that one could get lucky and make a single bid (costing $0.80) at the right time and get a $2,499 camera for $170 (or whatever it ends up selling for) while others may spend hundreds of dollars in bids and get nothing other than a potential down-payment on the full $2,499 price. That’s a lot more luck than I want in any of my purchases!
I recommend not doing business with them or anyone like them.
I have just noticed that Red Hat added Ext4 support to RHEL-5 in kernel 2.6.18-110.el5. They also added a new package named e4fsprogs (a break from the e2fsprogs name that has been used for so long). Hopefully they will use a single package for utilities for Ext2/3/4 filesystems in RHEL-6 and not continue this package split. Using commands such as e4fsck and tune4fs is a minor inconvenience.
Converting a RHEL 5 or CentOS 5 system to Ext4 merely requires running the command “tune4fs -O flex_bg,uninit_bg /dev/WHATEVER” to enable Ext4 on the devices, editing /etc/fstab to change the filesystem type to ext4, running a command such as “mkinitrd -f /boot/initrd-2.6.18-164.9.1.el5xen.img 2.6.18-164.9.1.el5xen” to generate a new initrd with Ext4 support (which must be done after editing /etc/fstab), and then rebooting.
When the system is booted it will run fsck on the filesystems automatically – but not display progress reports which is rather disconcerting. The system will display “/ contains a file system with errors, check forced.” and apparently hang for a large amount of time. This is however slightly better than the situation on Debian/Unstable where upgrading to Ext4 results in an fsck error on boot which forces you to login in single user mode to run fsck [1] – which would be unpleasant if you don’t have convenient console access. Hopefully this will be fixed before Squeeze is released.
I now have a couple of my CentOS 5 DomUs running with Ext4, it seems to work well.
I bought the Bose QC15 noise canceling headphones for my trip back from the US. See my previous posts about Noise Canceling Headphones [1] and Testing Noise Canceling Headphones [2] for the details of my search.
I first tried my new headphones in my hotel room and they worked really well at blocking the noise from the nearby road (El Camino Real in Menlo Park) as well as the noise from the heater in my room. At the airport they entirely blocked the sound of the airport air conditioning system (which was surprisingly loud – I didn’t realise how loud until I tried the headphones).
On the flight the headphones worked really well. I used them for hours when they weren’t plugged in to any source, just stopping the noise was a huge benefit. I was also able to listen to music (both MP3s on my laptop and the plane sound system) at a relatively low volume with an apparent high quality. The Bose store I visited in Stanford mall has a sound system set up to emulate the noise you experience inside a jet to demonstrate what the headset can do. It really lives up to this demonstration! I recommend them to anyone who wants over-ear noise canceling headphones and can afford $US300 + tax.
But one thing to note is that not everyone likes such things – my wife didn’t like the sound that they generate (the least bad way to describe it is as a soft hiss). This is definitely not something you would want to buy based on reviews alone, it must be tested in-store.
The main technical suggestion I can make for improvement to the QC-15 is for it to have slightly softer and thicker padding where it contacts the sides of the wearer’s head. I find that my glasses prevent it from making as good contact as I would like, and that when wearing it while eating the contact is significantly broken with every jaw movement which is really annoying. A minor suggestion is that every pair of headphones should have the left and right ear pieces clearly marked, I really shouldn’t have to read an instruction manual to discover which way to wear it.
One thing that surprised me was the inclusion of business cards for the headset! Here is a picture:
The picture links to a larger picture that also shows the French version of the same text on the other side.
I was astonished by this, encouraging happy customers to help sell your products is a reasonable and effective form of product promotion (really this is what I’m doing for Bose with this blog post). But giving customers business cards is going too far – anyone who wants me to hand out their business cards can offer to pay me to do so (and I probably won’t accept). But if such things are considered to be a good idea then here are a few suggestions for other things that they could do:
- Create a Bose dating site where one can meet people who like music and traveling (this does sound appealing). In about 10 years the children of people who meet that way would start buying audio gear…
- Start a Bose fan club.
- Create a template that be used by a tattoo parlor to make a Bose tattoo.
- Sell Bose fan t-shirts to people who aren’t dedicated enough to get a tattoo.
- Register Bose as a religion, that gets tax free status among other benefits.
It’s a pity that Bose doesn’t make any water-proof noise canceling headphones. It would be something for their marketing people to wear while jumping over a shark on water skis [3].
But seriously the best thing that Bose could do to have their products promoted would be to start by printing the web site for each product on the item, my headset has two patent numbers listed which seem unlikely to provide any benefit for anyone, in that space they could have printed the global.bose.com/qc URL that is on the business card. Of course providing the URL really doesn’t do any good when the URL actually is useless. It starts by giving me a page asking which country I am in – the correct thing to do is to use geoip to determine the country and then give the user the option of selecting another country if that one is not ideal. Then after I select a country it doesn’t take me to a specific page for the product! I could have typed in www.bose.com and got the same result (in terms of US shopping at least) while typing six fewer characters!
Next like most corporate web sites the Bose site doesn’t appear to be configured for longevity of URLs – URLs which are clearly designed for the computer rather than humans are expected to change without warning. This discourages linking to any page that one might discover through web searches or navigating the site, and causes them to lose a lot of potential links.
Having specific URLs for all the products (including the obsolete ones) that are designed firstly for humans to read and write would be a good idea. It would be really useful to be able to compare the features of new products with the ones that are going cheap on eBay. For someone who is considering buying a new product now the purchase decision would be easier if they knew that the company would provide resources to help them get a better price on eBay in a few years if they want to upgrade to a newer model. One thing to keep in mind is the fact that the reputation of a company (which makes a dramatic impact on the prices customers are prepared to pay) depends largely on a long history of making quality products. Telling customers about those historic products is one of the most sensible things that most corporations fail to do on the Internet.
Some time ago I wrote a little utility named memlockd [1]. Memlockd will lock files into memory which allows significantly faster access when the system pages heavily, in my simulated tests I have found that having the programs and shared objects needed for logging in locked in memory can make it possible to login without a timeout when there is heavy paging, this can make the difference between recovering a system with some processes that are out of control and having to reboot it (often without discovering the root cause).
As always happens some people use my software in ways that I never planned. One guy is using it to try and make OpenOffice.org load faster. I’m not sure that this is a good idea. In a typical installation when configured for the purpose that I intended it (system recovery from a rabbit process) memlockd will take a bit less than 10M of RAM on an i386 platform (that is for bash, login, sshd, getty, busybox, and all necessary shared objects and a few data files. Since RHEL 4 Red Hat distributions have whinged at boot time if there was less than 256M of RAM available, installation of a Red Hat based system on anything less than 128M of RAM has been impossible for some years, and Debian systems perform very poorly with less than about 128M of RAM when you run apt-get. I initially designed memlockd to run on my SE Linux Play Machine which has 128M of RAM in it’s current incarnation. Locking 7.5% of RAM on the system may impact performance, but as a large part of that RAM is used for things like libc and bash which tend to be partially paged in at all times this shouldn’t be a noticeable impact. But locking 100M or more of OpenOffice seems more likely to have the potential to hurt performance, I often run OpenOffice on a machine with 512M of RAM and the biggest desktop machine I use has 1.5G of RAM – for me it wouldn’t make sense to lock OpenOffice into memory.
But it could be that there is some unusual aspect of his system that makes running memlockd with OpenOffice likely to give worthy benefits in performance without significantly hurting performance for other programs, for example it could have 4G of RAM and a really slow disk. It is also a possibility that the usage of the system makes OpenOffice so much more important than other programs that any decrease in performance in other areas is not relevant. In any case I’m happy to help people use my software to do unusual things so I’ll support this use.
I’ve been asked why memlockd doesn’t seem to give much benefit when starting up OpenOffice when run with “+/opt/openoffice.org3/program/soffice.bin” in the config file, where the + means to lock all shared objects that ldd reports that the binary needs.
for n in `ldd /usr/lib/openoffice/program/soffice.bin|sed -e s/^.*=..// -e s/\ .*// | sort -u` ; do readlink -f $n ; done | sort -u > ldd.txt
I used the above command to get a list of all shared objects that ldd reports for the soffice.bin program. On my system (Debian/Lenny i386) it reports 77 shared objects loaded. When memlockd locks all those ps reports that the RSS is 51944K.
cat /proc/1234/maps|sed -e s/^.*\ //|sort -u > /tmp/map.txt
Then I used the above command to get a list of the files that are memory mapped by OpenOffice when running OpenOffice calc where 1234 is the PID of the soffice.bin process (I expect that the numbers will be similar for writer, impress, etc – I just happened to have a spreadsheet open). It reports 172 memory mapped files which include 9 files related to fonts and 64 shared objects under /usr/lib/openoffice/program which are not found by ldd among other things. It’s quite common for a large application to use dlopen(3) at run-time to map shared objects instead of linking against them. Running memlockd with this list gave an RSS of 118644K, which is more likely to give a useful performance boost to OpenOffice load times.
I have returned from the US and my SE Linux Play Machine [1] is online again.
It was unfortunate that I forgot to pack one of my Play machine shirts, I ended up attending a meeting of the SDForum [2] on the topic of Cloud Security (it was a joint meeting of the Cloud Services and Security SIGs) and it would have been good to have been wearing a root password.
I was doing some work on NRPE (the Nagios Remote Plugin Executor) and I noticed bug report #547092 [1] which concerns the fact that the default configuration uses the same SSL certificate for all Debian servers and provides a patch to fix the problem. After building the patched package I followed the advice of the DebianAdministration.org article on creating self-signed SSL certificates [2].
cert_file=/etc/ssl/certs/FOO-cert.pem
privatekey_file=/etc/ssl/private/FOO-key.pem
cacert_file=/etc/ssl/certs/cacert.pem
Then I added the above lines to /etc/nagios/nrpe.cfg to instruct the nrpe to use the certificates.
For the Nagios server I had the problem that most of the systems I monitor run old versions of NRPE while only a few are recent Debian systems that allow me to easily install a new SSL checking nrpe. So I installed the following script as /usr/lib/nagios/plugins/check_nrpe to run either the old or the new check_nrpe:
#!/bin/sh -e
if echo $2 | egrep -q server0\|server2\|mail ; then
/usr/local/sbin/check_nrpe -C /etc/cert/cert.pem -k /etc/cert/key.pem -r /etc/cert/cacert.pem $*
else
/usr/lib/nagios/plugins/check_nrpe.orig $*
fi
The reason I started working on Nagios was to try and solve bug #560002 [3] which I filed. The bug concerns the fact that applications such as mailq which are run as part of Nagios checks were inheriting a TCP socket file handle from the nrpe. SE Linux prevents such file handles from being inherited, but it does mean that I get audit messages (and this is not a good case for a dontaudit rule).
Update:
One thing I forgot to mention is that the SSL key checking requires that the server common name used in the SSL certificate of the nrpe system matches the name that is used by the check_nrpe program. So if you check by IP address then you need to use the IP address in the certificate name – which is rather ugly. So I have moved to putting the hostname of each server in /etc/hosts on the NAGIOS system and using the hostname in the SSL certificate. This required using $HOSTNAME$ instead of $HOSTADDRESS$ in the NAGIOS configuration (thanks to John Slee for a tip in that regard).
Update2:
I removed some printf debugging from the script. It seems that I included a pre-production version of the script in the first version of this blog post.
Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1].
Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner.
Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be “skeptical” of the science should watch this.
Mark Peters wrote an interesting article “A Happy Writer Is a Lousy Writer” about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details.
CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5].
Cory Doctorow writes in the Guardian about Peter Mandelson’s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it.
Charles Stross has been blogging a series of non-fiction essays about space colonisation, in “The Myth of the Starship” he describes how most ideas of space travel are bad and how the word “ship” is always going to be unsuitable [7].
Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is “underwater” (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn’t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house.
Mako is working on a project to allow prisoners to blog [9]. It’s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access.
PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic “medicines” include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO.
Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he’s giving it away) [11]. He concludes by noting that he wants “no license agreement except ‘don’t violate copyright law’“. The fact that he can’t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is.
Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System – a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive.
Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don’t like social networking) his web site ShuffleBrain.com relies on Facebook.
Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that.
Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don’t cause TCP implementations to throttle the throughput. This seems similar to my observations of the “Three” network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).
Making Light has a post about the Canadian sci-fi author Peter Watts who was beaten and jailed overnight without access to a lawyer by US border guards because he asked what they were doing [1]. Apparently one is supposed to cringe in fear whenever questioned by authority in the US, so much for “land of the free“.
In 12 hours time I will hopefully have got past the security police and be boarding a flight out of the US and it will be a great relief.
On a practical note, if al Quaeda was a serious threat, if anyone really cared to stop the importation of drugs, of if any of the other reasons for having security on borders were taken seriously then this sort of thing really wouldn’t help to achieve such goals.
One of the purposes of a justice system is to encourage cooperation from the population. If a suspect will be treated fairly and considered to be innocent until proven guilty then there is no reason not to report suspicious people to the police. If however a suspect will be summarily beaten and denied access to a lawyer then no decent person can make a police report unless they are certain of the suspect’s guilt beyond all reasonable doubt.
When Barack Obama was elected the majority of the world’s population was greatly relieved, we expected some significant changes in the way things are run. It doesn’t seem that he is living up to our expectations.
|
|
