Archives

Categories

Bizarre “No space left on device” error from Xen

What should have been a routine “remove DIMMs and run memtest until things work” procedure to solve a memory error became a lot more complex due to poor error handling in Xen.

The following error occured because the tdb database /var/lib/xenstored/tdb was corrupt. To fix it you must rm the file and kill the xenstored process (which will otherwise recreate the file with the same corrupt data). It took me a few hours to work this out.

# xm create -c smtp
Using config file “/etc/xen/smtp”.
Error: (28, ‘No space left on device, while writing /local/domain/0/backend/vbd/
18/768/online : 1’)

Strangely the command “/etc/init.d/xend restart” does not restart xenstore or xenconsole (which is why running “/etc/init.d/xend stop” before rm’ing the file didn’t do any good).

After fixing the above problem I encountered the following error condition. It seems that there is no support for restarting daemons such as xenstored. So I had to reboot the machine. After that it worked.

# xm create -c smtp
Using config file “/etc/xen/smtp”.
Error: Device 0 (vif) could not be connected. Hotplug scripts not working.

I’ve filed Debian bug 433780 about this.

The ABC and the Supposed Liberal Bias

A common meme is that the media is biased towards the political left. This meme was deliberately created to encourage well-meaning journalists to be more generous in their reporting of Neo-Cons in a misguided attempt at being fair and also as a mechanism for refuting criticism of any media reports that criticise Neo-Cons.

The ABC has been criticised for being biased towards the left as part of this attack on fair media and have recently demonstrated that they are in fact biased towards the Neo-Con ideas by airing a pseudo-documentary about climate change The Great Global Warming Swindle.

The fake documentary has been refuted by many scientists, here is one by George Monbiot published in The Age.

I wonder whether this blog post will get Google adverts related to Nostradamus and UFOs – I’m sure that the google search engine has worked out that cranks are interested in that fake-doco.

Religion and Cars

The Catholic Church has recently issued a set of 10 road commandments. Number 9 is “On the road, protect the more vulnerable party” which ties in with an article by Barney Zwartz published in the religious column of The Age about the spiritual issues related to 4WD cars.

Barney makes better points than the Catholic Church starting with “How can Christians living in suburban Melbourne justify getting around in huge four-wheel drives? It’s harder to imagine a more “me-first, up yours” statement to the rest of the world*“. He also links to a report by the Australian Academy of science showing that 4WDs are less safe for their owners (which fits the theory of some Christians that bad actions tend to get punished in this world as well as the next).

IEA predicts global oil crisis

The ABC reports that an oil shortage is predicted soon. The International Energy Agency predicts that oil supply problems will start within the next five years. Oil production has been dropping over the last two years (we have passed the “peak oil” point) and demand is steadily increasing. Regardless of wars oil prices will increase significantly in the near future.

According to the Australian Bureau of Statistics the average age of Australian cars was 10.7 years at the end of 1998. So the vast majority of the current fuel-hungry cars on Australian roads will still be in use in five years time – if the owners can afford them.

The government needs to start providing tax incentives for fuel efficient cars and to end tax incentives for inefficient cars. Currently 4WD vehicles have reduced taxes and the tax deduction for a business car is based on the engine size (get a less efficient car and get a bigger tax deduction).

The government claims to be protecting the economy (at the expense of the environment). But with the way things currently work the economy will be destroyed in less than five years.

Even though the government cares little for the environment (in spite of the effect that climate change is having on farmers) they need to do something if they want to fulfill their claims of protecting the economy!

Thanks to Chris Samuel for informing me of this.

Correspondent Inference Theory and the US

Bruce Schneier writes about Correspondent Inference Theory which deals with situations when the motives of an individual or group are inferred by the results of their actions. Both his article and the MIT article on which it is based only consider the results of terrorist actions against the US and allied countries.

I believe that this is a serious mistake by Bruce, the MIT people, and most people who write about terrorism. The most sensible writing about Terrorism is by Noam Chomsky. Noam considers the definition of Terrorism in both propaganda and literally. By the literal definition of terrorism the US government is responsible for more than it’s fair share of terrorist acts performed around the world.

There is no reason to believe that people in the Middle-East are any less intelligent than people in the US and Europe. It seems obvious that some of the people who’s countries are destroyed by violence sponsored by the US government will believe that the US is entirely inhabited by blood-thirsty monsters. The number of US citizens who realise what their government does and approve is very low as is the number of Muslims who know what Al Quaeda does and approve of it.

The US government claims that it wants democracy in the Middle-East, and Osama bin Laden claims to want the US military out of the Middle-East. If the US forces were withdrawn from Saudi Arabia then it would probably lead to a significant increase in democracy in the region (it couldn’t get any less democratic) – both sides could get what they claim to want.

The discussion of the MIT paper seems to be largely based on the fact that Correspondent Inference causes the US government (and other governments) to decrease the probability of doing anything that might meet the terrorist goals. But no-one has mentioned the possibility that the same may apply to the probability of non-state organisations doing anything that might meet the goals of the US government. The wars in Iraq and Iran have significantly decreased the capabilities of the US military, they can’t recruit enough new soldiers and the current soldiers have reduced effectiveness due to long tours of duty with short breaks. The US economy is stagnating partly due to the direct effects of financing the wars, partly due to the way the airline security theatre has hurt trade and tourism, and partly because everyone has been concentrating on other things instead of fixing the economy.

When two states have a war there is always the possibility of it being ended by a peace treaty or one side surrendering. With modern communications fighting can end in a matter of hours after a cease-fire has been arranged between states. But when non-state forces are involved things become much more difficult to manage. A state can make a deal with one non-state group only to discover that another non-state group (or a dissident faction within the original group) doesn’t like the treaty and continues fighting. With non-state terrorist acts connected to Al Quaeda in the US, the UK, Spain, and Indonesia (and more acts apparently planned in other countries) it’s obvious that we aren’t going to get a clean or quick solution to this problem.

It seems to me that the only way the US and allied countries can escape from Correspondent Inference is to withdraw from the Middle-East entirely. If the people of Iran or Palestine want to elect a government that you don’t like then let it go (that’s what democracy is about anyway). If a dictator seizes control of Iraq then either leave him in control or provide air-support to any province that wants to rebel and establish a democratic government. Either make a stand on the principle of support for freedom and democracy or do nothing on the principle of letting people in other countries sort out their own problems. An invasion for the wrong reasons might fool people on the other side of the world but is unlikely to fool many people who live in the target country.

Testing STONITH

One problem that I have had in configuring Heartbeat clusters is in performing a STONITH that originates outside the Heartbeat system.

STONITH was designed for the Heartbeat system to know when a node is not operating correctly (this can either be determined by the node itself or by other nodes in the network) and then force a hardware reset so that the non-functional node will not interfere with another node that is designated to take over the service.

However sometimes code that is called by Heartbeat will have more information about the state of the system than Heartbeat can access. For example if I have a service that accesses a filesystem on an external RAID then it’s common for the RAID to track who is accessing it. In some situations the RAID hardware has the ability to “fence” the access (so that when machine B mounts the filesystem machine A can no longer access it). In other situations the RAID may only be capable of informing the system that another machine is registered as the owner of the device. To solve this problem a machine that is to mount such a device must either prohibit the previous owner from accessing the device (which may be impossible or unreasonably difficult) or reset the previous owner.

Until recently I had been doing this by writing some code to extract the STONITH configuration from the CIB and call the stonith utility. The problem with this is that there is no requirement that every node be capable of performing a STONITH on every other node, and that even if every node is are designed to be capable of rebooting every other node a partial failure condition may restrict the set of nodes that are capable of performing a STONITH on the target.

Currently the recommended way of doing this is via the test program. Below is an example of the command used to reset the node node-1 with a timeout of 20000ms and the result of it being successfully completed. I have suggested that the Heartbeat developers make an official interface for doing this (rather than a test of the API) and I believe that this is being considered. In the mean time the following is the only way of doing it:

# /usr/lib/heartbeat/stonithdtest/apitest 1 node-1 20000 0
optype=1, node_name=node-1, result=0, node_list=node-0

A Free-Software Only Laptop

Mark Shuttleworth asks if people are interested in a high-end free-software laptop (it seems that Linspire is leading in the low-end free-software laptop stakes).

I am interested in such things. My last couple of laptops have been Thinkpad T series. They are reasonably light (not really heavy), are reasonably fast, have full-size keyboards and reasonable size screens (currently got a 1400×1050 screen on a 3yo laptop). Unfortunately for Mark I’m planning on making my current Thinkpad last for another three years.

The idea makes a lot of sense because laptops are not re-purposed very often. It’s quite common for a desktop machine or a server to be re-installed several times over it’s life – and often having significant hardware changes during the process. Laptops are extremely difficult for hardware upgrades to the degree that by the time people desire an upgrade it often makes sense to buy a new one. So having a BIOS that only supports Linux and prevents the machine from ever being used to run a lesser OS is not likely to reduce the utility of the machine.

The benefit of better Linux integration is that the greater degree of hardware control would decrease the power use and extend battery life.

Maybe in three years time I’ll buy a LinuxBIOS machine second-hand from a Ubuntu user.

License Fees for Music in Clubs

The Cyber Law Center has blogged about the Phonographic Performance Company of Australia Limited being permitted to charge night-club owners $1.05 per person per night for playing commercial music up from $0.07. The number of people is calculated on the maximum licensed capacity of the venue – not good if the venue isn’t full. Of course given the huge prices of drinks in clubs this probably makes no difference to the profit margins.

One thing that hasn’t been mentioned in any of the articles I’ve read is whether this applies to clubs that don’t play music from artists represented by the PPCA. If a club can freely play music produced under a Creative Commons license that permits commercial use then the fee increases would be good for the production of music. Also encouraging clubs to pay musicians to perform live is a good thing, most of the best musicians started out with live performances in bars.

The recording industry is the enemy of music. They under-pay artists and lock out competition giving us a succession of talent-less wage-slaves on the radio. It’s difficult to imagine any change other than Digital Restriction Management (DRM) that could make things worse.

Documentaries about Gifted Children

On several occasions I have watched part of a TV documentary on gifted children, but I have never been able to watch one completely because every one that I have seen has been offensively wrong.

One thing that they always seem to do is say that gifted children have special needs and often claim that they have problems socialising. This sounds quite reasonable, but if that’s the case then why would you make such children perform tricks on TV? Putting the children on TV is a very poor example of journalism and often of parenting – the parents’ desire to boast about their children’s performance (and implicitly their own parenting skills) is apparently more important than protecting the children. I think that the only situation in which gifted children should have their talents demonstrated on TV is if their skill is related to the performing arts. Not that TV coverage is necessarily good for children who have abilities related to performing (a casual scan of the news regarding adults in Hollywood shows the problems that people have dealing with fame), but it’s something that they will be driven to anyway.

Also if they are going to demonstrate the intelligence of a child on TV then they really should make sure that they don’t demonstrate a lack of intelligence (on the part of the child as well as the producers). Asking a child to provide a definition of a word is often used as an example of intelligence (when it’s really an example of vocabulary). In one documentary a child defined a philanthropist as “someone who has a lot of money” (according to dict on my system it is “Love to mankind; benevolence toward the whole human family; universal good will; desire and readiness to do good to all men“). In another a child defined a genius as “someone who knows a lot“, while the definition of genius is not clear and there is some disagreement about what it is, most people agree that it’s about ability not knowledge. Being able to recognise when you don’t know something and admit would surely be correlated with intelligence…

Telling a child that they are a genius or telling them their IQ seems like a bad idea at the best of times, and doing so in front of a documentary camera crew isn’t the best of times. Children are able to determine how their skills compare to others, there are more than enough attitude problems in schools related to skill comparisons without encouragement from adults. When I was in high school a friend who studied a martial art refused to tell me which belt he wore – he had been taught that such things shouldn’t be discussed outside the dojo to avoid creating a hierarchy based on belt rating in the community. I think that the same thing could be applied to IQ ratings.

The term genius is grossly overused generally, I believe that showing greater than average ability in one area is not enough to qualify. I think that the minimum criteria would be to produce dramatic new developments/inventions in one area of research (EG Albert Einstein or Stephen Hawking) or to advance multiple fields of science or art (EG Leonardo da Vinci) would be the minimum criteria. I had people call me a genius when I was young because I won prizes in maths competitions, was good at programming computers, playing chess, etc. The main criteria for achieving such things is to avoid wasting time on sport – in a school run by idiots with a focus on sport this was more a tribute to being stubborn than being smart!

The TV documentaries mention many things that gifted children supposedly require, but strangely I don’t recall any of them mentioning the need to meet adults who are significantly smarter than average! You might think that this is the most obvious thing. The Big Brother and Big Sister programs of mentoring children who are at risk of crime and drugs apparently provide significant benefits, something similar for gifted children might do some good, probably more based on meetings of a few intelligent adults with a small group of intelligent children instead of the BB-BS model of one-on-one meetings. Washingtonienne seems like a good example of the need for this.

There are some useful print articles however, this article in The Age makes some good points, it’s particularly interesting to note that some schools lie about special programs to support gifted children to attract students (someone should name the schools that do this).

But then you get awful ones like this, naming the child in such a situation is irresponsible journalism. The girl in question may be forced to change her name to escape google when she’s older. It’s strange that it’s illegal to name a child who is involved in a court case but it’s not illegal to name them in such an article (let’s hope that the journalist used pseudonyms). Also you might expect an organisation such as Mensa to have someone smart enough to realise that bringing such attention on a 2yo is not in the child’s best interests.

Does anyone know of a good TV documentary about gifted children? I guess that it would be extremely difficult to make one without showing the children on TV so that would restrict the film maker to children who’s abilities are related to the performing arts.

Desktop Machines and ECC RAM

In a comment on my post about memory errors Chris Samuel referred me to an interesting post on the Beowulf mailing list about memory errors. In that list posting Joe Landman says “it is pretty easy to deduce which chip is problematic (assuming it is ram) based upon the address” and then describes how to use Machine Check Exception (MCE) data from an error detected/corrected by the ECC system.

Damn the vendors of motherboards for switching to 8-bit RAM just when it was about to be useful to have 9-bit RAM!

286 class machines had 9 bits of RAM per byte with one bit used for parity. Parity errors were extremely rare, largely due to the fact that memory errors could affect more than one bit at a time and therefore would often give a correct parity – if multiple bit errors were totally random then parity might be expected to pass 50% of the time! The Pentium was the first commonly used CPU to operate with a 64bit memory bus. If it had 9 bits per byte it would have had 72 bit wide memory buses – a Hamming Code could use this to detect and correct single-bit errors, detect all double-bit errors, and detect some errors involving more bits. This would mean that some errors would be recoverable and would display the location of the memory problem instead of being fatal and giving no information.

Now it’s become a standard feature in servers to have ECC memory (at significantly greater cost) and most desktop machines don’t have ECC support – I wonder whether this is aimed at price-gouging people who need reliable servers (they can’t use cheap RAM from desktop machines).

Unfortunately due to issues of electricity use, noise, and price I have to run all the servers that are most important to me on desktop PC hardware. Is anyone selling ECC RAM in desktop systems? I am particularly interested in machines that are a couple of years old so I can get them cheap at auction…