2006 Open Source Symposium

Today (well yesterday as of 30 minutes ago) I spoke at the Open Source Symposium in Melbourne. This is an event sponsored by Red Hat. The first day was the business day and the second day was the Red Hat developers day.

I attended both days and spoke on the second day (today). My talk was about designing and implementing a secure system on Red Hat Enterprise Linux 4 (the Inumbers system for gatewaying SMS to email which is currently in Beta at the time of writing). I covered the issues of designing systems for least privilege via a set of cooperating processes under different UIDs. Secure coding principles, and SE Linux policy design. My presentation notes are HERE (in OpenOffice 2.0 format).

The talk seemed to be well accepted, so I’ll probably offer variations of it at other venues in the near future. I’m thinking of making a half-day workshop out of it.

While at the symposium one of the SGI guys mentioned that an XFS expert was in Melbourne temporarily. I suggested that such experts should be encouraged to give a talk about their work when they are in town. As a result of that I arranged a venue for a talk on XFS, I had the venue arranged in about 4 hours, which resulted in about 24 hours notice given to LUV members. I wasn’t able to attend the meeting due to prior commitments, so I’m not sure how it went.

fair trade is the Linux way

I have recently purchased a large quantity of fair trade chocolate. Fair trade means that the people who produce the products will be paid a fair price for their products which will enable them to send their children to school, pay for adequate health-care, etc. Paying a small price premium on products such as coffee and chocolate usually makes no notable difference to the living expenses of someone in a first-world country such as Australia, but can make a huge difference to the standard of living of the people who produce the products. Also fair-trade products are generally of a very high quality, you are paying for the best quality as well as the best conditions of the workers.

I will share this chocolate at the next LUV meeting, hopefully the people who attend will agree that the chocolate is both of a high quality as well as being good in principle and that they will want to buy it too.

The Fair Trade chocolate I bought cost $6.95 per 100g. I went to Safeway (local bulk food store with low prices) to get prices on other chocolate to compare. Lindt (cheaper Swiss chocolate) costs $3.09 per 100g and has a special of $2.54. The Lindt and the Fair Trade chocolate are both 70%, but the Fair Trade chocolate is significantly smoother, has a slightly better aroma, and a better after-taste. So the Fair Trade chocolate costs slightly more than twice as much as Lindt, but I believe that it has a quality to match the price. Then I compared the price of a cheap chocolate, Cadbury Old Gold chocolate is also 70% cocoa and costs $4.29 for 220g, this makes it between 3.5 and 4.4 times cheaper than the Fair Trade chocolate. But if you like chocolate then Cadbury products probably aren’t on the shopping list anyway. I believe that the Fair Trade chocolate I bought can be justified on the basis of flavor alone without regard to the ethical issues.

All Linux users know what it’s like to have their quality of life restricted by an oppressive monopoly. We are fortunate in that it only affects us in small ways, not in our ability to purchase adequate food and health care. As we oppose software monopolies that hurt us in the computer industry we must also oppose monopolies in the food industry that hurt people in third-world countries. The fair trade programs are the best way I know of doing that. Hopefully after tasting the chocolate many LUV members will want to buy it too.

Outsourcing – Bad for Corporations but Good for the World

There is ongoing discussion about whether outsourcing is good or bad. The general assumptions seem to be that it is bad for people who work in the computer industry (more competition for jobs and thus lower pay) and good for employers (more work done for less money).

I am not convinced that employers can get any benefit from outsourcing. The problem is that the pay rates for computer work are roughly proportional to the logarithm of the productivity of the person (at a rough estimation – it’s certainly not linear). Therefore if you get an employee on twice the base salary you might expect ten times the productivity, and an employee on three times the base salary could be expected to deliver one hundred times the productivity. These numbers may sound incredible to someone who has not done any technical work in the computer industry, but actually aren’t that exciting to people who regularly do the work. Someone who knows nothing may perform a repetitive task manually and waste a lot of time, someone who knows a little will write a program to automate it, and someone who knows a lot will write a program to automate it that won’t crash…

Programmers in Indian outsourcing companies are paid reasonably well by Indian standards, but they know that it’s possible to do a lot better. So all the best Indian programmers end up either migrating to a first-world country or running their own outsourcing company (there are a lot of great Indian programmers out there, but they aren’t working in sweat-shops). The Indians who actually end up doing the coding are not the most skilled Indian programmers.

It might be better to hire cheap Indian programmers of average skill than cheap first-world programmers of average skill. But hiring a single skilled programmer (from any country) rather than a team of average programmers will be a significant benefit (both in terms of price and productivity). In addition to this there are the communication problems that you experience with different time zones (the idea that one team can solve a problem while the team on another continent is asleep is a myth) and with different cultures.

I am not convinced that outsourcing does any real harm to good programmers in first-world countries. If someone does computer work strictly 9-5 and never does it for fun then they are not a serious programmer. People who aren’t serious about computers will probably be just as happy working in another industry if they get the same pay. Moving a few of the average computer programmer positions to India isn’t going to hurt anyone, especially as the industry is continually growing and therefore there is little risk of any given programmer being forced out of the industry. The people who are serious about computers (the ones who program for fun and would do it even if they weren’t paid to do so) are the most skilled programmers, they will always be able to find jobs. Will outsourcing reduce the income for such people? Maybe, but earning 5* the median income instead of 6* shouldn’t hurt them much.

The final question is whether outsourcing is a good thing. I think it is good even though it’s bad for first-world companies and not particularly good for programmers in first-world countries. Outsourcing benefits developing countries by injecting money into their economies and driving the development of a modern communications infrastructure (telephones, mobile phones, fast Internet access, reliable couriers, etc). I believe that the good which is being done in India by outsourcing money greatly exceeds the damage done to companies that use outsourcing services. Therefore I want this to continue and I also want to see outsourcing in other developing countries too. There is already a trend in outsourcing to eastern-European countries such as Russia, this is a good thing and I hope that it will continue.

terrorist “weakest link”

In the game show The Weakest Link competitors get voted off, usually not on whether they are weak but on whether the other contestents consider them to be a threat. It’s mildly amusing as a TV game show but not funny at all when carried out on an airline.

Recently a flight from Malaga to Manchester was delayed because two passengers were considered to be suspicious by other passengers (either 6 or 7 passengers refused to get on the plane because of this). The passengers were thought to be speaking Arabic (as if there was anyone on the plane who would recognise Arabic when they heard it) and because they were wearing coats and looking at their watches. The two men in question had been searched twice and found to be clean, but a bunch of idiots on a plane thought they knew better and demanded that the passengers in question be removed.

Lessons to be learned from this for travelling to/from coalition of the willing countries:

  1. Avoid the urge to check your watch when your flight is being delayed unless you are white. Non-white people who do what white people do in this situation are considered to be terrorists.
  2. When travelling to a cold place (such as Manchester) you want to have a coat to wear when getting off the plane. The airline staff won’t allow you enough hand-luggage space to store a coat so you will want to wear it when getting on the plane. This is fine if you are white, but if not white just deal with the fact that you will shiver when disembarking.
  3. Learn to speak English for your travels. If you speak another language you will be considered to be a terrorist.
  4. Whatever country you visit, stick to major cities as much as possible. Smaller cities have more racists and nationalistically bigoted people, there probably wouldn’t have been a problem on a flight to London.

Also just avoid the coalition of the willing countries in your travels as much as possible. There are much less problems in this regard when the government doesn’t depend on terrorism hysteria to justify going to war on the basis of lies.

car-pooling

I am constantly amazed at the apparent lack of interest in car-pooling when travelling between LUV meetings and the restaurant where we have dinner. After the last meeting I was one of the first five people to arrive at the restaurant and we had arrived in three separate cars. For the most luxurious travel you can have four people to a car and a standard sedan class vehicle can legally and safely carry five people. So an extra seven people could have been comfortably driven to the restaurant and an extra ten people could have been safely and legally driven to the restaurant. But instead most people were waiting in the cold at the tram stop.

Things are quite different in Europe. There was one occasion when after an LSM (Libre Software Meeting) conference in Bordeaux we got 8 people in a Mazda 323, now that’s what I call car-pooling! NB This is dangerous and illegal, so I can’t recommend doing it.

run an insecure system and get raped

After a recent mailing list discussion about computer security I’m going to be quoted in someone’s .sig so I think that I need to write a blog entry.

Here is an article about a 2001 case of a man who was arrested for pedophilia and spent 9 days in prison: http://www.xatrix.org/article.php?s=3549 .

This article on The Register has links to a few other articles and describes how a man has been found guilty due to the apparent actions of a hostile program on his machine (and served 20 days jail time).

Rumor has it that pedophiles are really disliked in prison and that they are often attacked by other prisoners. Even spending a few days in prison as a pedophile could be enough to get raped.

Run the latest version of the OS for your PC with all security patches. If you buy a second-hand machine reformat and reinstall as the first thing that you do just in case the last owner had kiddy porn (even though they may not have known of it).

laptop security on planes

There has been a lot of discussion recently about how to take laptops on planes following the supposed terror threat in the UK which has been debunked by The Register and other organizations. There is an interesting eWeek article about this that contains the interesting quote “The built-in locks don’t yet meet TSA specifications because they cannot be opened using the TSA master key” when reviewing a laptop case. Creating a master key is not that difficult and is explained in this PDF file. Theft by baggage handlers is quite a common occurance (see this google search for details).

So baggage handlers can easily reverse-engineer the TSA master key, steal laptops from baggage, smuggle drugs, and put bombs in baggage if they are so inclined.

There have been a number of cases of laptops containing sensitive financial, medical, and military data being stolen. Now someone who wants to steal data merely needs to work as a baggage handler and copy the hard drives of laptops before loading them. Data is more valuable if no-one knows that it has been stolen.

It would be ironic if an airline employee had their laptop hard drive copied and sensitive information about airport security was lost because of this.

more on anti-spam

In response to my last entry about anti-spam measures and the difficulty of blocking SPAM at the SMTP protocol level I received a few responses. Brian May pointed out that the exiscan-acl feature of Exim allows such blocking, and Johannes Berg referred me to his web site http://johannes.sipsolutions.net/Projects for information on how he implemented Exim SPAM blocking at the SMTP level.

It seems that this is not possible in Postfix at this time. The only way I know of to do this in Postfix would be to have a SMTP proxy in front of the Postfix server that implements the anti-SPAM features. I have considered doing this in the past but not had enough time.

Also a comment on my blog criticises SORBS for blocking Tor (an anonymous Internet system). As I don’t want to receive anonymous email and none of the companies I work for want to receive it either this is something I consider a feature not a bug!

blocking spam

There are two critical things that any anti-spam system must do, it must not lose email and it must not cause damage to the rest of the net.

To avoid losing email every message must be either accepted for delivery or the sender must be notified.

To avoid causing damage to the rest of the net spam should not be bounced to innocent third parties. To accept mail, process it, and then bounce messages that appear to be spam will result in spam being bounced to innocent third parties.

The only exception to these two conditions is for virus email which can be positively identified as being bad and therefore they can be silently discarded. For any other category of unwanted mail there is always a possibility of a false-positive and therefore the sender should be notified if the mail will not be accepted.

Therefore the only acceptable method of dealing with spam is to reject it at the SMTP protocol level. Currently I am not aware of any software that supports Bayesian filtering while the message is being received so that it can be rejected if it appears to be spam, it would be possible to do this (I could write the code myself if I had enough spare time) but AFAIK no-one has done it.

The most popular methods of recognising SPAM before it is accepted are through DNSBL lists (DNS based lists of IP addresses known to send SPAM), RHSBL lists (DNS based lists identifying domains that are known to be run by spammers), and Gray-listing (giving a transient error condition in the expectation that many spammers won’t try again).

Gray-listing is not effective enough to be used on it’s own, therefore DNSBL and RHSBL systems are required for a usable email system. Quickly reviewing the logs of some of my clients’ mail servers suggests that the DNSBL dnsbl.sorbs.net alone is stopping an average of 20 SPAMs per user per day! The SORBS system is designed to block open relays, machines that send mail to spam-trap addresses, and some other categories of obviously inappropriate use. The number of false-positives is very small. On average I add about one white-list entry per month, which isn’t much for the email for a dozen small companies. For every white list entry I have added I have known that the sender has had a SPAM problem. I have not had to add a white-list entry because of a DNSBL making a mistake, just because people want to receive mail from a system that also sends SPAM.

I was prompted to write about anti-spam measures by an ill-informed and unsubstantiated comment on my blog regarding DNSBL services.

If anyone wants to comment on this please feel free. But keep in mind that I have a lot of experience running mail servers including large ISPs with more than a million customers. The advice I give in terms of anti-spam measures concerns techniques that I have successfully used on ISPs of all sizes and that I have found to work well even when both ends use them. Make sure that you substantiate any comments you make and explain them clearly. Saying that something is stupid is not going to impress me when I’ve seen it work for over a million users.

a newbie question about SE Linux and anti-spam measures

An anti-spam measure that is used by a very small number of people is that of verifying the sender address by connecting to the sending mail server. For example when I send mail from russell@coker.com.au the receiving machine will connect to my mail server and see whether it accepts mail addressed to russell@coker.com.au and will reject my mail if that isn’t the case.

The problem with this is that if I try to send mail to someone who has their mail server listed as a SPAM source then their efforts to verify my email address will fail and then my message to them will bounce with a confusing error message. This means that if one of the two mail servers involved in the communication is listed in a DNSBL or RHSBL service then all communication will be impossible. There will not be an option for one person to say “please phone me on this number if you can’t send me an email”.

This happened recently when someone from Italy asked me a question about SE Linux. So I will answer here (maybe they read my blog). In any case the answer might be of general interest:

Firstly I have to note that I have a B.Sc degree and no post-graduate qualifications, so it is not accurate to address me as Dr. Coker.

The question is: Let’s imagine a user acquire root rights. Especially on Fedora Core, which modify su command to map it to sysadm_r role, couldn’t he/she simply disable SELinux, delete logs, and so on?

If a user obtains ultimate privileges then they can do all things including deleting logs etc.

One thing to note is that there is no need for any process other than kernel threads to have ultimate privs, it would be useful in some situations to make log files append-only for all processes and the SE Linux policy language supports this.

The nearest any release policy comes to implementing such things is the separation between sysadm_r and secadm_r in the MLS policy in recent versions of Fedora.

Also note that it is possible to configure a SE Linux policy that does not permit any process to request that a new policy be loaded, the policy files be changed on disk, or the use of programs such as debugfs. Using SE Linux to enforce a policy that can not be bypassed by anything less than booting from installation media is quite easy to achieve.

One idea that I had was to have GPG implemented in the system BIOS and have GPG checks performed on the kernel before it’s loaded (to verify that the kernel had not been modified). The kernel could be passed a decryption key for the root filesystem by the BIOS, and SE Linux would be enabled as soon as the root filesystem was mounted. Thus nothing less than disassembling the BIOS would allow a hostile person to access the data on the disk. This is all possible with technology that has been common for many years. I almost convincced a BIOS author to implement this in about 2002.