etbe – Russell Coker – Page 139 – Linux, politics, and other interesting things

blocking spam

There are two critical things that any anti-spam system must do, it must not lose email and it must not cause damage to the rest of the net.

To avoid losing email every message must be either accepted for delivery or the sender must be notified.

To avoid causing damage to the rest of the net spam should not be bounced to innocent third parties. To accept mail, process it, and then bounce messages that appear to be spam will result in spam being bounced to innocent third parties.

The only exception to these two conditions is for virus email which can be positively identified as being bad and therefore they can be silently discarded. For any other category of unwanted mail there is always a possibility of a false-positive and therefore the sender should be notified if the mail will not be accepted.

Therefore the only acceptable method of dealing with spam is to reject it at the SMTP protocol level. Currently I am not aware of any software that supports Bayesian filtering while the message is being received so that it can be rejected if it appears to be spam, it would be possible to do this (I could write the code myself if I had enough spare time) but AFAIK no-one has done it.

The most popular methods of recognising SPAM before it is accepted are through DNSBL lists (DNS based lists of IP addresses known to send SPAM), RHSBL lists (DNS based lists identifying domains that are known to be run by spammers), and Gray-listing (giving a transient error condition in the expectation that many spammers won’t try again).

Gray-listing is not effective enough to be used on it’s own, therefore DNSBL and RHSBL systems are required for a usable email system. Quickly reviewing the logs of some of my clients’ mail servers suggests that the DNSBL dnsbl.sorbs.net alone is stopping an average of 20 SPAMs per user per day! The SORBS system is designed to block open relays, machines that send mail to spam-trap addresses, and some other categories of obviously inappropriate use. The number of false-positives is very small. On average I add about one white-list entry per month, which isn’t much for the email for a dozen small companies. For every white list entry I have added I have known that the sender has had a SPAM problem. I have not had to add a white-list entry because of a DNSBL making a mistake, just because people want to receive mail from a system that also sends SPAM.

I was prompted to write about anti-spam measures by an ill-informed and unsubstantiated comment on my blog regarding DNSBL services.

If anyone wants to comment on this please feel free. But keep in mind that I have a lot of experience running mail servers including large ISPs with more than a million customers. The advice I give in terms of anti-spam measures concerns techniques that I have successfully used on ISPs of all sizes and that I have found to work well even when both ends use them. Make sure that you substantiate any comments you make and explain them clearly. Saying that something is stupid is not going to impress me when I’ve seen it work for over a million users.

a newbie question about SE Linux and anti-spam measures

An anti-spam measure that is used by a very small number of people is that of verifying the sender address by connecting to the sending mail server. For example when I send mail from russell@coker.com.au the receiving machine will connect to my mail server and see whether it accepts mail addressed to russell@coker.com.au and will reject my mail if that isn’t the case.

The problem with this is that if I try to send mail to someone who has their mail server listed as a SPAM source then their efforts to verify my email address will fail and then my message to them will bounce with a confusing error message. This means that if one of the two mail servers involved in the communication is listed in a DNSBL or RHSBL service then all communication will be impossible. There will not be an option for one person to say “please phone me on this number if you can’t send me an email”.

This happened recently when someone from Italy asked me a question about SE Linux. So I will answer here (maybe they read my blog). In any case the answer might be of general interest:

Firstly I have to note that I have a B.Sc degree and no post-graduate qualifications, so it is not accurate to address me as Dr. Coker.

The question is: Let’s imagine a user acquire root rights. Especially on Fedora Core, which modify su command to map it to sysadm_r role, couldn’t he/she simply disable SELinux, delete logs, and so on?

If a user obtains ultimate privileges then they can do all things including deleting logs etc.

One thing to note is that there is no need for any process other than kernel threads to have ultimate privs, it would be useful in some situations to make log files append-only for all processes and the SE Linux policy language supports this.

The nearest any release policy comes to implementing such things is the separation between sysadm_r and secadm_r in the MLS policy in recent versions of Fedora.

Also note that it is possible to configure a SE Linux policy that does not permit any process to request that a new policy be loaded, the policy files be changed on disk, or the use of programs such as debugfs. Using SE Linux to enforce a policy that can not be bypassed by anything less than booting from installation media is quite easy to achieve.

One idea that I had was to have GPG implemented in the system BIOS and have GPG checks performed on the kernel before it’s loaded (to verify that the kernel had not been modified). The kernel could be passed a decryption key for the root filesystem by the BIOS, and SE Linux would be enabled as soon as the root filesystem was mounted. Thus nothing less than disassembling the BIOS would allow a hostile person to access the data on the disk. This is all possible with technology that has been common for many years. I almost convincced a BIOS author to implement this in about 2002.

invasive vs inconvenient security

The recent news from the UK gives us an example of invasive security. Preventing passengers carrying on any hand luggage (even wallets) and frisking all of them is the type of treatment you expect for criminals and visitors to maximum security prisons. It’s not what you expect for people who are involved in routine (or what used to be routine) travel.

The security measures offered by SE Linux are sometimes described as invasive. I don’t believe that this is an accurate description. I admit that sometimes minor tweaks are required (such as setting the correct context of a file). But for most users (corporate users and typical home users) the distribution takes care of all this for them. A default Fedora install should just work for the typical home user and a default Red Hat Enterprise Linux install should just work for the corporate user.

The main reason that it’s so easy to use is that the default domain for user sessions and for daemons that are not specifically configured in the security policy is unconfined_t. This means that programs for which there is no policy and programs run from a user session do not have SE Linux access controls. The default configuration of SE Linux only restricts programs that are known to be at risk.

The most common case of SE Linux access controls causing inconvenience is the policy for Apache (the daemon with the most configuration options). There are a set of configuration options (known as booleans) that can be used to determine what aspects of Apache will be confined, generally it only takes a few minutes to determine and specify the correct settings to support the desired operation.

Next time you are being frisked at a UK or US airport and are facing the prospect of a long flight with books and all other forms of entertainment banned keep in mind that airlines have invasive security and should be avoided if possible. SE Linux offers security that is at most a minor inconvenience (usually not even noticed) and should be embraced.

the waste of closed lists

As I mentioned in my first post the amount of effort I’m prepared to invest in posting to a small group of people is limited. I don’t think that I am the only person with this opinion.

I also believe that the number of people who refuse to post to open lists is quite small, and that on many lists they aren’t the people who contribute much. I believe that they are outweighed in both number and contributions by the people who want open lists and who are unwilling to spend a large effort on posting to a closed list.

When posting to an open list you have to be concerned about your online reputation. Some lists are closed because of having NSFW content that people don’t want known by their colleagues and managers, I guess that this makes sense for some lists.

IMHO the only good reason for closed lists is for discussion of truly sensitive information. This ranges from security problems in software that have not yet been fixed to medical and psychiatric problems. There are many lists which should not be publicly archived, but for general discussion of computers there is no such motivation.

For a list with a primarily technical focus on answering basic questions secrecy does no good, it merely protects people who want to post off-topic messages and create pointless arguments about issues that they don’t understand.

My solution to some of these problems is to use this blog to comment on such things. I expect that my solution will also be adopted by other people on some of the closed lists that I use.

Also it has occurred to me that blogging about issues may improve the quality of list discussion. If instead of responding to a message in point-form you write an article about the general issue then it may reduce the level of personal dispute. I think it would be difficult to have a flame-war by blog.

Finally while on the topic I have to mention that I don’t believe in anonymous posting to technical forums. Any content that is worth having should come with someone’s name attached. IRC nicks etc are OK, but the person writing the content should be identifiable.

big and cheap USB flash devices

It’s often the case with technology that serious changes occur at a particular price or performance point in development. Something has small use until it can be developed to a certain combination of low price and high performance that everyone demands.

I believe that USB flash devices are going to be used for many interesting things starting about now. The reason is that 2G flash devices are now on sale for under $100. To be more precise 1G costs $45AU and
2G costs $85AU.

http://www.coker.com.au/hardware/usb.html

The above page on my web site has some background information on the performance of USB devices and the things that people are trying to do with them (including MS attempting to use them as cache).

One thing that has not been done much is to use USB for the main storage of a system. The OLPC machines have been designed to use only flash for storage as has the Familiar distribution for iPaQ PDAs (and probably several other Linux distributions of which I am not aware). But there are many other machines that could potentially use it. Firewall and router machines would work well. With 2G of storage you could even have a basic install of a workstation!

Some of the advantages of Flash for storage are that it uses small amounts of electricity, has no moving parts (can be dropped without damage), and has very low random access times. These are good things for firewalls and similar embedded devices.

An independent advantage of USB Flash is that it can be moved between machines with ease. Instead of moving a flash disk with your data files you can move a flash disk with your complete OS and applications!

The next thing I would like to do with USB devices is to install systems. Currently a CentOS or Red Hat Enterprise Linux install is just over 2G (I might be able to make a cut-down version that fits on a 2G flash device) and Fedora Core is over 3G. As Flash capacity goes up in powers of two I expect that soon the 4G flash devices will appear on the market and I will be able to do automated installs from Flash. This will be really convenient for my SE Linux hands-on training sessions as I like to have a quick way of re-installing a machine for when a student breaks it badly – I tell the students “play with things, experiment, break things now when no-one cares so that you can avoid breaking things at work”.

The final thing I would like to see is PCs shipped with the ability to boot from all manner of Flash devices (not just USB). I recently bought myself a new computer and it has a built-in capacity to read four different types of Flash modules for cameras etc. Unfortunately it was one of the few recent machines I’ve seen that won’t boot from USB Flash (the BIOS supported it but it didn’t work for unknown reasons). Hopefully the vendors will soon make machines that can boot from CF and other flash formats (the more format choices we have the better the prices will be).

wasted votes

In a mailing list to which I subscribe there is currently a discussion on US politics with the inevitable discussion of wasted votes. As I don’t want to waste my writing on this topic on a closed list I’m posting to my blog.

There is ongoing discussion on the topic of wasted votes. As a matter of principle, if a vote is considered to be wasted, then that should be considered a failure of the electoral system.

Having representatives for regions makes some sense in that a regional representative will have more interest in the region than a central government with no attachment to the region. I expect that representatives of regions were initially used because it was not feasible for people to vote for people that weren’t geographically local. Now there is no real requirement for geographical locality (only a very small fraction of the voters get to meet the person they are voting for anyway) but having a representative for a region still makes sense.

The requirement for a regional representative means that if you live in a region mostly filled with people who disagree with you then your vote won’t change much. For example I live in a strong Labor region so the REAL fight for the lower house seat (both state and federal) occurs in the Labor party room.

My vote for the senate counts as that is done on a state-wide basis. So of the two votes entered in one election one of them can be considered to not be wasted.

For the US system, the electoral college was developed in a time when it was impossible for the majority of voters to assess the presidential candidates, and it solved the requirements of those times reasonably well. Today it is quite easy to add up all the votes and use either a simple majority or the “Australian ballot”.

Currently there is some controversy over the actions of Senator Joe Lieberman who lost the support of his party and then immediately declared that he would stand as an independent candidate. I believe that this illustrates a failure of the electoral system. It should be possible to have multiple candidates from each party on the list. In the Australian system it is possible to do that, but as they are in random order on the voting cards no-one would be sure of which candidate of the winning party would get the seat unless there were actual reasons for preferring one candidate over another (which sadly often isn’t the case). This is good for voters (the minority of voters who care enough about internal party policies to prefer one party candidate over another should make the decision) but not good for the candidates who want a better chance of winning without actually demonstrating that they can represent their voters better than other candidates.

The Australian government system has nothing equivalent to the US presidential election. The prime minister is voted in by the members of parliament. So there is little chance of getting multiple candidates from one party contesting one position. For the US presidential election I think that the best thing to do would be to have an “Australian ballot” and permit multiple candidates from each party. For example you could have Bush and Cheney running as candidates for president with each promising to make the other their VP if they get elected. With the Australian ballot it wouldn’t matter if you put Bush and Cheney as the last two votes on your ticket, the order you use for them will still matter.

I think that with the US presidential and state governor elections there is enough knowledge of the candidates among the voters to make it worth-while for each of the major parties to run multiple candidates.

One of many advantages of having multiple candidates is that you might have real debates. If the main candidates from the two big parties have a set of strict rules for their debate that prevents any surprise then the people who are the less likely candidates from those parties (and who therefore have less to lose) could go for a no-holds-barred debate with a selection of random members of the public asking questions.

Of course none of this is likely to happen. Any serious change would have the potential to adversely affect at least one of the major parties, and any improvement would necessarily have a negative impact on most of the current politicians. Votes ARE being wasted, and most politicians seem to like it that way.