Lobbying for Free Software

March 17, 2009 | etbe

I am not aware of any Linux Users Group (LUG) being active in informing it’s members of how the policies of the various political parties compare with regard to free software and the other issues that are of interest to most members. I believe that this is a grave mistake.

Shortly before an election there are many social groups that send lists of questions to all the parties. They ask about the policies the parties have in regard to the issues that they care about, and helpfully mention the number of members that will receive the response. This of course doesn’t mean that every member of the group in question will cast their vote in the same way, merely that they will take note of the answers.

The committee members of the parties in question will then decide how to answer the questions and whether policy should be tweaked to allow answers that the lobby groups will like. So this process not only helps members of a group make informed voting decisions related to issues that they care about, but it also helps political parties choose policies that are least offensive to the group in question.

Here is a draft of a list of questions that I think should be asked of all political parties on behalf of Linux users:

It is important for all citizens to access all government data without being forced to buy new software or hardware, open standards allow everyone to access the data with free software. Do you support the use of open standards for data on government web sites and other forms of electronic communication between government agencies and citizens?
For long term archival of records it is important that file formats remain readable. The only effective way of doing this is to use open file formats that are implemented in free software. Do you support mandating that all data submitted to government agencies (by citizens or corporations) be in open file formats wherever possible?
In these difficult economic times there is a great interest in keeping jobs in the country instead of sending money overseas. To what extent do you support the use of free software that is installed and managed by locals (keeping the money in the economy) instead of importing software at great taxpayer expense?
Commercial software has a limited support period, after that time has elapsed there is no further support and systems become increasingly unreliable. Do you support mandating that all systems relating to the emergency services run on free software to allow quality long-term support by local citizens?
There has been a lot of concern recently about the spread of child-porn. The best available evidence shows that insecure home PCs that run “Trojan Horse” programs are a key part of distributing it and other illegal material. Do you support the introduction of government programs to train parents in installing one of the more secure free operating systems on their home PC to protect their children?

This is just a rough draft. Obviously there needs to be local differences (EG don’t use point 3 in the US because MS brings money into the US economy).

Does anyone have any suggestions for other questions?

Question about a “Secure Filesystem”

March 16, 2009 | etbe

I have just been asked for advice about “secure filesystem” and decided to blog my answers.

The first issue is what is meant by “secure filesystem, that could either mean the ability to restrict file access (EG by supporting SE Linux security contexts and using SE Linux for file access control) or the ability to encrypt data in case the machine is stolen. For access control I recommend SE Linux of course. For encryption on a local machine I mostly use dm-crypt which is configured with the cryptsetup utility. I encrypt at the LVM logical volume level as it is common that there are some LVs that don’t need to be encrypted. For files that need extra encryption or files that are shared between machines I use GPG.

A question was asked about kernel vs user-space filesystem encryption. AES is in the kernel so there is no lack in terms of strong encryption there. Also performance is pretty good (in most cases the CPU is fast enough that the hard drive is the bottleneck). For fine grained encryption (such as some of the experimental filesystems that encrypt data separately for each user) user-space is probably the only way to go.

If you want servers to be “high-security level” and protected from “hackers or unauthorised people” then it’s difficult to offer any advice that is smaller than a text book. I suggest that if you have such questions then you should do one of two things. If you are running a corporate IT department then hire an expert who can help with determine your specific requirements and meet them. If you want to learn about computer security and run your own systems in the best way possible then read as much from the experts as possible.

If you are looking for a project to contribute to related to security then if you choose SE Linux I could offer some specific advice on things that need work. I suggest not deciding on whether to do “kernel level or user level” work up front, but decide first which area of security you want to work on and then select a project which fits – then you should be able to determine whether your skills are best suited to kernel or user space coding. As for whether developing a new filesystem is necessary, I will note that SE Linux works well on Ext3 and XFS, it has just become usable on JFFS2, and it will work on other newer filesystems in the near future. Adding SE Linux support to a filesystem is not a difficult task if the filesystem supports XATTRs. I believe that there is a lot of scope for other access control systems to be developed which use XATTRs for security labels.

I can’t advise on e-books. I generally don’t read books, I read blogs and papers. Anything that I read which I consider to be worth recommending will probably have a link from my blog.

Why Cyrus Sucks

March 15, 2009 | etbe

I’m in the middle of migrating a mail server away from the Cyrus mail store [1]. Cyrus provides a POP and IMAP server, a local delivery agent (accepting mail via LMTP). It is widely believed that Cyrus will give better performance than other mail stores, but according to a review by linux-magazin.de Dovecot and Courier deliver comparable (and sometimes better) performance [2].

The biggest problem with Cyrus is that it is totally incompatible with the Unix way. This wouldn’t be a problem if it would just work and if it would display reasonable error messages when it failed, but it doesn’t. It often refuses to work as desired, gives no good explanation, and it’s data structures can’t be easily manipulated. Dovecot [3] and Courier [4] use the Maildir++ format [5] (as well as many other programs). I have set up a system with Courier Maildrop and Dovecot for the IMAP server [6] and it works well – it’s good to have a choice! But also Maildir++ is reasonably well documented and is an extension to the well known Maildir format. This means that it’s easy to manipulate things if necessary, I can use mv to rename folders and rm to remove them.

Cyrus starts with a database (Berkeley DB file) of all folders in all mailboxes. Therefore it is not possible to move a user from one back-end server to another by merely copying all the files across and changing the LDAP (or whatever else contains the primary authentication data) to point to the new one. It also makes it impossible to add or remove folders by using maildirmake or rm -rf. The defined way of creating, deleting, and modifying mailboxes is through IMAP. One of the problems with this is that copying a mailbox from one server to another requires writing a program to open IMAP connections to both servers at once (tar piped through netcat is much faster and easier). Also if you need to rename a mailbox that contains many gigabytes of mail then it will be a time consuming process (as opposed to a fraction of a second for mv).

Cyrus has a tendency to break while Dovecot is documented as being self-healing and Cyrus also seems to cope well in the fact of a corrupted mail store. Even manually repairing problems with Cyrus is a painful exercise. The Cyrus mail store is also badly designed – and it’s design was worse for older filesystems (which were common when it was first released) than it is for modern ones. The top level of a Cyrus maildir contains all the messages in the INBOX stored one per file, as well as three files containing Cyrus indexes and sub-directories for each of the sub-folders. So if I want to discover what sub-folders a mailbox has then I can run ls and wait for it to stat every file in the directory or I can use an IMAP client (which takes more configuration time). As opposed to a Maildir++ store where every file that contains a message is stored in a folder subdirectory named “new“, “cur“, or “tmp” which means that I can run ls on the main directory of the mail store and get a short (and quick) result. Using tools such as ls to investigate the operation of a server is standard practice for a sysadmin, it should work well!

A finall disadvantage of Cyrus seems to have many small and annoying bugs (such as the reconstruct program not correctly recursing the sub folders). I guess it’s because not many people use Cyrus that such things don’t get fixed.

One trivial advantage of Cyrus is that by default it splits users into different sub-directories for the first letter of the account name. Dovecot supports using a hash of the user-name this is better than splitting by first-letter for performance (it gives a more equal distribution) but will make it slightly more difficult to manipulate the mail store by script. Ext3 can give decent performance without a two level directory structure for as many as 31,998 sub-directories (the maximum that it will support) due to directory indexing and Linux caching of dentries. There may be some other advantages of Cyrus, but I can’t think of them at the moment.

Here is a script I wrote to convert Cyrus mail boxes to Maildir++. To make this usable for a different site would require substituting a different domain name for example.com (or writing extra code to handle multiple domains) and inserting commands to modify a database or directory with the new server name. There is no chance of directly using this script on another system, but it should give some ideas for people performing similar tasks.
Continue reading →

Maildrop, IMAP, and Postfixadmin

March 10, 2009 | etbe

I have recently configured my mail server to use IMAP. I started doing this when I was attending Linux.conf.au so that I could read urgent mail using my EeePC while at the conference and then be able to deal with the more complex stuff using my laptop later on.

The next logical step is to have mail delivered to different folders in the IMAP account. While there are ways of doing this via the Subject and other header fields, my needs are not that great. All I need to do is to support user+extension@example.com going to a folder named extension in the user’s mail store. While changing my mail server I decided to install Postfixadmin at the same time.

My first attempt to use Maildrop was to put the following in the /etc/postfix/main.cf file:
mailbox_command = /usr/bin/maildrop -d mail -f “$SENDER” “$DOMAIN” “$USER” “$EXTENSION”

That seems to only work when you have local accounts, so I ended up setting fallback_transport = maildrop and then putting the following in /etc/postfix/master.cf:

maildrop unix – n n – – pipe flags=DRhu user=vmail argv=/usr/bin/maildrop -d vmail ${nexthop} ${user} ${extension}

Where vmail is a Unix account I created for storing mail. Then I added the following to /etc/postfix/main.cf. Some of these are probably redundant (such as the virtual_mailbox_base). The recipient limit is set to 1 because there are no command-line parameters for maildrop to support two recipients for the same message.
virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf
virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_gid_maps = static:2000
virtual_uid_maps = static:2000
virtual_mailbox_base = /mail
vmaildir_destination_recipient_limit = 1
virtual_transport = maildrop
maildrop_destination_recipient_limit = 1

The files /etc/postfix/mysql* all have fields user=, password=, hosts=, and dbname=. The queries in each of the files are as follows:
mysql_virtual_alias_maps.cf:query = SELECT goto FROM alias WHERE address='%s' AND active = 1
mysql_virtual_domains_maps.cf:query = SELECT domain FROM domain WHERE domain='%s'
mysql_virtual_mailbox_maps.cf:query = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1

The /etc/courier/maildroprc file has the following contents:

# log the maildrop actions
logfile "/var/log/maildrop.log"
#
# parameters 1, 2, and 3 are the domain, user, and extension
DOMAIN=tolower("$1")
USER=tolower("$2")
EXTENSION=tolower("$3")
DEFAULT="/mail/$DOMAIN/$USER"
#
# try making a backup (cc) copy of the mail but do not abort if it fails
exception {
cc "$DEFAULT/.backup/"
}
#
# try delivering to the extension folder but do not abort if it fails
exception {
if(length("$EXTENSION") != 0 && "$EXTENSION" ne "backup")
{
to "$DEFAULT/.$EXTENSION/"
}
}
#
# deliver to the main inbox if there is no folder matching the extension or if no extension is specified
to "$DEFAULT/"

Installing Postfixadmin [1] was another challenge entirely. One of the complications of this is that there is no Debian package for Lenny (it seems that there will be one in Squeeze – Lenny+1).

I found David Goodwin’s tutorial on installing Postfixadmin and lots of other things on Debian/Etch [2] to be a very useful resource. I look forward to seeing a Lenny version of that document.

Please let me know if you can think of any way to improve this.

I need an LMTP server

March 9, 2009 | etbe

I am working on a system where a front-end mail server sends mail to what it considers to be a LDA (Local Delivery Agent) which actually sends mail to a back-end server via LMTP. I can’t remove that fake LDA from the design because it does a bunch of business specific processing along the way.

I am working on changing the back-end from Cyrus to Dovecot. Currently the mail goes from the fake LDA to the Cyrus LMTP server. What I would like to do is to have an LMTP server run on the back-end machine that launches the Dovecot deliver program immediately and then returns an appropriate code.

So far I have been experimenting with having Postfix run on the back-end to use deliver as the real LDA. The first problem with this is that the mail will be written to the Postfix queue and then written to the mail store. Doubling the number of writes is a real problem for a system that is going to be write-bottlenecked – it would significantly increase the hardware costs. The second problem is that when an account goes over quota the back-end server would be generating a bounce message. I would prefer the front-end server to generate the bounce on an un-munged message.

Basically all I need is a simple daemon (which could even be launched from inetd) that talks LMTP (a very simple cut-down version of SMTP) and executes a single command to receive the data. It might be necessary to serialise running the delivery process, in which case the mail data would have to be stored in memory and there would need to be a semaphore around executing the delivery program.

Does anyone know of such a program? If not then I’ll have to write it myself (which shouldn’t be difficult) and GPL it. If I have to do that then I need a suitable name for it. Any suggestions would be appreciated.

The FAIL Meme

March 8, 2009 | etbe

One of the recent poor trends in mailing list discussions is to reply to a message with a comment such as “FAIL” or “EPIC FAIL“.

The FAIL meme has been around for a while and actually does some good in some situations, slate has a good article about it [1]. The first example cited in that article is that ‘when Ben Bernanke and Henry Paulson testified before the Senate banking committee last month about Paulson’s proposed bailout bill, a demonstrator in the audience held up an 8.5-by-11 piece of paper with one word scrawled on it in block letters: “FAIL.”‘. This is an effective form of political demonstration, short words generally work well on placards (if only because the letters can be larger and therefore read from a greater distance) and anyone can understand the meaning of “FAIL” in that context.

There are some blogs dedicated to publicising supposed failures, failblog.org and ShipmentOfFail.com are two examples. I cite these as supposed failures because some of the pictures that they contain are obviously staged. It’s basically an Internet equivalent of the “Funniest Home Videos” shows that I never watched because they were not particularly funny.

So using the word “FAIL” on it’s own can be an effective form of political protest and can be used for mildly amusing web sites. But where it falls down is when it’s applied to a discussion that involves people who are from different cultures or have different levels of background knowledge – which covers most mailing list discussions.

Something that might be obviously wrong to some people is often not obvious at all to others. For example being forced to reboot a computer for any reason other than a kernel upgrade seems obviously wrong to me (and to most people who use Linux or other Unix systems) but Windows users seem happy to reboot machines after applying patches or upgrades. So writing a message with “FAIL” as the only word in a discussion with Windows users would not be productive. It could however be reasonable to forward a link to a page on a Microsoft web site to Linux people for their amusement with “FAIL” as the only comment – anyone who would find the link in question amusing would require no more explanation.

Sometimes when in a debate someone will write a message that only says “FAIL“, this is a very unconvincing argument that will not convince the opposition or any onlookers.

Generally it seems that using “FAIL” in a discussion with other like-minded people when talking about someone outside your group for the purpose of amusement can be effective. But any other use is going to be a “FAIL“.

As a more general rule single-word messages seem to have little value apart from certain limited situations. I have identified the following seven scenarios where a single word message is useful. Can anyone think of any others?

Code review – someone posts code (or design for code) and people who like it will write “ACK” or something similar.
Arranging a meeting – the question “who wants to meet for lunch tomorrow” has “me” as a valid answer.
Voting – “yes” and “no” are valid answers for a poll, but a mailing list or forum probably isn’t the best place for it.
Citing an example to refute a claim – often a single word won’t be a great response but may be adequate to prove a point.
Answering a request for a recommendation – if asked to recommend a laptop I might say “Thinkpad” or if asked to recommend a server I might say “HP“. Both those answers are poor (I recommend EeePC for netbooks and Dell for small/cheap servers), so while such an answer would be useful it would be below my usual quality standards for email (I prefer to write at least two paragraphs explaining why I recommend something).
Informing people that something has been done by replying to a request with the word “Done“.
Agreeing to a contract or proposal with “OK” or “Yes“.

Update: I added another two reasonable uses of single word messages,

[1] http://www.slate.com/id/2202262/pagenum/all/

Case Sensitivity and Published Passwords

March 6, 2009 | etbe

When I first started running a SE Linux Play Machine [1] I used passwords such as “123456“. Then for a while I had “selinux” but when I created a T-shirt design (see the main Play Machine page for details) I changed the password to “SELINUX” because that is easier to read on a shirt.

Unfortunately the last time I rebuilt the Play Machine I used a password of “selinux“, some people worked this out and still logged in so I didn’t realise that anything was wrong until a comment was placed on my blog yesterday. So for the past three weeks or so some people have been finding themselves unable to login. The password is now “SELINUX” again, sorry for any inconvenience.

It’s a pity that I can’t make sshd a little less case sensitive for passwords. A PAM module to implement a caps-lock mode where the opposite case is tried would be useful for this case and some others too.

[1] http://www.coker.com.au/selinux/play.html

SE Linux Lenny Status Update

March 5, 2009 | etbe

I previously described four levels of SE Linux support on the desktop [1].

Last night I updated my APT repository of SE Linux packages for Lenny (as described on my document about installing SE Linux [2]). I included a new policy package that supports logging in to a graphical session via gdm in either unconfined_t or user_t. This covers all the functionality I described as target 2 (some restricted users). I have tested this to a moderate degree.

Target 3 was having all users restricted and no unconfined_t domain (the policy module unconfined.pp not being linked into the running policy). I had previously done a large part of the work towards that goal in preparation for running a SE Linux Play Machine (with public root password) [3] on Lenny – but until last night I had not published it. The combination of the policy needed to run with no unconfined_t domain and the policy to allow logging in as user_t via gdm should mean that a desktop system with gdm for graphical login that has no unconfined_t domain will work – but I have not tested this. So target 3 is likely to have been achieved, if testing reveals any problems in this regard then I’ll release another policy update.

So now the only remaining target is MLS.

Also I have been setting up a mail server with a MySQL database for user account data and using Courier-Maildrop for delivery, so I’ve written policy for that and also made some other improvements to the policy regarding complex mail servers.

You Have the Right to Not Search My Bag

March 4, 2009 | etbe

This afternoon I was in a Safeway/Woolworths store (an Australian supermarket chain) and the lady on the checkout asked to inspect my backpack on the way out. The conversation went as follows:
Checkout Lady: Can I inspect your bag?
Me: Sure. – I put my backpack on the counter
CL: Could you open it for me?
Me: It’s OK, you can do it.
CL: I’m not allowed to open your bag, can you open it?
Me: I don’t mind, you can open it.

We iterated over the last two lines a few times, when it became clear that no progress was going to be made I asked “Can I go now?” and left.

It seems rather pointless to demand to search someone’s bag if you are not permitted to open it. Not that they have any power to search bags anyway. I discussed this with a police officer about 20 years ago and was told that store staff can do nothing other than refuse to allow you into their store in future if you don’t agree to a bag search. Stores claim that it’s a condition of entry that your bag be searched, but apparently that was not enforceable. Of course the law could have changed recently, I guess it would only require a terrorist threat related to supermarket products (baking soda can make your bread rise explosively) to get the law changed.

The last time my bag was searched was when leaving a JB Hi-Fi store. I had a brand new EeePC (purchased from a different store) in one hand and a bag in the other. The EeePC was identical to ones that they had on display and they didn’t even ask about it. It seems hardly worth the effort of searching bags when anyone can carry out expensive gear in their hand without being questioned.

A Police SMS about Fire Risk

March 2, 2009 | etbe

My wife and I have each received SMS messages from “Vic.Police” that say:

Extreme weather expected tonight (Monday) & tomorrow. High wind & fire risk. Listen to the ABC local radio for emergency update. Do not reply to this message.

Presumably the police are trying to contact everyone in Victoria. The problem seems to be related to the high wind speed that is forecast, the temperature is only predicted to be 32C (as opposed to the 38C that they were forecasting a few days ago and the temperatures of 46C or more a few weeks ago).

The last reports were that the firefighters were still working on putting out fires, and the unclear news coverage seemed to suggest that some of the fires had been burning since the 7th of February. A day of extreme fire danger that starts without any fires would be bad enough, but starting with some fires that are already out of control is destined to give a very bad result.

Below is the link to my previous post about people trying to take advantage of a tragedy to benefit their own political causes. For anyone who wants to rail against abortion, homosexuality, or the Greens party, please show some decency and do so based on relevant facts and do it at an appropriate time. I suggest that anyone who writes later this week about ways to avoid bushfires should be careful to check their claims for accuracy and scientific evidence (hint – the CSIRO and NASA have published a lot of useful background information).

http://etbe.coker.com.au/2009/02/25/tragedy-and-profit/

etbe – Russell Coker

Linux, politics, and other interesting things