DomainKeys and OpenSSL have Defeated Me

July 3, 2009 | etbe

I have previously written about an error that valgrind reported in the STL when some string operations were performed by the DKIM library [1]. This turned out to be a bug, Jonathan Wakely filed GCC bug report #40518 [2] about it, Jonathan is one of many very skillful people who commented on that post.

deb http://www.coker.com.au lenny gcc

I’m still not sure whether that bug could actually harm my program, Nathan Myers strongly suggested that it would not impact the correct functionality of the program but mentioned a possible performance issue (which will hurt me as the target platform is 8 or 12 core systems). Jaymz Julian seems to believe that the STL code in question can lead to incorrect operation and suggested stlport as an alternative. As I’m not taking any chances I built GCC with a patch from Jonathan’s bug report for my development machines and then built libdkim with that GCC. I created the above APT repository for my patched GCC packages. I also included version 3.4.1 of Valgrind (back-ported from Debian/Unstable) in that repository.

Nathan Myers also wrote: “Any program that calls strtok() even once may be flagged as buggy regardless of any thread safety issues. Use of strtok() (or strtok_r()) is a marker not unlike gets() of ill thought out coding.” I agree, I wrote a program to find such code and have eliminated all such code where it is called from my program [3].

I think it’s unfortunate that I have to rebuild all of GCC for a simple STL patch. My blog post about the issue of the size and time required to rebuild those packages [4] received some interesting comments, probably the most immediately useful one was to use --disable-bootstrap to get a faster GCC build, that was from Jonathan Wakely. Joe Buck noted that the source is available in smaller packages upstream, this is interesting, but unless the Debian developers package it in the same way I will have to work with the large Debian source packages.

I have filed many bug reports against the OpenSSL packages in Debian based on the errors reported by Valgrind [5]. I didn’t report all the issues related to error handling as there were too many. Now my program is often crashing when DomainKeys code is calling those error functions, so one of the many Valgrind/Helgrind issues I didn’t report may be the cause of my problems. But I can’t report too many bugs at once, I need to give people time to work on the current bug list first.

Another problem I have is that sometimes the libdkim code will trigger a libc assertion on malloc() or free() if DomainKeys code has been previously called. So it seems that the DomainKeys code (or maybe the OpenSSL code it calls) is corrupting the heap.

So I have given up on the idea of getting DomainKeys code working in a threaded environment. Whenever I need to validate a DomainKeys message my program will now fork a child process to do that. If it corrupts the heap while doing so it’s no big deal as the child process calls exit(0) after it has returned the result over a pipe. This causes a performance loss, but it appears that it’s less than 3 times slower which isn’t too bad. From a programming perspective this was fairly easy to implement because a thread of the main program prepares all the data and then the child process can operate on it – it would be a lot harder to implement such things on an OS which doesn’t have fork().

DomainKeys has been obsoleted by DKIM for some time, so all new deployments of signed email should be based on DKIM and systems that currently use DomainKeys should be migrating soon. So the performance loss on what is essentially a legacy feature shouldn’t impact the utility of my program.

I am considering uploading my libdomainkeys package to Debian. I’m not sure how useful it would be as DomainKeys is hopefully going away. But as I’ve done a lot of work on it already I’m happy to share if people are interested.

Thanks again for all the people who wrote great comments on my posts.

SE Linux Lenny Status Update

March 5, 2009 | etbe

I previously described four levels of SE Linux support on the desktop [1].

Last night I updated my APT repository of SE Linux packages for Lenny (as described on my document about installing SE Linux [2]). I included a new policy package that supports logging in to a graphical session via gdm in either unconfined_t or user_t. This covers all the functionality I described as target 2 (some restricted users). I have tested this to a moderate degree.

Target 3 was having all users restricted and no unconfined_t domain (the policy module unconfined.pp not being linked into the running policy). I had previously done a large part of the work towards that goal in preparation for running a SE Linux Play Machine (with public root password) [3] on Lenny – but until last night I had not published it. The combination of the policy needed to run with no unconfined_t domain and the policy to allow logging in as user_t via gdm should mean that a desktop system with gdm for graphical login that has no unconfined_t domain will work – but I have not tested this. So target 3 is likely to have been achieved, if testing reveals any problems in this regard then I’ll release another policy update.

So now the only remaining target is MLS.

Also I have been setting up a mail server with a MySQL database for user account data and using Courier-Maildrop for delivery, so I’ve written policy for that and also made some other improvements to the policy regarding complex mail servers.

Lenny Play Machine Online

February 17, 2009 | etbe

As Debian/Lenny has been released and the temperatures in my part of the world are no longer insanely hot I have put my SE Linux Play Machine [1] online again. It is running Debian/Lenny and is a Xen DomU on a Debian/Lenny Dom0.

To get this working I had to make a few more fixes to the SE Linux policy and will update my Lenny repository (as mentioned in my document on installing SE Linux on Lenny [2]) in the near future.

I have reformatted most of the text from the thanks.txt file on my Play Machine and put is online on my documents blog [3]. I have also graphed the logins to my Play Machine using Webalizer [4] with 1KB transfer in the graph meaning one minute of login time. Below is the Perl code I used to convert the output of “last -i” to what looks like an Apache log file, the program takes a single command-line parameter which indicates the year that the data is from (which is not included in last output) and takes the output of “last -i” on standard input and gives a web log on standard output.

#!/usr/bin/perl

my @output;

while(<STDIN>)
{
if(not $_ =~ /^root.*pts/)
{
next;
}
$_ =~ s/ +/ /g;
$_ =~ s/^root pts.[0-9]+ //;
chomp $_;
my @arr = split(' ', $_);
my $url = "/";
if($arr[6] =~ /crash/)
{
$url = "/crash";
}
my $t = $arr[7];
$t =~ s/[()]//g;
my @times = split(':', $t);
if($times[0] =~ /\+/)
{
my @hours = split('\+', $times[0]);
$t = $hours[0] * 24 * 60 + $hours[1] * 60 + $times[1];
}
else
{
$t = $times[0] * 60 + $times[1];
}
$t *= 1024;
if($t == 0)
{
$t = 1;
}
if(length($arr[3]) == 1)
{
$arr[3] = "0" . $arr[3];
}
$output[$#output + 1] = "$arr[0] – – [$arr[3]/$arr[2]/$ARGV[0]:$arr[4]:00 +0000] \"GET $url HTTP/1.0\" 200 $t \"-\"\n";
}

my $i;
for($i = $#output; $i > -1; $i--)
{
print $output[$i];
}

Xen and Lenny

February 16, 2009 | etbe

Debian GNU/Linux 5.0 AKA “Lenny” has just been released [1].

One of the features that is particularly noteworthy is that Xen has been updated and now works fully and correctly on the 2.6.26 kernel (see the Debian Wiki page about Xen for details [2]). This may not sound exciting, but I know that a lot of people put a lot of work into getting this going, and for a long time in Unstable it wasn’t working well. I’ve just upgraded three Xen servers from Etch to Lenny (actually one was Etch kernel with Lenny user-space), and they all worked!

Those three servers were all running the i386 architecture, the next thing to do is to try it out with the AMD64 architecture. One of my plans is to try the latest Debian kernel on the server I use in Germany, but I’ll try on a few other AMD64 machines first.

Status of SE Linux in Debian LCA 2009

January 20, 2009 | etbe

This morning I gave a talk at the Security mini-conf of LCA about the status of SE Linux in Debian. Here is a summary of the issues I covered:

General Status

In Lenny (the new release of Debian that will come out in a month or two) SE Linux is working well. Considerably better than in Debian/Etch. There is an installation document on my documents blog [1], it’s very easy, only two scripts need to be run with no parameters to do most of the work (5 commands in total). There is more detail on installing SE Linux in Lenny (and other issues) in the Debian Wiki [2].

The default configuration of SE Linux is “targeted”. Previously we had separate policy packages for “targeted” and “strict”, now they are configuration options for selinux-policy-default. It is also possible to have some users in the unconfined_t domain (like the “targeted” policy) and some in confined domains such as user_t. Changing to strict can be done one user at a time, this needs further documentation.

Backports

I maintain an APT repository of i386 and AMD64 packages for better SE Linux support. This includes libraries built to not need an executable stack (see my previous blog post for details [3]). It also includes i386 libraries that don’t need text relocations AKA execmod (see my blog post about why i386 must die for details [4]).

My Lenny repository includes policy packages before they appear in Testing as well as the packages that are modified to fix the execmod and executable stacks issues. I plan to maintain this repository for some time, at least as long as I am actively using Lenny, but the content will change.

I might back-port the newer upstream policy to Lenny at some later date. If I do this it will be near the time that Lenny+1 is released and I will put it in a different location to my current Lenny repository.

I am currently deciding what to do with packages from external repositories such as debian-multimedia (see my previous post for the background) [5]. I may have to create a separate repository for non-standard Debian packages which I then modify to better support SE Linux.

I also plan to build packages of Security Enhanced PostgreSQL [7] for Lenny and Lenny+1. After demonstrating it’s capabilities I will suggest that it be considered for Lenny+1.

Play Machine

I have been running a Play Machine (open root machine) [6] for most of the last seven years. In the near future (probably the week after LCA) I will upgrade it to Lenny. One thing that I didn’t mention is the fact that I plan to demonstrate other things such as SE-PostgreSQL in Play Machines.

Training

I have a Xen server that is used for my Play Machine, I will run it as a SE Linux training machine and grant temporary ownership of a DomU to anyone who wants to learn and have a document with a list of tasks to complete to learn about SE Linux. I might be able to get it online this week. If so then I’ll make it available first to LCA delegates.

I will also set up a Bittorrent server for a Xen image for anyone who wants to go through the same SE Linux training program on their own machine – this will allow them a greater time limit and also avoid contention for my server. Unfortunately I have some problems with BitTorrent, I would appreciate any advice about running a torrent tracker.

Post Lenny

SE PostgreSQL is an exciting new development that I want to get in Debian. Initially I will create my own APT repository for it and include it in my Lenny repository. Hopefully it will become a standard feature in Lenny+1.

Security Enhanced X (the X window access controls) is a significant security feature. I hope to have that in Lenny+1, but it might not be possible.

Debian Multimedia and SE Linux

January 12, 2009 | etbe

I have just had a need to install packages from Debian-Multimedia.org to correctly play .3gp files from my mobile phone (the stock Mplayer in Debian would not play the sound).

As part of getting this to work in a way that I like I rebuilt some packages so that shared objects would not demand an executable stack and added them to My SE Linux Etch repository [1]. The liblzo2-2 package is in Debian so I filed bug report #511479 against it. Not that I expect it to be fixed for Etch now that Lenny is about to be released. But it’s good to have the data in the bug tracking system for the benefit of all interested people.

The lame and xvidcore packages are only in the Debian Multimedia archive. I’ve sent email to the maintainer with patches. Not sure if he will accept them (again it’s not a good time for filing bug reports about Etch), but there’s no harm in sending them in.

The lame package also required execmod access, but I don’t have enough time to devote to this to fix it. For background information about execmod see my previous post [2].

See my previous post about executable stacks for more background information [3].

The next thing to do is to test this out in Lenny, hopefully I’ll get time to work on this tomorrow.

Per-process Namespaces – pam-namespace

December 13, 2008 | etbe

Mike writes about his work in using namespaces on Linux [1]. In 2006 I presented a paper titled “Polyinstantiation of directories in an SE Linux system” about this at the SAGE-AU conference [2].

Newer versions of the code in question has been included in Debian/Lenny. So if you want to use namespaces for a login session on a Lenny system you can do the following:
mkdir /tmp-inst
chmod 0 /tmp-inst
echo “/tmp /tmp-inst/ user root” >> /etc/security/namespace.conf
echo “session required pam_namespace.so” >> /etc/pam.d/common-session

Then every user will have their own unique /tmp and be unable to mess with other users.

If you want to use the shared-subtrees facility to have mount commands which don’t affect /tmp be propagated to other sessions then you need to have the following commands run at boot (maybe from /etc/rc.local):
mount â€“make-shared /
mount â€“bind /tmp /tmp
mount â€“make-private /tmp

The functionality in pam_namespace.so to use the SE Linux security context to instantiate the directory seems broken in Lenny. I’ll write a patch for this shortly.

While my paper is not particularly useful as documentation of pam_namespace.so (things changed after I wrote it), it does cover the threats that you face in terms of hostile use of /tmp and how namespaces may be used to solve them.

Getting Started with Amazon EC2

October 4, 2008 | etbe

The first thing you need to do to get started using the Amazon Elastic Compute Cloud (EC2) [1] is to install the tools to manage the service. The service is run in a client-server manner. You install the client software on your PC to manage the EC2 services that you use.

There are the AMI tools to manage the machine images [2] and the API tools to launch and manage instances [3].

The AMI tools come as both a ZIP file and an RPM package and contain Ruby code, while the API tools are written in Java and only come as a ZIP file.

There are no clear license documents that I have seen for any of the software in question, I recall seeing one mention on one of the many confusing web pages of the code being “proprietary” but nothing else. While it seems most likely (but far from certain) that Amazon owns the copyright to the code in question, there is no information on how the software may be used – apart from an implied license that if you are a paying EC2 customer then you can use the tools (as there is no other way to use EC2). If anyone can find a proper license agreement for this software then please let me know.

To get software working in the most desirable manner it needs to be packaged for the distribution on which it is going to be used, as I prefer to use Debian that means packaging it for Debian. Also when packaging the software you can fix some of the silly things that get included in software that is designed for non-packaged release (such as demanding that environment variables be set to specify where the software is installed). So I have built packages for Debian/Lenny for the benefit of myself and some friends and colleagues who use Debian and EC2.

As I can’t be sure of what Amazon would permit me to do with their code I have to assume that they don’t want me to publish Debian packages for the benefit of all Debian and Ubuntu users who are (or might become) EC2 customers. So instead I have published the .diff.gz files from my Debian/Lenny packages [4] to allow other people to build identical packages after downloading the source from Amazon. At the moment the packages are a little rough, and as I haven’t actually got an EC2 service running with them yet they may have some really bad bugs. But getting the software to basically work took more time than expected. So even if there happen to be some bugs that make it unusable in it’s current state (the code for determining where it looks for PEM files at best needs a feature enhancement and at worst may be broken at the moment) then it would still save people some time to use my packages and fix whatever needs fixing.

DKIM and Mailing Lists

October 3, 2008 | etbe

Currently we have a problem with the Debian list server and Gmail. Gmail signs all mail that it sends with both DKIM and DomainKeys (DomainKeys has been obsoleted by DKIM so most mail servers implement only one of the two standards although apart from space there is no reason not to use both). The Debian list servers change the message body without removing the signatures, and therefore send out mail with invalid signatures.

DKIM has an option to specify the length of the body part that it signs. If that option is used then an intermediate system can append data to the body without breaking the signature. This could be bad if a hostile party could intercept messages and append something damaging, but has the advantage that mailing list footers will not affect the signature. Of course if the list server modifies the Subject to include the list name in brackets at the start of the subject line then it will still break the signature. However Gmail is configured to not use the length field, and a Gmail user has no option to change this (AFAIK – if anyone knows how to make Gmail use the DKIM length field for their own account then please let me know).

I believe that the ideal functionality of a sending mail server would be to have the configuration of it’s DKIM milter allow specifying which addresses should have the length field used. For example I would like to have all mail sent to an address matching @lists\. have the length field used (as well as some other list servers that don’t match that naming scheme), and I would also like to be able to specify which recipient addresses should have no DKIM signatures (for example list servers that modify the subject line). I have filed Debian bug #500967 against the dkim-filter package requesting this feature [1].

For correct operation of a list server, the minimal functionality is implemented in the Mailman package in Lenny. That is to strip off DKIM and DomainKey signatures. The ideal functionality of a list server would be that for lists that are not configured to modify the Subject line it would leave a DKIM header that uses the length field and otherwise remove the DKIM header. I have filed Debian bug #500965 against lists.debian.org requesting that the configuration be changed to strip the headers in question in a similar manner [2] (the Debian list servers appear to use SmartList – I have not checked whether the latest version of SmartList does the right thing in this regard – if not it deserves another bug report).

I have also filed Debian bug report #500966 requesting that the list servers sign all outbound mail with DKIM [3]. I believe that protecting the integrity of the Debian mail infrastructure is important, preventing forged mail is a good thing, and that the small amount of CPU time needed for this is worth the effort.

Also the Debian project is in a position of leadership in the community. We should adopt new technologies that benefit society first to help encourage others to do the same and also to help find and fix bugs.

An Update on DKIM Signing and SE Linux Policy

September 21, 2008 | etbe

In my previous post about DKIM [1] I forgot to mention one critical item, how to get Postfix to actually talk to the DKIM milter. This wasn’t a bad thing because it turned out that I hadn’t got it right.

I had configured the DKIM milter on the same line as the milters for ClamAV and SpamAssassin – in the smtpd_milters section. This was fine for relaying outbound mail via my server but didn’t work for locally generated mail. For locally generated mail Postfix has a directive named non_smtpd_milters which you need to use. So it seems that a fully functional Postfix DKIM milter configuration requires adding the following two lines to /etc/postfix/main.cf:

smtpd_milters = unix:/var/run/dkim-filter/dkim-filter.sock
non_smtpd_milters = unix:/var/run/dkim-filter/dkim-filter.sock

This also required an update to the SE Linux policy. When I was working on setting up DKIM I also wrote SE Linux policy to allow it and also wrote policy for the ClamAV milter. That policy is now in Debian/Unstable and has been approved for Lenny. So I now need to build a new policy package that allows the non_smtpd_milter access to the DKIM milter and apply for it to be included in Lenny.

SE Linux in Lenny is going to be really good. I think that I’ve already made the SE Linux support in the pre-release (*) of Lenny significantly better than Etch plus all my extra updates is. More testers would be appreciated, and more people joining the coding would be appreciated even more.

(*) I use the term pre-release to refer to the fact that the Lenny repository is available for anyone to download packages.

[1] http://etbe.coker.com.au/2008/09/18/installing-dkim-postfix-debian/

etbe – Russell Coker

Linux, politics, and other interesting things

Tag Archives: Debian