The Main Security Problem

May 15, 2009 | etbe

All security problems are to some degree people problems. Code may be buggy, but it was written by people who could have been better trained, had more time to spend on code review, etc. When there are multiple programs, OSs, libraries, etc to choose from then choosing a suitable combination of software is a matter of the skill and background knowledge of the people involved.

There are issues of software choice where there is no provable benefit of making one particular choice, EG choosing between a popular product that is OK and for which it is easy to hire skilled people to use it and a less popular product that has better security features but less public knowledge. But this is minor compared to other security problem.

I believe that the greatest security problem is stupid people. Stupid people in technical positions write buggy code and configure servers to be insecure. In consulting and analysis roles they develop bad procedures. In management they hire bad people to do technical work.

The vast majority of security problems can be fairly directly and immediately traced back to stupidity. In the corporate environment that is stupid programmers, stupid managers who hire people who are obviously stupid, and often stupid executives for mandating that software that everyone knows to be insecure should be used across the entire enterprise. In both the home and corporate environments there are a huge number of people who run machines that they know to be compromised. Apparently using a computer that is known to be under the control of an unknown hostile person is something that they don’t consider to be a problem – in spite of the obvious risks of fraud, data destruction, and risk of being implicated in crimes such as the distribution of child porn.

Email Passwords

April 27, 2009 | etbe

I was doing some routine sysadmin work for a client when I had to read mail in the system administration mailbox. This mailbox is used for cron job email, communication with ISPs that run servers for the company, and other important things. I noticed that the account was subscribed to some mailing lists related to system administration, the following is from one of the monthly messages from a list server:

Passwords for sysadmin@example.com:
List Password // URL
---- --------
whatever-users@example.org victoria3

That doesn’t seem terribly exciting, unless you know that the password used for the list server happens to be the same as the one used for POP and IMAP access to the account in question, and that it is available as webmail… Of course I didn’t put the real password in my blog post, I replaced it with something conceptually similar and equally difficult to guess (naturally I’ve changed the password). The fact that the password wasn’t a string of 8 semi-random letters and digits is not a good thing, but not really bad on it’s own. It’s only when the password gets used for 3rd party servers that you have a real problem.

I wonder how many list servers are run by unethical people who use the passwords to gain access to email accounts, and how many hostile parties use such lists of email addresses and passwords when they compromise servers that run mailing lists.

Now there would be an obvious security benefit to not having the list server store the password in clear-text or at least not send it out every month. Of course the down-side to doing that is that it doesn’t give someone like me the opportunity to discover the problem and change the password.

Amusing Thanks.txt Entry

April 21, 2009 | etbe

My SE Linux Play Machine [1] has a file named thanks.txt for users to send messages to me [2].

On a number of occasions people have offered to give me things in exchange for the password for the bofh account (the one with sysadm_r privileges). I’ve been offered stolen credit cards, a ponzi scheme of root access to servers on the net, and various other stuff. Today I received an amusing joke entry:

Hello Kind Sir,
I am Dr. Adamu Salaam, the the bank manager of bank of africa (BOA) Burkina Faso West
I am sending you this message about the $3.14159 million dollars in bank account number 2718281828450945. I will give you this money in exchange for the password to the ‘bofh’ account.

The amount of money is based on the value of Pi. The account number is based on the mathematical constant e [3].

It’s a pity that the author of that one didn’t sign their real name. Whoever created that should have claimed credit for their work.

Question about a “Secure Filesystem”

March 16, 2009 | etbe

I have just been asked for advice about “secure filesystem” and decided to blog my answers.

The first issue is what is meant by “secure filesystem, that could either mean the ability to restrict file access (EG by supporting SE Linux security contexts and using SE Linux for file access control) or the ability to encrypt data in case the machine is stolen. For access control I recommend SE Linux of course. For encryption on a local machine I mostly use dm-crypt which is configured with the cryptsetup utility. I encrypt at the LVM logical volume level as it is common that there are some LVs that don’t need to be encrypted. For files that need extra encryption or files that are shared between machines I use GPG.

A question was asked about kernel vs user-space filesystem encryption. AES is in the kernel so there is no lack in terms of strong encryption there. Also performance is pretty good (in most cases the CPU is fast enough that the hard drive is the bottleneck). For fine grained encryption (such as some of the experimental filesystems that encrypt data separately for each user) user-space is probably the only way to go.

If you want servers to be “high-security level” and protected from “hackers or unauthorised people” then it’s difficult to offer any advice that is smaller than a text book. I suggest that if you have such questions then you should do one of two things. If you are running a corporate IT department then hire an expert who can help with determine your specific requirements and meet them. If you want to learn about computer security and run your own systems in the best way possible then read as much from the experts as possible.

If you are looking for a project to contribute to related to security then if you choose SE Linux I could offer some specific advice on things that need work. I suggest not deciding on whether to do “kernel level or user level” work up front, but decide first which area of security you want to work on and then select a project which fits – then you should be able to determine whether your skills are best suited to kernel or user space coding. As for whether developing a new filesystem is necessary, I will note that SE Linux works well on Ext3 and XFS, it has just become usable on JFFS2, and it will work on other newer filesystems in the near future. Adding SE Linux support to a filesystem is not a difficult task if the filesystem supports XATTRs. I believe that there is a lot of scope for other access control systems to be developed which use XATTRs for security labels.

I can’t advise on e-books. I generally don’t read books, I read blogs and papers. Anything that I read which I consider to be worth recommending will probably have a link from my blog.

Case Sensitivity and Published Passwords

March 6, 2009 | etbe

When I first started running a SE Linux Play Machine [1] I used passwords such as “123456“. Then for a while I had “selinux” but when I created a T-shirt design (see the main Play Machine page for details) I changed the password to “SELINUX” because that is easier to read on a shirt.

Unfortunately the last time I rebuilt the Play Machine I used a password of “selinux“, some people worked this out and still logged in so I didn’t realise that anything was wrong until a comment was placed on my blog yesterday. So for the past three weeks or so some people have been finding themselves unable to login. The password is now “SELINUX” again, sorry for any inconvenience.

It’s a pity that I can’t make sshd a little less case sensitive for passwords. A PAM module to implement a caps-lock mode where the opposite case is tried would be useful for this case and some others too.

[1] http://www.coker.com.au/selinux/play.html

SE Linux Lenny Status Update

March 5, 2009 | etbe

I previously described four levels of SE Linux support on the desktop [1].

Last night I updated my APT repository of SE Linux packages for Lenny (as described on my document about installing SE Linux [2]). I included a new policy package that supports logging in to a graphical session via gdm in either unconfined_t or user_t. This covers all the functionality I described as target 2 (some restricted users). I have tested this to a moderate degree.

Target 3 was having all users restricted and no unconfined_t domain (the policy module unconfined.pp not being linked into the running policy). I had previously done a large part of the work towards that goal in preparation for running a SE Linux Play Machine (with public root password) [3] on Lenny – but until last night I had not published it. The combination of the policy needed to run with no unconfined_t domain and the policy to allow logging in as user_t via gdm should mean that a desktop system with gdm for graphical login that has no unconfined_t domain will work – but I have not tested this. So target 3 is likely to have been achieved, if testing reveals any problems in this regard then I’ll release another policy update.

So now the only remaining target is MLS.

Also I have been setting up a mail server with a MySQL database for user account data and using Courier-Maildrop for delivery, so I’ve written policy for that and also made some other improvements to the policy regarding complex mail servers.

Lenny Play Machine Online

February 17, 2009 | etbe

As Debian/Lenny has been released and the temperatures in my part of the world are no longer insanely hot I have put my SE Linux Play Machine [1] online again. It is running Debian/Lenny and is a Xen DomU on a Debian/Lenny Dom0.

To get this working I had to make a few more fixes to the SE Linux policy and will update my Lenny repository (as mentioned in my document on installing SE Linux on Lenny [2]) in the near future.

I have reformatted most of the text from the thanks.txt file on my Play Machine and put is online on my documents blog [3]. I have also graphed the logins to my Play Machine using Webalizer [4] with 1KB transfer in the graph meaning one minute of login time. Below is the Perl code I used to convert the output of “last -i” to what looks like an Apache log file, the program takes a single command-line parameter which indicates the year that the data is from (which is not included in last output) and takes the output of “last -i” on standard input and gives a web log on standard output.

#!/usr/bin/perl

my @output;

while(<STDIN>)
{
if(not $_ =~ /^root.*pts/)
{
next;
}
$_ =~ s/ +/ /g;
$_ =~ s/^root pts.[0-9]+ //;
chomp $_;
my @arr = split(' ', $_);
my $url = "/";
if($arr[6] =~ /crash/)
{
$url = "/crash";
}
my $t = $arr[7];
$t =~ s/[()]//g;
my @times = split(':', $t);
if($times[0] =~ /\+/)
{
my @hours = split('\+', $times[0]);
$t = $hours[0] * 24 * 60 + $hours[1] * 60 + $times[1];
}
else
{
$t = $times[0] * 60 + $times[1];
}
$t *= 1024;
if($t == 0)
{
$t = 1;
}
if(length($arr[3]) == 1)
{
$arr[3] = "0" . $arr[3];
}
$output[$#output + 1] = "$arr[0] – – [$arr[3]/$arr[2]/$ARGV[0]:$arr[4]:00 +0000] \"GET $url HTTP/1.0\" 200 $t \"-\"\n";
}

my $i;
for($i = $#output; $i > -1; $i--)
{
print $output[$i];
}

Do Spammers target Secondary MX Servers

February 14, 2009 | etbe

Rumour has it that some types of spammer target the secondary MX servers. The concept is that some people have less control over the secondary MX server and less ability to implement anti-spam measures. Therefore if they accept all mail from the secoondary then a spammer will have more success if they attack the secondary server.

True secondary servers are becoming increasingly uncommon, the lower priority servers listed in MX records tend to have the same configuration as the primary, so the benefit for the spammer in attacking the secondary server is probably minimal. But it would be good to know whether they do this.

I decided to analyse the logs from a mail server that I run to see if I can find evidence of this. I chose a server that I run for a client which has thousands of accounts and tens of thousands of messages delivered per day, my own server doesn’t get enugh traffic to give good results.

I analysed the logs for a week for the primary and secondary MX servers to see if the ratio of spam to ham differed. Now this does have some inherent inaccuracy, some spam will slip past the filters and occasionally a legitimate email will be rejected. But I believe that the accuracy required in a spam filter to avoid making the users scream is vastly greater than that which is required to give a noteworthy result.

I produced totals of the number of messages delivered, the number rejected by SpamAssassin (which has a number of proprietary additions), the number of message delivery attempts that were prevented due to rate limiting (most of which will be due to spammers), and the number of attempts to deliver to unknown accounts (some of which will be due to spammers having bad addresses in their lists).

For each of these rejection criteria I produced a ratio of the number of rejections to the number of delivered messages for each of the servers.

The rate limit count didn’t seem useful. While the primary server had a ratio of 0.75 messages rejected due to rate limiting to every message accepted the secondary had a ratio of 0.08. It seems that the secondary just didn’t get enough traffic to trigger the limits very often. This is an indication that the more aggressive bots might not be targetting the secondary.

The ratio of messages rejected by SpamAssassin to legitimate mail was 0.76:1 on the primary server and on the secondary server it was 1.24:1. The ratio of messages addressed to unknown users to successful deliveries was 3.05:1 on the primary and 7.00:1 on the secondary! This seems like strong evidence to show that some spammers are deliberately targetting the secondary server.

In this case both the primary and secondary servers are in server rooms hosted by the same major ISP in the same region. The traceroute between the two mail servers is only 7 hops, and there is only one hop between the two server rooms. So it seems unlikely that there would be some connectivity issue that prevents spammers from connecting to the primary.

One other factor that may be relevant is that the secondary server has been in service for some years while the primary is only a few months old. Spammers who store the server IP address with the email address (which happens – change the DNS records to send your mail to a different server and you will see some spam go to the old server) will be sending mail to what is now the secondary server. The difference in the rejected mail volume on the secondary server and the amount that would be rejected if it had the same ratio as the primary amounts to 7% of all mail rejected by SpamAssassin and 14% of all mail addressed to unknown users. I think it’s unlikely that any significant fraction of that would be due to spammers caching the server IP address for months after the DNS records were changed. So therefore it seems most likely that something between 7% and 14% of spam is specifically targetted at the secondary server.

While the ratio of spam to ham seems significantly worse on the secondary it is still a relatively small portion of the overall spam sent to the service. I had been considering setting up secondary mail servers with extra-strict anti-spam measures but the small portion of the overall spam that is targetted in such a way indicates to me that it is not going to be worth the effort.

Another thing that has occurred to me (which I have not yet had time to investigate) is the possibility that some spammers will send the same messages to all MX servers. If that happens then the ratio of spam to ham would increase every time the number of MX servers is increased. In that case it would make sense to minimise the number of MX servers to reduce the amount of CPU power devoted to runing SpamAssassin.

Note that I have intentionally not given any numbers for the amount of mail received by the service as it is a commercial secret.

Update: One thing I realised after publishing this post is that the secondary MX server is also the main server for mail sent between local users. While the number of users who send mail to other users on the service is probably a small portion of the overall traffic (it’s not a really big ISP) it will make a difference to the ratios. Therefore the ratio of spam to ham would be even worse on the secondary MX (assuming for the sake of discussion that local users aren’t spamming each other).

It’s too Hot in Melbourne

January 29, 2009 | etbe

The Bureau of Meteorology has forecast temperatures of 43, 43, and 35 for today and the next two days. Those temperatures are in celcius. Yesterday was also above 40C so my entire house is hot.

As my airconditioner is not overly large (a smaller unit is more efficient) the back part of my house will get really hot even without extra computers so I’m turning off my SE Linux Play Machine. Also a couple of years ago a SE Linux Play Machine died during summer in a similar situation, and I prefer not to lose hardware.

It will be on again in a few days.

SE-LAPP

January 22, 2009 | etbe

On Tuesday afternoon I gave a talk on behalf of KaiGai Kohei about SE Linux and the LAPP (Linux Apache, PostgreSQL, PHP/Perl) stack. KaiGai has blogged about this [1], unfortunately Google Translation does a poor job of Japanese and has particular problems with KaiGai’s work (could anyone who knows Japanese and English well please submit some tips to Google). KaiGai’s post is useful for links to his notes which are good background reading.

My talks about SE-LAPP and SE-PostgreSQL have been getting some notice, Bob Edwards referenced SE-PostgreSQL in his talk about database security.

It’s good to see KaiGai’s great work getting the notice that it deserves. I hope that it becomes a standard feature of the PostgreSQL code base in the near future!

Also Casey Schaufler, James Morris, and I have bought KaiGai a present of some Tasmanian wine, in recognition of his great work.

etbe – Russell Coker

Linux, politics, and other interesting things

Category Archives: Security