New Play Machine

October 28, 2009 | etbe

Update:
Thanks to Sven Joachim and Andrew Pollock for informing me about /etc/init.d/mountoverflowtmp which exists to mount a tmpfs named overflow if /tmp is full at boot time. It appears that the system was not compromised. But regular reinstalls are always a good thing.

On the 24th of August this year I noticed the following on my SE Linux Play Machine [1]:
root@play:/root# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda 1032088 938648 41012 96% /
tmpfs 51296 0 51296 0% /lib/init/rw
udev 10240 24 10216 1% /dev
tmpfs 51296 4 51292 1% /dev/shm
/dev/hdb 516040 17128 472700 4% /root
/dev/hdc 1024 8 1016 1% /tmp
overflow 1024 8 1016 1% /tmp

The kernel message log had the following:
[210511.546152] su[769]: segfault at 0 ip b7e324e3 sp bfa4b064
error 4 in libc-2.7.so[b7dbb000+158000]
[210561.527839] su[778]: segfault at 0 ip b7eb14e3 sp bfec84d4 error 4 in
libc-2.7.so[b7e3a000+158000]
[210585.270372] su[784]: segfault at 0 ip b7e044e3 sp bff1b534 error 4 in
libc-2.7.so[b7d8d000+158000]
[210595.855278] su[789]: segfault at 0 ip b7e014e3 sp bfd18324 error 4 in
libc-2.7.so[b7d8a000+158000]
[210639.496847] su[796]: segfault at 0 ip b7e874e3 sp bf99e7b4 error 4 in
libc-2.7.so[b7e10000+158000]

Naturally this doesn’t look good, the filesystem known as “overflow” indicates a real problem. It appears that the machine was compromised. So I’ve made archival copies of all the data and reinstalled it.

As the weather here is becoming warmer I’ve used new hardware for my new Play Machine. The old system was a 1.8GHz Celeron with 1280M of RAM and two IDE disks in a RAID-1 array. The new system is a P3-800 with 256M of RAM and a single IDE disk. It’s a Compaq Evo which runs from a laptop PSU and is particularly energy efficient and quiet. The down-side is that there is no space for a second disk and only one RAM socket so I’m limited to 256M – that’s just enough to run a Xen server with a single DomU.

I put the new play machine online on Friday the 23rd of October after almost two months of down-time.

[1] http://www.coker.com.au/selinux/play.html

Ownership of Laptops for Work

September 4, 2009 | etbe

Jetstar has announced some new changes to the way they manage their IT infrastructure [1]. Some parts of it are obvious things that people have been doing (or wanting to do) for a long time – such as using thin clients with no moving parts (not even cooling fans).

But the really interesting part is their plan for managing laptops. They are using a virtual machine image on a flash storage device that can run on any system. So deploying a new system will only require installing the virtual machine software and inserting a storage device. Moving a user’s environment to a different system (EG due to hardware failure) will merely require inserting the storage device in a new system.

That raises the issue of ownership of the device. It seems that Jetstar are considering using systems that are owned by employees, Stephen Tame said “In two years’ time a laptop should be a condition of employment, and this includes bringing your own laptop“. When introducing that I expect there would be some resistance by employees who don’t want to spend the money. However
I have previously estimated the costs of running a car [2] which works out to more than $1,650 per year for insurance, registration, basic maintenance, and the interest that would have been received if the car had not been purchased and the money had been invested. Laptops can be purchased for significantly less than $1000 (currently the EeePC 701 is on sale for $219) and can be expected to last for three years or more if you are careful to avoid damage and don’t run demanding software. So a job that demands ownership of a laptop is asking for a much smaller financial investment than one which demands ownership of a car. But I expect that many employees won’t see it that way.

The up-side for employees to bring their own laptops is that they can choose a model that suits their preference. Everyone has preferences regarding the size of keys on a keyboard, the distance that they travel and the pressure required to register a key-press. For desktop machines it’s easy to swap keyboards but for laptops there is no such option. Then there’s the issue of the trade-off between physical size and weight vs display resolution, personal preferences in this regard will depend to some extent on the body mass and strength of the employee.

Now there are a number of security issues related to personal laptop use. Obviously if the laptop has a Trojan-horse program installed then it could sniff any data that goes past on the network. The most trivial case of this could be addressed by running VPN software inside the emulated environment. This would force a Trojan to compromise the virtual environment (EG by modifying the address space) or to compromise the files on disk (insert a Trojan inside the filesystem for the virtual environment). The former would be tricky to get right while the latter would be trivial. Both attack methods have been used in the past and proven to work. This is why many companies prohibit their employees from connecting their own systems to the corporate network.

One example of a system that is based around running virtual machines for all desktop operations is the NSA NetTop project [3]. NetTop involves a SE Linux system that runs multiple instances of VMWare for different desktop environments. Each VMWare instance runs at a particular sensitivity level and uses a VPN connection to a back-end network running at the same level. The aim of NetTop is to prevent applications in the different VMWare instances from communicating with each other. The significant difference between a typical NetTop installation and what JetStar might be doing is that NetTop runs on a secure base – it’s hardware that has been purchased and installed by a military organisation and is run in a secure facility. While personal laptops that are owned by employees can be expected to be infected with viruses and Trojan-horse programs.

In the past I have suggested that an employment package for any skilled employee should include some budget for buying things that facilitate the work [4]. It seems to me that a company like JetStar could best achieve their goals by assigning a budget to each new employee to buy a machine for their use. The employee then gets to choose a machine up to that budget – which would only be for work purposes. Then when the employee leaves or the machine becomes due for replacement it could be sold at auction. When considering all the costs involved in hiring a new person, spending something less than $1,000 to buy a laptop is nothing.

Finally if buying machines for work purposes, you really don’t want employees using them for surfing porn. Porn sites tend to be particularly bad for malware distribution. To reduce the incidence of such problems I think that work machines should have their sound hardware disabled and laptops should not be purchased with overly large displays. There is no need to make work machines totally unsuitable for porn surfing (which would also make them less effective for work), but making them less suitable than a $500 budget PC should dramatically reduce the scope of the problem.

Free K-12 Text Books

August 29, 2009 | etbe

The CK12 project is developing free (CC by SA) textbooks for the K-12 market (with a current focus on the early years of high school) [1]. Their primary aim seems to be flex-books – text books that can be localised and modified to better suit the needs of the students. But of course there are many other benefits, according to my best estimates storing text books on an ebook reader or one of the lighter NetBooks is necessary to avoid childhooh back injuries [2].

Another major benefit of flexible text books is the possibility of teaching a wider range of subjects. A subject does not need the level of interest that is required to get a publishing contract (which generally means acceptance by the education department of a state) to have a text book. Independent schools and home-schoolers can select subjects that are not in the mainstream curriculum.

The information for potential authors of text books is here (they didn’t make it particularly easy to find) [3].

One thing I would like to see is a text book about computer security. I really don’t think that this would be an overly difficult subject for an 11yo who is interested in computers. When I was 11 I read a text book on nuclear physics in the form of a comic book, I don’t think that computer security is inherently more difficult or harder to teach than nuclear physics. Naturally full coverage would require several texts aimed at different ages. But that’s possible too. It would probably be easiest to start with an age of ~16. Also as computer security is a subject that is both difficult at one end of the scale and essential at the other it would be necessary to have A and B streams (as is done with maths in the Australian education system).

Please leave a comment if you are interested in participating in the development of computer security related text books. Incidentally it would be good to get a contributor who has had experience in teaching teenagers even if they don’t have any knowledge of computer security – I don’t expect to find someone with good technical skills and teaching experience.

Mail Server Security

August 20, 2009 | etbe

I predict that over the course of the next 10 years there will be more security problems discovered in Sendmail than in Postfix and Qmail combined. I predict that the Sendmail problems will be greater in number and severity.

I also predict that today’s versions of Postfix and Qmail will still be usable in 10 years time, there will be no remote security problems discovered other than DoS attacks.

I’ve been having arguments about MTA security with Sendmail fans for over 10 years. I would appreciate it if the Sendmail fans would publish their own predictions, then we can wait 10 years and see who is more accurate.

I don’t recommend using Qmail (Postfix is what I use). But I think he wrote code that is unlikely to be exploited.

The Lack of Browser Security

August 18, 2009 | etbe

For a long time the use of HTTP cookies [1] for tracking the web browsing habits of users has been well known. But I am not aware of any good solution to the problem. A large part of the problem is the needless use of cookies, it seems that many blog servers use cookies even though they provide no benefit to the user. A major culprit in this regard is the Google Analytics service which sets a cookie with a two year expiry time when you first visit a web site. The CustomizeGoogle.com Firefox plugin allows you to block the Google Analytics cookies [2] and much more.

It’s unfortunate that Firefox/Iceweasel seems to lack the cookie management functions of Konqueror. Konqueror (the KDE web browser) can be configured to prompt the user for the appropriate action when a cookie is offered, the options include once-only accept or reject and permanent accept or reject status for the site in question. Of course even this has some issues, when a web site is on the “permanently block cookies” list it is one that has obviously been viewed intensively on at least one occasion (IE many page views) or viewed on multiple occasions, in some situations this may be a fact that the user does not want revealed. An option to store a list of the hashes of the names of web sites which should be blocked would be useful. It’s also unfortunate that Konqueror (like most browsers) is unable to use Firefox plugins, so given a choice between Konqueror and Firefox I’m always going to lose some features.

Update: Andrew Pollock points out that Firefox does allow you to control when cookies are accepted [5]. It’s listed as “Keep Until” with the value of “ask me every time“.

The next issue relates to the storage of cookies. It is a good security feature to have certain types of cookie expire after some period of time. Unfortunately the expiry process requires that the user run the web browser in question. So if for example my browser preferences were to change then I would probably end up with the cookies from the old browser remaining in my home directory for years after their planned expiry date. My home directory has the untouched configuration and data files of many programs that I have not used for four years or more. I’m not sure whether any of them include cookies from web browsers (I have used many web browsers over the years).

I think that the best solution to this problem would be to have a common directory such as ~/.session-state which has files with an MTIME indicating when they should expire. A program that wants to store such session data could create a subdirectory such as ~/.session-state/Firefox and then use one file per cookie under that directory. Then the user could have a cron job which deletes all session state files that are older than the current date. Such a cron job would not need to know anything about the actual data in the files, it would just delete the files that are out of date. The exact format of the files would be determined by the application, so if there were thousands of cookies (which would lead to a performance problem on some systems if one file was used for each) then there could be one file for each week (if deleting the old cookies as much as 6 days too late is a serious problem then you are probably going to suffer anyway). Such a state directory could be used for any data which has a fixed expiry time, it would not need to be limited to cookies.

This would be a minor misuse of the mtime field, but it’s the most reliable way of implementing this and making it difficult to mess it up (in terms of exposing private data). Note that the MTIME would not have to be the sole source of such data, an application such as Firefox could reset the MTIMEs on the files to values it considers appropriate (based on file name, file contents, or some metadata stored elsewhere). It is expected that certain backup/restore operations among other things can result in the timestamp data on files being lost.

Now cookies are not the extent of the problem. It seems that Macromedia/Adobe have some similar functionality in the Flash player [3], but the insidious thing is that Flash cookies are used to respawn HTTP cookies if the user deletes them! After reading about that I discovered some Flash cookies that were stored on my laptop since 2005 (which was probably the last time I ran Flash). It seems that if you desire security you need to first avoid software from companies that are at best disinterested and sometimes seem overtly hostile towards the privacy needs of users – this is why I haven’t used Flash on machines that matter to me for many years. If I had a lot of spare time I would help out with the GNASH project.

One thing I have been considering is to change my browsing habits to use a different account for untrusted content. The switch user functionality that has been in most Linux distributions for a few years seems to have the potential to alleviate this. I am considering setting up a system to allow me to ssh to a guest account to open a web browser window. Then I can switch to the X desktop that has untrusted web sites open and read them. It would be nice if I could extend a web browser to add an extra entry to the menu that is displayed when the secondary mouse button is pressed on a link, then I could make that run a script to launch the URL in a new window. I could also use that when I’m at home to launch the URL on a different system.

One thing that I have to do is to get XGuest (the SE Linux Kiosk Mode) [4] running in Debian. It’s been in Fedora since version 8. With the XGuest used for untrusted browsing nothing gets stored.

This is not the extent of security issues related to web browsing. It’s just a small set of issues that need to be fixed, we have to start somewhere.

Why you should maintain old URLs

August 17, 2009 | etbe

Below is a message from the thanks file on my SE Linux play machine [1]:

Hello from San Juan, Puerto Rico!
I just found out about this server by reading the SELinux book from O’Reilly. The book is pretty old (2004) and I’m glad to know the URL provided on the book still works!
All the best,

I had forgotten that the URL was included in the book.

[1] http://www.coker.com.au/selinux/play.html

New Lamps for Old – Light Changing and Burglary

July 29, 2009 | etbe

A few weeks ago I had a guy from “enviro saver” visit my home to replace incandescent globes with CFLs. Â The original plan was to deliver a water-saver shower head as well but he told me that because I have solar hot water there are no renewable energy certificates in installing a shower head so I couldn’t get one.

The brochure they gave me at my local shopping center when I signed up for this claimed that they are “acting on a genuine concern about the
environmental impact of our Australian lifestyle”. Â But it seems that renewable energy certificates and up-selling of Photo-Voltaic systems is the real aim. I’ve been planning to get a PV system installed so I’ll be interested to see what they offer me.

The CFLs that they gave me appeared to be very cheap ones. Â They take about 500ms to start while the better ones appear to take less than 100ms. It seems to be a reasonable business model to give people a few dollar’s worth of cheap CFL lights in exchange for a good sales opportunity and some renewable energy certificates.

This sort of thing has become popular enough that some thieves are copying the plan. According to the reports people will knock on your door offering free CFL lights, case your home while installing them (got to check every room for old incandescent lights), and then rob it the next week if it looks like there are good things in there.

The companies that offer a legitimate service of replacing lights apparently don’t send people knocking on doors. They have the customers sign up for the service in advance.

But the solution to such problems seems obvious. Firstly get the name of everyone who wants to enter your home. If you signed up for a service then make sure you know who you called. If someone appears on your doorstep then demand photo ID. Camera phones are good things, if someone refuses to adequately identify themself then take their picture, note the number of their car, and give the details to the police.

Keep a log of everything that seems relevant to home security, thieves may attack your home weeks after casing it, so you can’t rely on your memory. Also keep a log in a place where it’s not likely to be stolen, storing it on a computer that is in your home would be a bad idea.

DRM and Rogue Employees

July 19, 2009 | etbe

ZDNet has an interesting article about Amazon unselling books to Kindle owners [1]. Apparently the books Animal Farm and 1984 were added to the Kindle list by unauthorised people (Engaget has the original story [2]). So Amazon decided to just remove the books from the Kindles and refund the purchase price.

Amazon has stated a plan to not unsell books in such situations in future – although they will apparently reserve the right to do so if they wish.

It seems to me that Amazon management are amazingly stupid. One thing we need to consider is that Amazon employs a large number of people, some of whom will be criminals and some will act in irrational ways for various reasons. Of the Amazon employees who won’t consistently act in an honest and reliable way on behalf of their employer some will have access to the database which controls the content that is permitted on Kindles. The Journalspace fiasco should be sufficient proof of this problem [3].

If a rogue employee wiped the database of sales in progress it would really hurt the Amazon business model, but if a rogue employee also unsold the existing works (stole property from customers) then it would be much worse.

The “features” of the Kindle would be useful to anyone who wants to make some money shorting Amazon stock. This should be of concern to the directors of Amazon.

Valgrind and OpenSSL

June 28, 2009 | etbe

I’ve just filed Debian bug report #534534 about Valgrind/Helgrind reporting “Possible data race during write” [1]. I included a patch that seems to fix that problem (by checking whether a variable is not zero before setting it to zero). But on further testing with Valgrind 3.4.1 (backported from Debian/Unstable) it seems that my patch is not worth using, I expect that Valgrind related patches won’t be accepted into the Lenny version of OpenSSL.

I would appreciate suggestions on how to fix this, the problem is basically having a single static variable that is initialised to the value 1 but set to 0 the first time one of the malloc functions is called. Using a lock for this is not desirable as it will add overhead to every malloc operation. However without the lock it does seem possible to have a race condition if one thread calls CRYPTO_set_mem_functions() and then before that operation is finished a time slice is given to a thread that is allocating memory. So in spite of the overhead I guess that using a lock is the right thing to do.

deb http://www.coker.com.au lenny gcc

For the convenience of anyone who is testing these things on Debian and wants to use the latest valgrind, the above Debian repository has Valgrind 3.4.1 and a build of GCC to fix the problem I mentioned in my previous blog post about Valgrind [2].

if (default_RSA_meth == NULL)
default_RSA_meth=RSA_PKCS1_SSLeay();

I have also filed bug #534656 about another reported race condition in the OpenSSL libraries [3]. Above is the code in question (with some C preprocessor stuff removed). This seems likely to be a problem on an architecture for which assignment of a pointer is not an atomic operation, I don’t know if we even have any architectures that work in such a way.

static void impl_check(void) Â {
Â Â Â Â CRYPTO_w_lock(CRYPTO_LOCK_EX_DATA);
Â Â Â Â if(!impl)
Â Â Â Â Â Â Â Â impl = &impl_default;
Â Â Â Â CRYPTO_w_unlock(CRYPTO_LOCK_EX_DATA);
}
#define IMPL_CHECK if(!impl) impl_check();

A similar issue is my bug report bug #534683 [4] which is due to a similar issue with the above code. If the macro is changed to just call impl_check() then the problem will go away, but at some performance cost.

I filed bug report #534685 about a similar issue with the EX_DATA_CHECK macro [5].

I filed bug report #534687 about some code that has CRYPTO_w_lock(CRYPTO_LOCK_EX_DATA); before it [6], so it seems that the code may be safe and it may be an issue with how Valgrind recognises problems (maybe a Valgrind bug or an issue with how Valgrind interprets what the OpenSSL code is doing). Valgrind 3.3.1 reported many more issues that were similar to this, so it appears that version 3.4.1 improved the analysis of this but didn’t do quite enough.

I filed bug report #534706 about the cleanse_ctr global variable that is used as a source of pseudo-randomness for the OPENSSL_cleanse() function without locking [7]. It seems that they have the idea that memset() is not adequate for clearing memory. Does anyone know of a good research paper about recovering the contents of memory after memset()? I doubt that we need such things.

I filed bug report #534699 about what appears to be a potential race condition in int_new_ex_data() [8]. The def_get_class() function obtains a lock before returning a pointer to a member of a hash table. It seems possible for an item to be deleted from the hash table (and it’s memory freed) after def_get_class() has returned the pointed but before int_new_ex_data() accesses the memory in question.

I filed bug report #534889 about int_free_ex_data() and int_new_ex_data() which call def_get_class() before obtaining a lock and then use the data returned from that function in a locked area[9] (it seems that obtaining the lock earlier would solve this).

I filed bug report #534892 about another piece of code which would have a race condition if pointer assignment isn’t atomic, this time in err_fns_check() [10]. In my first pass I didn’t bother filing bug reports about most of the issues helgrind raised with the error handling code (there were so many that I just hoped that there was some subtle locking involved that eluded helgrind and my brief scan of the source). But a new entry in my core file collection suggests that this may be a problem area for my code.

I think that it is fairly important to get security related libraries to be clean for use with valgrind and other debugging tools – if only to allow better debugging of the code that calls them. I would appreciate any assistance that people can offer in terms of fixing these problems. I know that there are security risks in terms of changing code in such important libraries, but there are also risks in leaving potential race conditions in such code.

As an aside, I’ve filed a wishlist bug report #534695 requesting that valgrind would have a feature to automatically add entries to the suppressions file [11]. As a function that is considered to be unsafe can be called from different contexts, and code that is considered unsafe can be in a macro that is called from multiple functions there can be many different suppressions needed. Pasting them all into the suppressions file is tedious.

The Millennium Seed Bank

June 11, 2009 | etbe

Jonathan Drori gave an interesting TED talk about the Millenium Seed Bank [1]. The potential for discovering new uses of plants for food, medicine, and construction is obvious, so it also seems obvious to me that we should preserve as many varieties of plant as possible to allow for future uses. As well as those obvious uses there are other potential uses of plants to cope with the changing climate and new diseases. Seeds from salt-tolerant plants have already been sent to Australia to help deal with the salinity problems related to the ongoing process of desertification and excessive use of bore water.

The seeds are stored in bunkers that are designed to withstand nuclear attack, I doubt that such protection will be necessary – or that it would be successful it it was needed.

Jonathan also gave a TED interview with more detail on this topic [2]. One particularly interesting issue is the work on testing seeds for viability and for developing germination protocols to specify the best combination of changes in temperature, moisture, etc to germinate seeds. This research seems to have a lot of potential to improve crop yields.

He mentioned in passing a project to collect folk-tales related to plants which apparently has led to some scientific discoveries.

A related project is the Norwegian Svalbard seed vault which seems mostly aimed at crop seeds [3]. The main difference is that Svalbard provides black-box storage (like a bank safe-deposit vault) while the Millennium Seed Bank owns the seeds. Incidentally the Bill and Melinda Gates Foundation provides significant support to Svalbard (so Bill does do some good things).

One thing that seems strange to me is the fact that governments are prepared to spend such large amounts of money on anti-terrorism but spend so little on seed banks and other projects that can help protect the food supply. If a country such as Australia (which exports a lot of food) was suddenly unable to produce enough food to even support the local population then the consequences would be much worse than anything Osama could dream up.

etbe – Russell Coker

Linux, politics, and other interesting things

Category Archives: Security