|
|
Last year at LCA Crispin Cowan suggested to me that I make a joint offer of a combined tutorial on SE Linux and AppArmor as a way of publicly comparing the two technologies. I ended up not accepting the challenge, among other things I had a long-term project going in production in early December that needed some ongoing support.
Crispin’s plan B was to just give a lecture about AppArmor. Recently Crispin joined Microsoft [1] and John Johansen of Suse was going to give the talk in his place. The LCA people made a minor mistake by having the conference web site give the description of the tutorial option [2] (which I don’t believe was ever going to happen as I had not accepted the offer), but it’s easy to understand the webmaster copying the wrong description when the one person makes two offers (which I believe is not a common practice).
So this morning I and about 150 other people were waiting in the main lecture theatre and no-one from Suse turned up.
Being fairly audacious when the announcement was made that the event was officially cancelled I stood up and asked if anyone would like an impromptu talk about SE Linux instead. The audience received that idea quite well.
My talk wasn’t as good as I had hoped, not having had a proper breakfast or any caffeinated drinks reduced my mental stack space. So I could talk well on one topic but when questions diverted me to a side topic I found it difficult to remember the previous point I was making. Fortunately when giving an impromptu talk with no notes or presentation materials the audience expectations for a consistent plan of the talk seem reasonably low. ;)
I started by talking about my SE Linux Play Machines. Some of that material had been covered in previous talks at other conferences (such as at a previous SE Linux Symposium), but some things (such as my use of Xen) I had not previously covered, but none of it had been mentioned in a talk in Australia for a while. Having given an hour-long talk about SE linux yesterday to an audience with many of the same members I wanted to start by talking about something that they hadn’t heard before, and I was also wearing a Play Machine T-shirt (with the root password printed on it) [4]. After I finished talking about my Play Machines and started covering some of the same material as yesterday about a quarter of the audience left (which was fair enough).
I then spoke about general SE Linux issues, largely in response to questions. I covered the differences between the policies (including the history of policy development), the JFFS2 XATTR development (and how SE Linux couldn’t be used on an iPaQ without it), issues of disk space usage for XATTRs for SE Linux labelling on various filesystems and how it drives the use of context mount options, poly-instantiated directories (including some discussion on how the actual storage location for such directories can be on a different filesystem and how this could be convenient when using encrypted filesystems), how Apache/PHP work in a SE Linux environment, and a lot more.
I couldn’t resist mentioning to the audience the irony that I had declined a challenge for a joint presentation and then got a sole presenter spot (and a large audience) due to the Suse guy not showing up.
For future situations I plan to load lecture notes from all my common talks on my iPaQ. Then next time I have such a speaking opportunity I can give a better prepared talk.
Update: It turned out that the LCA people had been informed that the Suse talk was cancelled and had made a mistake, see this link for details.
My previous post about my LCA mini-conf talk received an interesting comment from Christopher Neugebauer.
He said that he had some trouble understanding me because I speak quickly, he wasn’t the first person to make that complaint (it’s the most common complaint I receive). If a talk goes well then I have a lot to say and little time to say it and end up speaking quickly if I don’t concentrate enough on speaking slowly. If a talk doesn’t go well then I get nervous and speak quickly.
When a speaker talks too quickly it is appropriate to call out a request for them to speak more slowly. I know I’m not the only person who has difficulty in speaking slowly enough and I expect that others also wouldn’t mind such requests from the audience.
Chris suggested giving a talk with a small number of words used on the projector, it’s an interesting idea and may be worth a try. However I have recently watched Lawrence Lessig’s talk published on TED.com [1] which used that technique, I was disappointed in the result. His talk appeared to be very well received by the audience, I’m not sure whether that is because the audience was less familiar with his ideas than I am or whether it’s a technique that works better for an audience than for a video.
I would appreciate further suggestions in this regard.
Update: It’s interesting to note that Bruce Schneier’s keynote for LCA had no presentation material, he spoke from written notes.
Due to the lack of entries so far I am amending the rules. It is no longer required that an entry be on the blog of the person who submitted it. Being on any blog that is aggregated by the conference Planet will do.
This is known as a “guest post“. All it requires is that you email the post content to a blogger who you trust and they post it crediting you as the author. Guest posts are fairly common among serious bloggers, a google search will surely give more information.
Today I gave a talk about Debian security at the security mini-conf of LCA.
Before I started the talk I asked for suggestions as to how to get more entries in my security blogging contest [0]. During the talk I asked for suggestions as to how to get more people involved in security development. One suggestion was to offer incentives. I’m experimenting with that with my blogging contest and may do future variations of the same thing.
I started with describing some of the history of security in Debian (primarily things that involved me in some way):
In 2003 I suggested that exec-shield be a standard feature in Debian kernel images [1]. I created a kernel-patch-exec-shield package in 2003 and Marcus Better took it over in 2004. We are hoping to get it included in Lenny. AMD64 architecture doesn’t need exec-shield as the CPU has separate write and execute bits in the page table, but it would be nice to get exec-shield included before the last P4 machine gets decommissioned.
A presentation at the security miniconf at LCA 2005 on the topic of stack smashing is interesting [2]. At the time Adamantix was a distribution based on Debian which used PaX (similar to exec-shield). Adamantix has gone away. Hardened Gentoo has been available with Pax for all this time (but is not widely used). RHEL and Fedora have been available for all this time with exec-shield…
In mid 2002 I demonstrated the first SE Linux Play machine at a conference in Germany. It was fully operational with root as the guest user. At that time SE Linux support in Debian was essentially complete. Since that time the scope of the SE Linuc project has increased slightly (EG controlling DBUS access) so the amount of work required for full support is greater. Most of that support is in Debian and Etch is basically working with SE Linux (although not quite as well as it was in 2002 due to lack of support for the strict policy). The aim is to have Lenny SE Linux become as functional as SE Linux in Fedora Core 5. While FC5 has more SE Linux features than the SE Linux project supported in 2002 it’s still a great disappointment that it’s taken so long.
FC5 had pam_namespace to polyinstantiate directories such as /tmp. Lenny will hopefully have it.
I described the current status:
The hardening-wrapper package in unstable allows environment variables starting with DEB_BUILD_HARDENING_ to be used to control execution of GCC. Documented on the Debian Hardening Wikipedia page [3]. It’s still a little experimental and may change in the near future, but it works.
Lucas Nussbaum is working on automatically building Debian packages with warnings for security related issues. The aim is to build all packages and maintain a central location for the logs to allow DDs to find and fix the problems in their packages.
The Alioth Hardening project [4] will hopefully get some action soon (the people involved are busy doing work but not updating the project). The current plan is to base the Debian Hardening work around it.
SE Linux in Debian is something that I want to get working correctly. There are still some significant issues that make strict policy unusable (such as correct labelling of /etc/passwd) as of last time I tested it.
Finally I described the future plans. There were many questions about usability features for SE Linux, I mentioned in concept the features that Red Hat and Tresys people are developing (which I often don’t use as I prefer vi for policy editing).
There were some questions about how SE Linux works. More than half the audience indicated that they had used it so I assumed some basic knowledge of SE Linux when describing how SE Linux works in regard to minimum privilege and the benefits of MAC in terms of limiting the scope of attack. I noted that every program has bugs and every program which performs security related tasks (which includes serving data to the net without being owned) should be assumed to inevitably have security related bugs (see the The Inevitability of Failure paper []).
Based on the Twilight of the Books [5] article I decided to give this talk with no slides as an experiment. I talked from notes that I wrote and advised the delegates to read my blog for the details. Not presenting any slides meant that the room lights were all left on, which made things much easier when answering the many questions (I prefer an interactive format to my talks and have more questions than most speakers). It will be interesting to get some feedback from delegates about how they regarded this.
When previously writing about how I partition disks [1] I mentioned that I use smaller RAID partitions than the maximum size to reduce reconstruction time in the event of a crash.
Linux software RAID has a feature known as write intent bitmaps which means that every time some data is about to be written the region of the RAID array is marked as dirty. Then after a power failure all that needs to be done to ensure that all disks in the array have matching data is to check the regions that are listed as dirty not the entire disk. So instead of spending an hour or more checking the data there would only be a few seconds of work required.
To enable this feature you use the mdadm option -binternal (for an internal bitmap – most people would never want an external bitmap). This can be done at array creation time or at any other time via the grow option. For example if you want to enable this feature on /dev/md0 then you would use the command mdadm -G /dev/md0 -binternal.
Here is an example of /proc/mdstat when it’s not enabled:
Personalities : [raid1]
md1 : active raid1 hda2[0] hdb2[1]
38981632 blocks [2/2] [UU]
md0 : active raid1 hda1[0] hdb1[1]
96256 blocks [2/2] [UU]
Here is the same system with the feature enabled:
Personalities : [raid1]
md1 : active raid1 hda2[0] hdb2[1]
38981632 blocks [2/2] [UU]
bitmap: 3/149 pages [12KB], 128KB chunk
md0 : active raid1 hda1[0] hdb1[1]
96256 blocks [2/2] [UU]
bitmap: 0/12 pages [0KB], 4KB chunk
The down-side to this feature is that it will slightly reduce performance. But when comparing the possibility of a few percent performance loss all the time and the possibility of a massive performance loss for an hour or two after a crash it seems that losing a few percent all the time is almost always the desired option.
The Planet installation for the Linux.Conf.Au (the main Linux conference in Australia and one of the biggest and best Linux conferences in the world) is designed to only syndicate posts about the conference. I think that this is a bad idea, people who attend the conference actually see things and don’t have a great need to read blog posts about the conference. I believe that the benefit in having a Planet installation related to a conference is to allow delegates to easily read the blogs of other delegates. Then they can track down the bloggers if they want to discuss the blogs, or add them to their favourite feed reader so continue reading after the conference.
So I created my own Planet for it [1]. I started the installation with a feed from the official Planet LCA 2008 [2], then added the full feeds for people who appear to only have a partial feed aggregated on the official Planet. I also added Bruce Schneier’s Cryptogram [3] blog (Bruce is the opening keynote speaker for the conference).
If you have a partial feed of your blog syndicated on Planet LCA 2008 then please let me know so I can syndicate your blog’s full feed.
Update:
Atom feed of my Planet [4].
RSS 2.0 feed of my Planet [5].
Yesterday when walking down Flinders St I noticed that a new store has opened up selling organic food. It’s Flinders Organics and the address is 260 Flinders St Melbourne VIC 3000 (just across the road from Flinders St Station, not far from the Swanston St intersection). I bought some fruit, some Green and Black organic hot-chocolate powder (recently I’ve been making hot chocolate with Green and Black dark chocolate – the milk needs to be heated a lot and some stirring is needed – it is easier with powder) and some fruit juice. The fruit juice was good, one litre for about $4.50 which is significantly cheaper than any of the juice-bars which offer freshly squeezed juice (but not organic). Being sold in a bottle that can be re-sealed meant that I could carry it around the city and drink some whenever I was thirsty.
People who are attending LCA might want to keep this in mind, both for food that they want to prepare themselves (EG making sandwiches in their hotel room) and for take-away stuff such as bottled juice. The location is almost within walking distance of the conference.
I’ve just been working with a Flash device used as /dev/hda (the root filesystem) for a router. The aim is much the same as with my idea of using an EeePC as a router [1]. The client in question may consider the EeePC for future deployments but are concerned about the ability of ASUS to supply when needed. The typical customer at a computer store probably isn’t going to be too upset if they have to order their computer and wait a week, but when rolling out a network for a few corporate offices it’s a major problem if there aren’t enough devices to get them all online at once!
One problem I encountered is that the default configuration had IDE DMA enabled and this didn’t work well on the system, rumour suggests that it might be related to the bridge chipset used to run the PATA controller which talks to the device that houses a SD Flash module and connects to the IDE cable. But I discovered that it would work reasonably well (at about 75% full speed) if I turned off DMA. When I run the Flash device with no DMA and 16bit IO I get about 2.4MB/s, I get 4.4MB/s if I use 32bit IO, and 6MB/s with occasional system hangs if I use DMA.
The machine in question is running CentOS so the trick was to run hdparm early enough in the boot process to turn off DMA before the machine could hang. In most cases the machine would lock up solidly when udev was started.
So I added the following lines to the start of /etc/rc.d/rc.sysinit (just after #!/bin/bash):
/sbin/restorecon -R /dev 2> /dev/null
/sbin/hdparm -d0 -c1 /dev/hda
Now the system seems to be working correctly.
The flash device seems like a good concept, no moving parts means less heat and more reliability, and the seek time is really good. But slow performance for bulk IO and problems with DMA increase the difficulty of implementing it. The advantage of having it in a pre-packaged device such as an EeePC is that it’s all one unit that is known to work together and has the drivers configured.
Having had a number of hard drives fail over the years I use RAID whenever possible to reduce the probability of data loss caused by hardware failure. It’s unfortunate that some machines make it impractically difficult to install a second hard drive (my laptop and some small form factor desktop machines I have given to relatives). But when it’s possible I have two hard drives running software RAID-1.
I use two partitions, one for /boot and one as a LVM physical volume (PV). When using RAID I make both /boot and the PV be software RAID-1 devices (the vast majority of machines that I install don’t have budget available for hardware RAID). /boot is a small partition, approximately 100M. For a machine with only one disk I make the second partition take all the available space as there is no benefit in doing otherwise (LVM allows me to create an arbitrary number of block devices out of the space at run-time).
When using software RAID I often make the PV take less than the full capacity of the disks. When the disks are 40G or more I usually use less than half the capacity. For most machines that I install or run the full capacity of the disks is not required. One deficiency of Linux software RAID is that on a power failure the entire RAID device must be checked to ensure that all disks have matching data. Reading the entire contents of a pair of disks can take a large amount of time if the disks are large, and as the size of disks is increasing at a greater rate than the speed of disks this problem is getting worse. See my ZCAV benchmark results page for graphs of the contiguous IO performance of some disks I’ve owned [1]. It seems that with a 10G disk you may expect to be able to read it all in less than 1000 seconds, for a 46G disk it’ll be something greater than 1,500 seconds, and for 300G disks you are looking at something well in excess of 5,000 seconds.
Almost all disks (and all IDE and SATA disks for which I’ve seen benchmark results) have the lower block numbers mapped to the outside tracks which are longer and give a higher contiguous IO speed. So by using the first half of the disk the RAID synchronisation time is reduced to less than half what it might be (in the absence of motherboard bottlenecks).
When there is no need for more than about 10G of storage space there’s no benefit in making a large RAID device and wasting recovery time. While the system can operate while the RAID is synchronising the performance will be significantly lower than normal.
If the usage pattern of the machine changes such that it needs more space it’s easy to create new partitions, make a new software RAID, and then add it to the LVM volume group (I’ve done this before). So the down-side to this is minimal.
When creating LVM logical volumes (LVs) I create volumes for the root filesystem and swap space when doing the install. This should result in the swap space being near enough to the start of the disk to get reasonable performance (but I haven’t verified this). I make the root filesystem reasonably large (EG 6G) as disk space is plentiful and cheap nowadays and the root filesystem is the only one that you can’t easily extend when running LVM (trying to do so deadlocks the disk IO). After the base install is done I create other LVs.
Chris Lamb has suggested storing a GPG key on a RAID-5 device [1]. The idea is that it can be stored on several physical block devices such that losing just one will not give the key to an attacker.
A default GPG secret key will be about 1.2K in size (3 sectors of a hard drive). A minimal key (with 1024 bit DSA keypair) will be 0.9K (2 sectors). I expect that very few people have secret keys greater than 4K in size.
To create a software RAID-5 device under Linux the tool mdadm is used. The default chunk size is 64K, so a 1.2K file will probably be on a single device. If you use the -c option of mdadm to specify a smaller chunk size then the smallest that is accepted is 4K which still permits a default GPG secret key to be on a single device. The Ext2 and Ext3 filesystems will always align such file data to a 4K boundary unless the device is smaller than a certain size (or a special mkfs option is used) to give a 1K block size for the filesystem. If an Ext2 or Ext3 filesystem is used with 1K blocks then you might get a 1.2K file split across multiple 4K RAID chunks.
So storing a GPG key on RAID-5 won’t prevent an attacker who steals one part from getting all the most valuable data. It will make it more inconvenient for them (if you are lucky it will prevent them getting all the data) and it will also make it difficult for the owner of the GPG key to determine which of the devices actually contains the secret data (probably all of them will end up having copies if you edit the secret key).
Now if RAID-5 did allow chunk sizes that were smaller than the secret key or if you have Ext2/3 with 1K blocks and get lucky with file fragmentation then the problem still isn’t solved. The reason is that you don’t require N-1 of the N disks to get some useful data out of a RAID-5 array (run strings on one element of a RAID-5 array to verify this). A single disk on it’s own will have some data that can be used, as file(1) can recognise GPG secret keys so you could just copy 1K chunks of data into separate files and use file to determine which (if any) has the data in question.
The really exciting question is, what do you get if you have the first 1K of a 1.2K GPG secret key? If it could be proved that the first 1K does not give an attacker any advantage then this might provide some benefit. But I think that this is a very dubious assumption, when dealing with such things it’s best to assume the worst. Assume that an attacker who has 1K out of 1.2K of secret data has the ability to reconstruct the rest. In that case the Linux kernel RAID-5 provides no benefit for storing a GPG secret key.
Just try not to get your devices that contain secret data stolen. Maybe a watch with a built-in USB device is a good idea. Thieves seem to be targetting mobile phones instead of watches nowadays and something that’s strapped to your wrist is difficult to lose.
|
|
