Release Dates for Debian

May 13, 2008 | etbe

Mark Shuttleworth has written an interesting post about Ubuntu release dates [1]. He claims that free software distributions are better able to meet release dates than proprietary OSs because they are not doing upstream development. The evidence that free software distributions generally do a reasonable job of meeting release dates (and Ubuntu does an excellent job) is clear.

But the really interesting part of his post is where he offers to have Ubuntu collaborate with other distributions on release dates. He states that if two out of Red Hat (presumably Enterprise Linux), Novell (presumably SLES), and Debian will commit to the same release date (within one month) and (possibly more importantly) to having the same versions of major components then he will make Ubuntu do the same.

This is a very significant statement. From my experience working in the Debian project and when employed by Red Hat I know that decisions about which versions of major components to include are not taken lightly, and therefore if the plan is to include a new release of a major software project and that project misses a release date then it forces a difficult decision about whether to use an older version or delay the release. For Ubuntu to not merely collaborate with other distributions but to instead follow the consensus of two different distributions would be a massive compromise. But I agree with Mark that the benefits to the users are clear.

I believe that the Debian project should align it’s release cycles with Red Hat Enterprise Linux. I believe that RHEL is being released in a very sensible manner and that the differences of opinion between Debian and Red Hat people about how to manage such things are small. Note that it would not be impossible to have some variations of version numbers of components but still stick mostly to the same versions.

If Debian, Ubuntu, and RHEL released at about the same time with the same versions of the kernel, GCC, and major applications and libraries then it would make it much easier for users who want to port software between distributions and run multiple distributions on the same network or the same hardware.

The Debian Social Contract [2] states that “Our priorities are our users and free software“. I believe that by using common versions across distributions we would help end-users in configuring software and maintaining networks of Linux systems running different distributions, and also help free software developers by reducing the difficulty in debugging problems.

It seems to me that the best way of achieving the goal that Mark advocates (in the short term at least) is for Debian to follow Red Hat’s release cycle. I think that after getting one release with common versions out there we could then discuss how to organise cooperation between distributions.

I also believe that a longer support cycle would be a good thing for Debian. I’m prepared to do the necessary work for the packages that I maintain and would also be prepared to do some of the work in other areas that is needed (EG back-porting security fixes).

The Purpose of Planet Debian

April 21, 2008 | etbe

An issue that causes ongoing discussion is what is the purpose of a Planet installation such as Planet Debian [1]. The discussion usually seems to take the less effective form of what is “appropriate” content for the Planet or what is considered to be “abuse” of the Planet. Of course it’s impossible to get anything other than a rough idea of what is appropriate is the purpose is not defined, and abuse can only be measured on the most basic technical criteria.

My personal use of Planet Debian and Planet Linux Australia [2] is to learn technical things related to Linux (how to use new programs, tricks and techniques, etc), to learn news related to Linux, and to read personal news about friends and colleagues. I think that most people have some desire to read posts of a similar nature (I have received a complaint that my blog has too many technical posts and not enough personal posts), but some people want to have a Planet with only technical articles.

In a quick search of some planets the nearest I found to a stated purpose of a Planet installation was from the Wiki to document Planet Ubuntu [3] which says ‘Subscribed feeds ought to be at least occasionally relevant to Ubuntu, although the only hard and fast rule is “don’t annoy people”‘. Planet Perl [4] has an interesting approach, they claim to filter on Perl related keywords, I initially interpreted this to mean that if you are on their list of blogs and you write a post which seems to refer to Perl then it will appear – but a quick browse of the Planet shows some posts which don’t appear to match any Perl keywords. Gentoo has implemented a reasonable system, they have a Universe [5] configuration which has all blog posts by all Gentoo bloggers as well as a Planet installation which only has Gentoo related posts.

It seems to me that the a reasonable purpose for Planet Debian would be to have blog feeds which are occasionally specific to Debian and often relevant to Debian. Personal blog posts would be encouraged (but not required). Posts which are incomprehensible or have nothing to say (EG posts which link to another post for the sole purpose of agreeing or disagreeing) would be strongly discouraged and it would be encouraged to make links-posts rare.

Having two installations of the Planet software, one for posts which are specific to Debian (or maybe to Debian or Linux) and one for all posts by people who are involved with Debian would be the best option. Then people who only want to read the technical posts can do so, but other people can read the full list. Most blog servers support feeds based on tag or category (my blog already provides a feed of Debian-specific posts). If we were going to have a separate Planet installation for only technical posts then I expect that many bloggers would have to create a new tag for such posts (for example my posts related to Debian are in the categories Benchmark, Linux, MTA, Security, Unix-tips, and Xen) and the tag Debian is applied to only a small portion of such posts. But it would be easy to create a new tag for technical posts.

Ubuntu is also the only organisation I’ve found to specify conditions upon which blogs might be removed from the feed, they say: We reserve the right to remove any feed that is inaccessible, flooding the page, or otherwise interfering with the operation of the Planet. We also have the right to move clearly offensive content or content that could trigger legal action.

That is reasonable, although it would be good to have a definition for “flooding the page” (I suggest “having an average of more than two posts per day appear over the period of a week or having posts reappear due to changing timestamps”). Also the “could trigger legal action” part is a minor concern – product reviews are often really useful content on a Planet…

Some time ago my blog was removed from Planet Fedora for some reason. I was disappointed that the person who made that change didn’t have the courtesy to inform me of the reason for their action and by the fact that there is no apparent way of contacting the person who runs the Planet to ask them about it. Needless to say this did not encourage me to write further posts about Fedora.

If a blog has to be removed from a feed due to technical reasons then the correct thing to do is to inform the blogger of why it’s removed and what needs to be fixed before it can be added again.

If a blog is not meeting the content criteria then I expect that in most cases the blogger could be convinced to write more content that matches the criteria and tag it appropriately. Having criteria for some aspects of blog quality and encouraging the bloggers to meet the criteria can only improve the overall quality.

Currently there is a Planet installation on debian.net being recommended which is based on Planet Debian, but with some blogs removed (with no information available publicly or on debian-private as to what the criteria are for removing the blogs in question). It seems to me that if it’s worth using Debian resources to duplicate the Planet Debian then it should be done in a way that benefits readers (EG by going to the Planet vs Universe model that Ubuntu follows), and that if blogs are going to be removed from the feed then there should be criteria for the removal so that anyone who wants their blog to be syndicated can make whatever changes might be necessary.

[1] http://planet.debian.org/
[2] http://planet.linux.org.au/
[3] https://wiki.ubuntu.com/PlanetUbuntu
[4] http://planet.perl.org/
[5] http://planet.gentoo.org/universe/

Preparing for a Collapse

April 19, 2008 | etbe

Rick Falkvinge (leader of the Swedish Pirate Party) has written his predictions about an economic crash in the US [1]. Predicting that the US economy will crash is no great stretch, it’s gross failures seem obvious. The Pirate Party [2] is a one-issue political party that is based on reform of intellectual property laws. It derived it’s name from the term Software Piracy [3] which originally referred to using software without paying for it, but in recent times has been broadened in scope to cover doing anything that copyright holders don’t like. The term “Piracy” is deprecated in the free software community based on the fact that it’s unreasonable to compare armed robbery and murder on the high seas (which still happens today and costs between $US13,000,000,000 and $US16,000,000,000 per year [4]) with copying some files without permission. But that battle has been conclusively lost, so it seems that the mis-use of the term “Piracy” will continue.

The majority of the acts which are considered to be “Piracy” are well accepted by the community, the acts of the music industry in taking legal action against young children have only drawn more public support for the “Pirate” cause. Such support is increasing the changes of the Swedish Pirate Party getting a seat in parliament at the next election, and has caused the major Swedish parties to change their positions on IP legislation.

Now Rick’s background related to Intellectual Property issues causes him to analyse the IP aspects of the current US problems. His claim is that the US economy was trashed during the Vietnam war, has been getting worse ever since, and that the US position on IP legislation is either intentionally or accidentally helping to finance the US while it’s production of useful things is steadily decreasing. He also claims that some multi-national financial customs (such as using the US dollar for the international oil trade) is propping up the US currency and effectively allowing the US government (and the US residents) to borrow money from the rest of the world.

Dmitry Orlov’s presentation titled “Closing the ‘Collapse Gap’: the USSR was better prepared for collapse than the US” [5] provides some interesting information on what happens during an economic collapse. He also has some specific advice on what can be done (by both governments and individuals) to prepare for an impending collapse. However he doesn’t mention some issues which are important to people like us (although not as important as food, water, and shelter).

On my document blog I’ve got a post with some ideas of how to run an Internet Infrastructure after a medium-scale collapse of the economy as we know it [6].

Software Development is a Team Sport

April 16, 2008 | etbe

Albert writes about software development and how much teamwork is used [1]. He makes an interesting clash of analogies by suggesting that it’s not a “team sport” because “its not like commercial fishing where many hands are used to pull in the net at the same time“.

I think that software development for any non-trivial project is a team sport. You don’t have the same level of direct coordination as required for pulling in a net (or the rugby scrum [2] to use a sporting analogy), but that doesn’t stop it being a team effort.

Some parts of team development projects are like a relay event, in corporate environments the work is done in parallel simply because everyone is working the same hours but in free software development projects the work is often serialised. I think that it’s often more effective to serialise some work, if someone is actively working on one sections of code it may save time to avoid working in that area until they are finished. There is little benefit in writing code to old interfaces.

Some parts of team projects have specialised skill areas (EG debugging, skills in particular programming languages, and graphical design). Soccer is one sport where different rules apply to different players (the goal keeper can use their hands). In ice-hockey the protective clothing used by the goal keeper is considerably different from that used by other players. In most team sports where the aim is to put a ball through a goal at one end (EG basketball and all versions of football) there seems to be some degree of specialisation, some players are dedicated to scoring goals while others are dedicated to defense. The fielding team in cricket has every player assigned to a different part of the field – with slight differences in the skills required.

Then there is the issue of large projects such as Linux distributions. It seems to me that a Linux distribution will always comprise multiple teams as well as some individual projects. Maybe we could consider Linux distributions (and distributions of the other free OSs) to be similar to countries that compete in the Olympics. The culture of the Free Software (or Open Source if that’s your idea) community can be compared to the Olympic Spirit. Of course the Olympic idea that people should come together in peace for the Olympic Games and that it’s about honor not money is pretty much dead.

Maybe the Free Software development processes should be compared to an ideal of what sporting contests would be if there weren’t unreasonable amounts of money (and therefore corruption) involved.

Of course no analogy is perfect and there are many ways in which this one breaks down. One of which is the cooperation between distributions. There is a lot of private discussion between developers of various distributions and upstream developers about how to plan new features. It’s not uncommon for developers to announce certain development decisions as soon as they are made to help other distributions make decisions – for a developer in a distribution project if there is an issue which doesn’t matter much to you or your users then it’s often good to strive for compatibility with other distributions.

When users advocate new features or changes they sometimes try multiple distributions. It’s not uncommon for a feature request to be rejected by one distribution and then accepted by another. Once a feature is included in a major distribution the upstream developer is more likely to accept it due to it’s wide testing. Then when the feature is in upstream it’s almost certain to be included in all other distributions. I often recommend that when someone disagrees with one of their bugs being closed as “not a bug” that they try reproducing it in another distribution and reporting it there. As a side note, the criteria for reporting a bug in any free software distribution is that you can describe it in a way that allows other people to reproduce it – whether it’s a bug that afflicts you every day or whether you installed the distribution for the sole purpose of reporting the bug in a new forum is not relevant. As a general rule I recommend that you not have the same bug report open in more than one distribution at any time (if you notice a bug reported in multiple distributions then please add a note to each bug report so that work can be coordinated). As a general rule the only situation where I will open the same bug in multiple forums is if I have been told that the responsible person or people in one forum are unwilling or unable to fix it.

Finally, the people who consider that they don’t need to be a team player because they do their coding alone might want to consider Qmail. Dan Bernstein is a great coder and Qmail is by most metrics a fine piece of software, in terms of security Qmail is as good as it gets! If Dan was more of a team player then I believe that his mail server would have been much more successful (in terms of the number of sites using it). However I do understand his desire to have a great deal of control over his software.

Trust and My SE Linux Play Machine

April 3, 2008 | etbe

Currently my SE Linux Play Machine [1] is running as a Xen DomU. So if someone cracks it they would also have to crack Xen to get access to directly change things on the hardware (EG modifying the boot process). As documented in my last post [2] a user of my Play Machine recently managed to change my password. Of course this was just two days after the vmsplice() kernel security flaw had been discovered [3]. Of course any machine that offers shell access to remote users (or the ability to run CGI-BIN scripts or other programs that users can upload) is immediately vulnerable to such exploits and while SE Linux has blocked local kernel exploits in the past [4] there will always be the possibility of kernel exploits that SE Linux can’t block or which can be re-written to work in a way that is not stopped by the SE Linux policy. So it’s best to assume that SE Linux systems are vulnerable to kernel exploits.

At the time that the vmsplice() exploit was announced there was a claim that it could be used to de-stabilise a Xen Dom0 when run within a DomU. It’s best to assume that any attack which can make some software perform in an unexpected manner can also be used to successfully attack it. So at the time I was working on the assumption that the Dom0 could have been exploited.

Therefore I reinstalled the entire machine, I firstly installed a new Dom0 (on which I decided to run Debian/Unstable) and then I made a fresh install of Etch for the Play Machine. There is a possibility that an attacker could compromise the hardware (changing the BIOS or other similar attacks), but this seems unlikely – I doubt that someone would go to such effort to attach hardware that I use for demonstrating SE Linux and for SE Linux development (it has no data which is secret).

If someone attacks my Play Machine they would have to first get root on the DomU in question and then crack Xen to get access to the hardware. Then the machine is on a separate Ethernet segment which has less access to my internal network than the general Internet does (so they would not gain any real benefit).

One thing an attacker can do is launch a DOS attack on my machine. One summer a Play Machine overheated and died, I suspect that the extra heat produced by a DOS attack contributed to that problem. But losing a low-end machine I bought second-hand is not a big deal.

When discussing the machine there are two common comments I get. One is a suggestion that I am putting myself at risk, I think that the risk of visiting random web sites is significantly greater. Another is a challenge to put the machine on my internal network if I really trust SE Linux, as noted I have made mistakes in the past and there have been Linux kernel bugs – but apart from that it’s always best to have multiple layers of protection.

SE Linux Play Machine and Passwords

April 2, 2008 | etbe

My SE Linux Play Machine [1] has been online again since the 18th of March.

On Monday the 11th of Feb I took it offline after a user managed to change the password for my own account (their comment was “ohls -lsa! i can change passwordls -lsals -lsa HACKED!“). Part of the problem was the way /bin/passwd determines whether it should change a password.

The previous algorithm (and the one that is currently used in Debian/Etch) is that if the UID of the account that is having it’s password changed doesn’t match the UID of the process that ran /bin/passwd then an additional SE Linux check is performed (to see if it has permission to change other user’s passwords). The problem here is that my Play machine has root (UID==0) as the guest account, and that according to the /bin/passwd program there is no difference between the root account (for unprivileged users) and the bofh account (which I use and which also has UID==0). This means of course that users of the root account could change the password of my account. My solution to this was to run chcon on the /bin/passwd program to give it a context that denied it the ability to change a password. The problem was that I accidentally ran the SE Linux program restorecon (which restores file contexts to their default values) which allowed /bin/passwd to change passwords, and therefore allowed a user to change the password of my account.

The semanage tool that allows changing the default value of a file context does not permit changing the default for a file specification that matches one from the system policy (so the sys-admin can’t override compiled in values).

I have now fixed the problem (the fix is in my Etch SE Linux repository [2] and has been accepted for Debian/Unstable and something based on it will go into the upstream branch of Shadow. See the Debian bug report #472575 [3] for more information.

The summary of the new code is that in any case where a password is not required to change the user’s password then SE Linux access checks will be performed. The long version is below:

The new algorithm (mostly taken from the Red Hat code base which was written by Dan Walsh) is that you can only change a password if you are running as non-root (which means that the pam_unix.so code will have verified the current password) or if you are running as root and the previous SE Linux security context of the process is permitted access to perform the passwd operation in the passwd class (which means it is permitted to change other user’s passwords).

The previous context (the context before one of the exec family of system calls was called) is used for such access checks because we want to determine if the user’s shell (or other program used to launch /bin/passwd) was permitted to change other user’s passwords – executing a privileged program such as /bin/passwd causes a domain transition and the context is different) than the program that was used to execute it. It’s much like a SETUID program calling getuid(2) to get the UID of the process which launched it.

To get the desired functionality for my Play Machine I don’t want a user to change their own password as the account is shared. So I appended password requisite pam_deny.so to the file /etc/pam.d/passwd (as well as the chfn and chsh commands) so that hostile users can’t break things. The new code in /bin/passwd will prevent users from taking over the machine if my PAM configuration ever gets broken, having multiple layers of protection is always a good thing.

The end result is that the Debian package and the upstream code base are improved, and my Debian Etch repository has the code in question.

Redirecting Output from a Running Process

February 27, 2008 | etbe

Someone asked on a mailing list how to redirect output from a running process. They had a program which had been running for a long period of time without having stdout redirected to a file. They wanted to logout (to move the laptop that was used for the ssh session) but not kill the process (or lose output).

Most responses were of the form “you should have used screen or nohup” which is all very well if you had planned to logout and leave it running (or even planned to have it run for a long time).

Fortunately it is quite possible to redirect output of a running process. I will use cat as a trivial example but the same technique will work for most programs that do simple IO (of course programs that do terminal IO may be more tricky – but you could always redirect from the tty device of a ssh session to the tty device of a screen session).

Firstly I run the command “cat > foo1” in one session and test that data from stdin is copied to the file. Then in another session I redirect the output:

Firstly find the PID of the process:
$ ps aux|grep cat
rjc 6760 0.0 0.0 1580 376 pts/5 S+ 15:31 0:00 cat

Now check the file handles it has open:
$ ls -l /proc/6760/fd
total 3
lrwx—— 1 rjc rjc 64 Feb 27 15:32 0 -> /dev/pts/5
l-wx—— 1 rjc rjc 64 Feb 27 15:32 1 -> /tmp/foo1
lrwx—— 1 rjc rjc 64 Feb 27 15:32 2 -> /dev/pts/5

Now run GDB:
$ gdb -p 6760 /bin/cat
GNU gdb 6.4.90-debian
Copyright (C) 2006 Free Software Foundation, Inc
[lots more license stuff snipped]
Attaching to program: /bin/cat, process 6760
[snip other stuff that’s not interesting now]
(gdb) p close(1)
$1 = 0
(gdb) p creat(“/tmp/foo3”, 0600)
$2 = 1
(gdb) q
The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: /bin/cat, process 6760

The “p” command in GDB will print the value of an expression, an expression can be a function to call, it can be a system call… So I execute a close() system call and pass file handle 1, then I execute a creat() system call to open a new file. The result of the creat() was 1 which means that it replaced the previous file handle. If I wanted to use the same file for stdout and stderr or if I wanted to replace a file handle with some other number then I would need to call the dup2() system call to achieve that result.

For this example I chose to use creat() instead of open() because there are fewer parameter. The C macros for the flags are not usable from GDB (it doesn’t use C headers) so I would have to read header files to discover this – it’s not that hard to do so but would take more time. Note that 0600 is the octal permission for the owner having read/write access and the group and others having no access. It would also work to use 0 for that parameter and run chmod on the file later on.

After that I verify the result:
ls -l /proc/6760/fd/
total 3
lrwx—— 1 rjc rjc 64 2008-02-27 15:32 0 -> /dev/pts/5
l-wx—— 1 rjc rjc 64 2008-02-27 15:32 1 -> /tmp/foo3 <====
lrwx—— 1 rjc rjc 64 2008-02-27 15:32 2 -> /dev/pts/5

Typing more data in to cat results in the file /tmp/foo3 being appended to.

Update: If you want to close the original session you need to close all file handles for it, open a new device that can be the controlling tty, and then call setsid().

Laptop vs Book Weight

February 18, 2008 | etbe

Matt Bottrell wrote an interesting and informative post about laptops for school kids [1]. His conclusion is that based on technical features the OLPC machine is best suited for primary school children and one of the ASUS EeePC, the Intel Classmate, and the Everex Cloudbook would be best suited for high-school students.

The Asus EeePC [2] is a good option, it runs a variant of Debian and the Debian Eeepc Team are active in getting full Debian support for it [3].

The Intel Classmate [4] has a choice of Windows XP, Mandriva, and Metasys Classmate. The web page says that it’s designed “for primary students (ages 5-14)“, so I think that Matt made a mistake in listing this as a possibility for high-schools, of course when running Mandriva it could have software installed for any age group but the hardware design may be better suited to younger children.

The Everex Cloudbook [5] runs the GOS Rocket [6] OS which seems to be an Ubuntu variant with an Enlightenment based GUI and a configuration aimed at using Google services (blogger, gmail, etc). Configuring Ubuntu to suit your needs is easy enough (it’s based on Debian). Note that Matt did not mention where one might purchase a Cloudbook in Australia and I don’t recall seeing one on any of my many window-shopping expeditions to Australian consumer electronics stores, while the EeePC is widely available (except when sold out). But I’m sure that if the government wanted to place an order for a couple of million units then Everex would ramp up production quickly enough.

Matt made one statement that I strongly disagree with, he wrote “A traditional notebook is far too heavy for high-school kids to lug around“.

To test this theory I searched for some high-school text books and a set of scales. A year 11 Maths A text book from ~1988 weighed 600g and the pair of year 12 Maths A and Maths B texts weighed 1.6Kg. When I was at high-school the day was divided into seven “periods”, some classes took two periods so four different classes which required text books (or other books) was typical. Carrying 3Kg of books to school would not be uncommon for year 12 students. The Lenovo T series (advertised as “premier performance” and the model I personally prefer) is listed as having a starting weight of 2.1Kg (which presumably doesn’t include the power pack). My Thinkpad T series (from about 2004) weighs about 2.4Kg according to my kitchen scales and has a battery weighing just over 400g.

My practice for a long time was to own a spare power pack for my Thinkpad so that I could leave it at work (saving 400g when travelling to and from work). I have also had the practice of buying a spare battery when I buy a Thinkpad (you need a spare battery for a long trip). So if I had really wanted to save weight I could have left a battery at work and reduced by travel weight by another 400g (with the cost being that I couldn’t use it when on a train or bus).

A spare power pack is not overly expensive. In the usual case students would only need a battery when at school (it’s a little known fact that Thinkpads work perfectly without a battery plugged in). So if a student had a power pack at home as well as one at school and if they left their battery at school and they owned one of the latest Thinkpad T series (listed with a starting weight of 2.1Kg) then their travel weight might be about 1.7Kg. If the majority of school texts could be stored on their laptop then the result of using a Thinkpad T series would be a significant weight reduction! If the students were using a Thinkpad X series (more expensive so maybe not a good option) then the list weight is 1.57Kg and the travel weight might be as low as 1.3Kg (at a rough estimate).

The EeePC offers significant benefits for school use, it is light, cheap (children tend to break or lose things more frequently than adults so you should budget for buying two of anything that they use), and having no hard drive (flash storage) it should cope well with being dropped. The screen on the EeePC is unreasonably small buy Asus could release a new model with a bigger screen (they may do this in the future anyway or a government contract could encourage them to do it sooner).

I agree that the EeePC or the Everex Cloudbook is probably the best option for high-school students, but I can’t agree with any claim about a traditional laptop being too heavy, the only reason for excluding a traditional laptop is that those new ultra-lights are better.

Another reason that might be cited for not using laptops is the cost. While prices of $1000 or more for a traditional laptop are rather expensive, the $500 for an EeePC is not that expensive – and the government could surely negotiate a better deal, I would be surprised if they couldn’t get the price down to $350 by some bargaining and by removing the middle-man. A careful child could use the same laptop for the entire duration of high-school and their parents would incur less expense than they currently would spend on text books.

As for the current lack of electronic text books. Currently when the education department selects a book it’s a license to print money for the author and publisher. All that the education department has to do is to declare that they will do a deal with the first company to release their books under a creative commons license. The idea would be that an author (or publishing company) would get paid a fixed sum of money for a CC release of a text book which would then be available for use by anyone anywhere in the world. World-wide free distribution would be no loss to the author (each country tends to have unique books anyway) but would be a good act of charity from our government to developing countries.

Once books were available under a creative commons license (without the “no modifications” clause) they could be freely improved by anyone. Improving text books for younger students could be a good school project.

Update:

Thanks to Steve Walsh for pointing out that the Classmate can run Linux. It’s a pity that he didn’t link to my post so that his readers could see what he was referring to. I take it as a good sign of the quality of my posts that such small errors get pointed out.

Linux Resource Controls

February 7, 2008 | etbe

Using the “ulimit” controls over process resource use it is possible to limit RAM for processes and to limit the number of processes per UID. The problem is that this often is only good for accidental problems not dealing with malicious acts.

For a multi-user machine each user needs to be allowed to have two processes to be able to do anything (IE the shell and a command that they execute). A more practical limit is five processes for a single shell session (one or two background jobs, a foreground job where one process pipes data to another, and the shell). But even five processes is rather small (a single Unix pipe can have more than that). A shell server probably needs a limit of 20 processes per user if each user will have the possibility of running multiple logins. For running the occasional memory intensive process such as GCC the per-process memory limit needs to be at least 20M, if the user was to compile big C++ programs then 100M may be needed (I’ve seen a G++ process use more than 90M of memory when compiling a KDE source file). This means that a single user who can launch 20 processes which can each use 20M of memory could use 400M of memory, if they have each process write to a pages in a random order then 400M of RAM would be essentially occupied by that user.

If a shell server had 512M of RAM (which until quite recently was considered a lot of memory – the first multi-user Linux machine I ran on the net had 4M of RAM) then 400M of that could be consumed by a single hostile user. Leaving 100M for the real users might make the machine unusable. Note that the “hostile user” category also encompasses someone who gets fooled by the “here’s a cool program you should run” trick (which is common in universities).

I put my first SE Linux Play Machine [1] on the net in the middle of 2002 and immediately faced problems with DOS attacks. I think that the machine had 128M of RAM and because the concept was new (and SE Linux itself was new and mysterious) many people wanted to login. Having 20 shell users logged in at one time was not uncommon, so a limit of 50 processes for users was minimal. Given that GCC was a necessary part of the service (users wanted to compile their own programs to test various aspects of SE Linux) the memory limit per process had to be high. The point of the Play Machine was to demonstrate that “root” was restricted by SE Linux such that even if all Unix access control methods failed then SE Linux would still control access (with the caveat that a kernel bug still makes you lose). So as all users logged into the same account (root) the process limit had to be adequate to handle all their needs, 50 processes was effectively the bare minimum. 50 processes with 5M of memory each is more than enough to cause a machine with 128M of RAM to swap to death.

One thing to note is that root owned system processes count towards the ulimit for user processes as SE Linux does not have any resource usage controls. The aim of the SE Linux project is access control not protection against covert channels [2]. This makes it a little harder to restrict things as the number of processes run by daemons such as Postfix varies a little over time so the limits have to be a little higher to compensate, while Postfix is run with no limits the processes that it creates apply to the global limit when determining whether user processes can call fork().

So it was essentially impossible to implement any resource limits on my Play Machine that would prevent a DOS. I changed the MOTD (message of the day – displayed at login time) to inform people that a DOS attack is the wrong thing to do. I implemented some resource limits but didn’t seriously expect them to help much (the machine was DOSed daily).

Recently I had a user of my Play Machine accidentally DOS it and ask whether I should install any resource limits. After considering the issue I realised that I can actually do so in a useful manner nowadays. My latest Play Machine is a Xen DomU which I have now assigned 300M of RAM, I have configured the limit for root processes to be 45, as the system and my login comprise about 30 processes that leaves 15 for unprivileged (user_r) logins. Of recent times my Play Machine hasn’t been getting a lot of interest, having two people logged in at the same time is unusual so 15 processes should be plenty. Each process is limited to 20M of memory so overflowing the 300M of RAM should take a moderate amount of effort.

Recently I intentionally have not used swap space on that machine to save on noise when there’s a DOS attack (on the assumption that the DOS attack would succeed regardless of the amount of swap). Now that I have put resource limits in place I have installed 400M of swap space. A hostile user can easily prevent other unprivileged users from logging in by keeping enough long-running processes active – but they could achieve the same goal by having a program kill users shells as soon as they login (which a few people did in the early days). But it should not be trivial for them to prevent me from logging in via a simple memory or process DOS attack.

Update: It was email discussion with Cam McKenzie that prompted this blog psot.

Low Power – They Just Don’t get it

January 2, 2008 | etbe

For a while I’ve been reading the Lenovo blog Inside The Box [1], even though I plan to keep my current laptop for a while [2] (and therefore not buy another Thinkpad for a few years) I am interested in the technology for it’s own sake and read the blog.

A recent post concerns a new desktop machine billed as “our greenest desktop ever” [3]. The post has some interesting information on recycling plastic etc, and the fact that the machine in question is physically small (a volume of 4.5L and no PCI expansion slots) means that less petro-chemicals are used in manufacture (and some of the resins used are recycled). However the electricity use is 47W when idle!!!

On my documents blog I have a post about the power use of computers I own(ed) [4] which includes my current Thinkpad (idles at 23W) and an IBM P3 desktop system which idles at 38W. Both machines in question were manufactured before Lenovo bought Thinkpad and IBM’s desktop PC business (so they technically aren’t Lenovo machines) and they weren’t manufactured with recycled resins. But the claim that the new machine is the greenest ever is at best misguided and could be regarded as deceptive.

I think that the machine is quite decent, but it’s obvious that they can do a lot better. There’s no reason that a low-power desktop machine (which uses some laptop technology) should take more than twice the power of what was a high-end laptop a few years ago. Also comparing power use with P3 machines (which are still quite useful now, my IBM P3 desktop runs 24*7 as a server) is quite relevant – and we should keep in mind that before the Pentium was released no system which an individual could afford had anything other than a simple heat-sink to cool it’s CPU.

This is largely a failing of Intel and AMD to make power efficient CPUs and chipsets. It’s also unfortunate that asymmetric multi-processing has not been implemented in recent times. A system with a 64bit CPU core of P3 performance as well as some Opteron class cores that could be suspended independently would be very good for power use with correct OS support. For example when reading documents and email my system will spend most of it’s time idling (apart from when I use Firefox which is a CPU hog) and the CPU use will be minimal for scrolling – a P3 performing core would be more than adequate for that task (which comprises a significant portion of my computer use). Then when I launch a CPU intensive task (composing a blog post in WordPress or compiling) the more powerful CPU cores could start.

It would be good if Intel would release a Pentium-M CPU (32bit) with the latest technology (smaller tracks on the silicon means less power use as well as higher clock speeds). A Pentium-M running at 2GHz produced with the latest Intel fabrication technology would probably use significantly less power than the 1.7GHz Pentium-M that is in my Thinkpad. Put that in a desktop machine and you would have all the compute power you need for most tasks other than playing games and running vista and you could get an idle power less than 23W.

The new Lenovo machine in question does sound like a nice machine, I wouldn’t mind having one for testing and running demos. But the claims made about it seem poorly justified if you know the history.

etbe – Russell Coker

Linux, politics, and other interesting things

Tag Archives: Best Posts