more about Fedora

In a comment on a previous blog entry I was described as an active Fedora advocate, I don’t think that is an accurate description. I advocate it to appropriate people, which is mostly non-programmers – but as I mentioned that means a larger proportion of the population than to whom I can advocate Debian. It’s not that I’m trying to advocate Fedora, just that it fills a need for many people. I believe that the term Fedora advocate means someone has an objective of increasing the use and to use Fedora, I don’t have such an objective. I am a Linux advocate, a Free Software advocate, and sometimes a Unix advocate (Unix meaning the entire family of Unix-like operating systems). Merely promoting something does not make you an advocate for it. I don’t think of myself as a Debian advocate at this time, but as I am a Debian developer this may change.

It seems that the people who run the Fedora Planet think that my blog has suitable Fedora content, it’s been added to that planet. Also the Fedora Planet appears to be running an older version of the Planet software as it has the same problem with my blog that Debian Planet had before the upgrade.

Now on the issue of gratis vs libre: As I am not a Red Hat employee I can’t maintain a kernel-xen-nopae package and give it the same status as the kernel-xen package. Even when I was a Red Hat employee I couldn’t have done that – it would require some amount of management approval. I believe that this fundamentally makes Fedora less of a libre distribution. There is no room in Fedora for someone who is an upstream developer and who just wants to maintain their own package. There is Fedora-Extras, but that has a second-class status. Only Red Hat employees can maintain packages in Fedora Core. This makes Fedora fundamentally less libre than Debian. I am not trying to suggest that Red Hat change things in this regard, I believe that Fedora is meeting all it’s goals and that making Fedora as libre as Debian is not possible given the goals of making a profit on selling support of RHEL.

Chris made a good point. I also believe that MP3 codecs should not be in Debian/main. But I believe that people making mistakes about some issues is not a factor in judging the entire project. I believe that Debian is more libre although some bad decisions were made – largely due to lack of overall management. Fedora has hierarchical management, so when the legal team declares that some software can not be distributed then it gets removed without debate. I guess I could propose a GR to exclude MP3 codecs from main.

Also it should be noted that RHEL Extras has some of this software that is not in Fedora (RealPlayer for example). The Red Hat legal advice was that MP3 codecs need a license, so they ship a licensed version in their commercial distribution. This is the right thing to do for their customers (it’s handy to have and I’m sure that they get a good deal by paying license fees for all their customers) and removing such things from Fedora is the right way to offer a gratis product without unreasonable legal liability.

Naturally Fedora is much more libre than any secret-source OS. Every user has the option of downloading the Fedora source and recompiling it as they wish. I could compile Fedora with a Xen kernel that runs on my hardware and with SE Linux policy that is more restrictive than that which Fedora currently has. I could build custom Fedora install CDs to install things the way I want (which I considered doing when I worked for Red Hat). But the liberty to fork a project does not compare to the liberty to join it, and the liberty to create your own packages in extras does not compare to the liberty to add your own packages that do things differently to the default package.

There are of course positive and negative aspects to this. I started work on SE Linux in Debian in 2001. In 2003 I joined Red Hat to work on SE Linux, in Red Hat I was not the only person dedicated to SE Linux work and other people spent part of their time working on it. The SE Linux work in Red Hat soon eclipsed that of Debian because there was management support. There was no possibility for a package maintainer to refuse to fix a bug that affected SE Linux simply because they didn’t care for it. The positive side of this is that the SE Linux work proceeded quickly and efficiently. The negative side of this is that things which don’t have management support don’t appear in Fedora Core. Exim is a fine MTA but is not in Fedora Core. Some people think that AppArmor is a better option than SE Linux, they are wrong – but in Debian any developer has the option to add AppArmor support and neither I nor any other DD can prevent them. The libre nature of Debian means that as long as basic technical criteria are met DDs can add any package that they wish to the distribution.

These issues however are all related to people who are actively involved in Free Software development. For a typical Free Software user it often doesn’t make much difference, until of course your favourite program doesn’t get management approval to appear in Fedora Core. But the counter argument is that the quality of some of the >10,000 packages in Debian is not so high. You can install a Fedora Core package and have a reasonable expectation about how well it works, but Debian packages are sometimes rather experimental.

I also don’t believe that Debian is a very functional Democracy. Some of the problems of Direct Democracy are demonstrated in Debian. In many ways it is more anarchistic, anarchy gives you liberty for good and bad. Maybe we should consider a Representative Democracy model for Debian.

The benefits of SE Linux

Today I discovered a bug in one of my programs, it called system() and didn’t correctly escape shell eta-characters. Fortunately I had written custom SE Linux policy for it which did domain_auto_trans(foo_t, shell_exec_t, very_restricted_t) so there was no possibility of damage.

The log files (which were not writable by the daemon by both SE Linux access control and Unix permissions) indicated that no-one had attempted to exploit the bug.

dunc-tank and motivation

The dunc-tank project was established to raise money to compensate some Debian developers who are essential to producing a timely release of Debian. There has been a lot of acrimoneous debate about whether this is a good or bad thing. The positive side of it is that the release managers will get to spend more time working on Debian, the negative side is that some volunteers will lose motivation.

However I have felt more motivated to do my unpaid Debian work. During the time that I was employed by Red Hat I was fairly slack about my Debian development work (incidentally Red Hat management were happy for me to continue Debian work so there was no pressure from Red Hat in this regard). Since leaving Red Hat I have been busy doing paid work.

Recently I have started getting involved in Debian work again. I am about to upload a new version of Postal for the first time in three years, I have set up a Xen server for Debian SE Linux development, and I am about to start serious Debian SE Linux development work again.

One factor in this has been my impression that other DDs are taking the release seriously. In the past schedules for release have slipped repeatedly without end. Now there is a schedule and this gives me more motivation to get bugs fixed!

Ruxcon and SLUG

This weekend I was in Sydney for Ruxcon. Ruxcon is a computer security conference with a focus on penetration testing and related skills.

The presentation on Unusual Bugs by Ilya van Sprudel was particularly interesting. He spoke about a number of issues that could do with some improvement in Linux, I will file some bug reports shortly.

There was a chilli eating contest. I was one of six people to enter. I survived the first two rounds and got onto the middle-strength chilli before giving up. There were 100 tickets to the Google party for the ~200 person conference and everyone who entered a contest got a ticket. My aim in the contest was to eat more chilli than I enjoy eating but less than the amount required to make me sick, with a secondary goal of tasting at least the second level of chilli. I achieved my goals and left the contest after tasting the second chilli.

One man appeared to be impressed by my chilli eating and was telling everyone that I am famous for eating chilli. It’s good to be famous for something in the computer security community. :-#

At the end of the conference there was a panel discussion that I was invited to attend. I had to leave early to catch my flight, at the time I left everyone who was on the panel had each finished a few drinks and a couple of new guys had just joined. I think I missed the most exciting part of the panel discussion.

Thanks to whoever paid for the drinks for panel members. Things were a little hectic when we were given the drinks and I forgot to thank whoever paid for them.

In other news Sydney trains are slow and unreasonably expensive, $13 to get from the airport to the SLUG meeting at St. Leonards seems excessive. With all the problems with Sydney roads they really need to get a better public transport system!

While in Sydney I attended a SLUG meeting and gave a short talk about Postal (my mail server benchmark suite). I will present a paper about Postal at the OSDC conference later this year.

SAK, ctrl-alt-del, and Linux keyboard mapping

A common problem with Linux systems is when Windows users press CTRL-ALT-DEL at the login prompt and reboot the machine.

To fix this some people change the ^ca line in /etc/inittab to just disable the reboot function. However this is not desirable because sometimes you want to reboot a machine with a simple keypress.

Another problem that has not been widely considered is the use of fake login prompts by attackers. This can be implemented in either text mode or graphics mode. All the fake login prompt has to do is display something that looks like a real login prompt, accept a user-name and password, verify the password (a localhost ssh connection is a good way of doing this) and then abort. In the case of a text-mode login the user will think that they entered the wrong password, in the case of a GUI login via an XDM program the user will think that the login program just crashed. Then the attacker has access to their account.

The solution to the fake-login problem is the use of the Secure Attention Keyboard (SAK) feature. When invoked this feature makes the kernel kill all processes that are on the virtual console in question. If you make CTRL-ALT-DEL the SAK combination then pressing those keys will cause the kernel to kill any processes that are attached to the current virtual console and preventing the ability of hostile programs to forge a login prompt (which is the same as it’s purpose in Windows).

The next thing to do is to make another combination used for system boot. A reasonable combination seems to be CTRL-ALT-BREAK as those keys are widely separated and the combination is not used for anything else.

If you put the following in a file named sak.map (or whatever you want to call it) then the command loadkeys sak.map will apply the change. Note that when creating a keyboard map you should do it on a machine for which you don’t mind being forced to perform a hardware reboot. It’s easy to make a mistake and give yourself a keyboard mapping that is not usable. Another possibility is to do such testing on a machine that allows ssh logins, you can then login via ssh and run loadkeys -d to correct any errors you might make.

control alt keycode 119 = Boot
control alt keycode 83 = SAK
control alt keycode 111 = SAK
control altgr keycode 119 = Boot
control altgr keycode 83 = SAK
control altgr keycode 111 = SAK

Note that the above covers both ALT and ALT-Gr keys as well as the numeric keypad and regular versions of the delete key.

dumpkeys -l gives you a list of all possible keyboard combinations. showkey will display the number matching any key you press and will exit after 10 seconds of inactivity.

tcpdump and ps

Today I was doing some network tracing and figured out how to track the start and end of TCP connections. The following tcpdump command will get all SYN, FIN, and RST packets on port 80 and all ICMP packets:

tcpdump -i bond0 -n “port 80 and tcp[tcpflags] & (tcp-syn|tcp-fin|tcp-rst) != 0 or icmp”

Also recently I was tracking down some minor security issues related to programs that call setuid() to drop privs but never call setgid() and therefore always run with GID==0 which gives them a lot of access to the system. The following ps command gives the real, effective, saved, and filesystem UIDs and GIDs mapped to names. Note that with some versions of ps different fields have different truncation lengths.

ps -eo pid,user,euser,suser,fuser,group,egroup,sgroup,fgroup,comm

The next thing I have to do is to patch PS to show the supplementary groups.

Ethernet bonding

Bonding is one of the terms used to describe multiple Ethernet cables used to form a single virtual network link. This can be done for performance or reliability.

Bonding for performance used to be common when 100baseT was the fastest network technology that was commonly available. In 1999 servers could usually sustain considerably more than 10MB/s so a single 100baseT network interface was a performance bottleneck. At that time I worked with Cisco switches and Solaris machines that had up to four 100baseT links bonded for performance.

Nowadays Gigabit Ethernet is commonly available, most laptops have Gigabit Ethernet on the motherboard. Gigabit PCI cards are as cheap as $35, and Gigabit switches can be purchased for as little as $139. Server hardware is a little more expensive, but it’s still quite cheap and commonly available.

Most people don’t need more than Gigabit speed, in fact most systems can not saturate a Gigabit link due to poor application design, a slow operating system, or slow disks used to provide the data. So at this time there is little speed for bonded Gigabit networking for performance.

There is still the issue of reliability. Often you want to have two ethernet cards and cables configured so that if one breaks the network won’t go down.

One annoying thing about bonding in Linux (in 2.6.x kernels) is that the module has to be loaded separately for each bond interface, and the parameters for an interface can’t be changed without unloading and loading the driver (very painful if you log in to the machine via ssh over the bonded interface to do sys-admin work).

The parameters I have in /etc/modprobe.conf for bonding are:

alias bond0 bonding
options bond0 mode=1 arp_interval=500 arp_ip_target=192.168.0.1

This means that if there is no traffic on the link then every 500ms an ARP request will be sent for the address 127.128.129.130 (I used the address of my router but substituted a different value for this blog entry). An ARP request for a machine on the local LAN is a request that will always be satisfied if the machine in question and the network link are working.

The idea is that you have two switches and every computer that matters has two ethernet ports. If one port stops working (broken Ethernet card, cable, or router) then the other takes over.

The special file /proc/net/bonding/bond0 can be used to view the current configuration of the bond0 device.

Below are sample configuration files for Fedora and Red Hat Enterprise Linux to configure bonding:

/etc/sysconfig/networking/devices/ifcfg-bond0:
DEVICE=bond0
IPADDR=192.168.0.2
NETMASK=255.255.255.0
NETWORK=192.168.0.0
BROADCAST=192.168.0.255
ONBOOT=yes
BOOTPROTO=static
# GATEWAY should be the IP address to ARP ping
GATEWAY=192.168.0.1
TYPE=Ethernet

/etc/sysconfig/networking/devices/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx
ONBOOT=yes
SLAVE=yes
MASTER=bond0
TYPE=Ethernet

/etc/sysconfig/networking/devices/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx
ONBOOT=yes
SLAVE=yes
MASTER=bond0
TYPE=Ethernet

Note that there is nothing preventing you from having more than two devices bonded together for reliability, but I doubt that you really need that.

car-pooling

I am constantly amazed at the apparent lack of interest in car-pooling when travelling between LUV meetings and the restaurant where we have dinner. After the last meeting I was one of the first five people to arrive at the restaurant and we had arrived in three separate cars. For the most luxurious travel you can have four people to a car and a standard sedan class vehicle can legally and safely carry five people. So an extra seven people could have been comfortably driven to the restaurant and an extra ten people could have been safely and legally driven to the restaurant. But instead most people were waiting in the cold at the tram stop.

Things are quite different in Europe. There was one occasion when after an LSM (Libre Software Meeting) conference in Bordeaux we got 8 people in a Mazda 323, now that’s what I call car-pooling! NB This is dangerous and illegal, so I can’t recommend doing it.