licence for lecture notes

January 19, 2007 | etbe

While attending LCA it occurred to me that the lecture notes from all the talks that I have given lack a copyright notice. So I now retrospectively license my lecture notes in the manner that probably matches what everyone was already doing. The Creative Commons web site has a form to allow you to easily choose a license. So I have chosen the below license, it applies to all lecture notes currently on my web site and all that I publish in future unless they contain special notice of different license conditions.

Update: From now on I am releasing all lecture notes under a non-commercial share-alike license. I had previously not given a specific license to the content on my blog – now I am specifically licensing it under a non-commercial share-alike license. This means (among other things) that you may not put my content on a web page that contains Google AdWords or any other similar advertising.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

new release of postal

January 18, 2007 | etbe

Today I have released a significant new version of my mail server benchmark Postal! The list of changes is below:

Added new program bhm to listen on port 25 and send mail to /dev/null. This allows testing mail relay systems.
Fixed a minor bug in reporting when compiled without SSL.
Made postal write the date header in correct RFC2822 format.
Removed the name-expansion feature, it confused many people and is not needed now that desktop machines typically have 1G of RAM. Now postal and rabid can have the same user-list file.
Moved postal-list into the bin directory.
Changed the thread stack size to 32K (used to be the default of 10M) to save virtual memory size (not that this makes much difference to anything other than the maximum number of threads on i386).
Added a minimum message size option to Postal (now you can use fixed sizes).
Added a Postal option to specify a list of sender addresses separately to the list of recipient addresses.
Removed some unnecessary error messages.
Handle EINTR to allow ^Z and “bg” from the command line. I probably don’t handle all cases, but now that I agree that failure to handle ^Z is an error I expect bug reports.
Made the test programs display output on the minute, previously they displayed once per minute (EG 11:10:35) while now it will be 11:10:00. This also means that the first minute reported will have something less than 60 seconds of data – this does not matter as a mail server takes longer than that to get up to speed.
Added support for GNUTLS and made the Debian package build with it. Note that BHM doesn’t yet work correctly with TLS.
Made the programs exit cleanly.

Thanks to Inumbers for sponsoring the development of Postal.

I presented a paper on mail server performance at OSDC 2006 that was based on the now-released version of Postal.

I’ve been replying to a number of email messages in my Postal backlog, some dating back to 2001. Some of the people had changed email address during that time so I’ll answer their questions in my blog instead.

/usr/include/openssl/kssl.h:72:18: krb5.h: No such file or directory
In file included from /usr/include/openssl/ssl.h:179

One problem reported by a couple of people is having the above error when compiling on an older Red Hat release (RHL or RHEL3). Running ./configure –disable-ssl should work-around that problem (at the cost
of losing SSL support). As RHEL3 is still in support I plan to fix this bug eventually.

There was a question about how to get detailed information on what Postal does. All three programs support the options -z and -Z to log details of what they do. This isn’t convenient if you only want a small portion of the data but can be used to obtain any information you desire.

One user reported that options such as -p5 were not accepted. Apparently their system had a broken implementation of the getopt(3) library call used to parse command-line parameters.

political compass

January 17, 2007 | etbe

It appears that some people don’t understand what right-wing means in terms of politics, apart from using it as a general term of abuse.

I recommend visiting the site http://www.politicalcompass.org/ to see your own political beliefs (as determined by a short questionnaire) graphed against some famous people. The unique aspect of the Political Compass is that they separate economic and authoritarian values. Stalinism is listed as extreme authoritarian-left and Thatcherism as medium authoritarian-right. Nelson Mandela and the Dalai Lama are listed as liberal-left.

I score -6.5 on the economic left/right index and -6.46 on the social libertarian/authoritarian index, this means that I am fairly strongly liberal-left. Previously the Political Compass site would graph resulta against famous people but they have since removed the combined graph feature and the scale from the separate graphs. Thus I can’t determine whether their analysis of the politics of Nelson Mandela and the Dalai Lama indicate that one of those men has beliefs that more closely match mine than the other. I guess that this is because the famous politicians did not take part in the survey and an analysis of their published material was used to assess their beliefs, this would lead to less accuracy.

The Wikipedia page on Right-Wing Politics provides some useful background information. Apparently before the French revolution in the Estates General the nobility sat on the right of the president’s chair. The tradition of politically conservative representatives sitting on the right of the chamber started there, I believe that such seating order is still used in France while in the rest of the world the terms left and right are used independently of seating order.

Right-wing political views need not be associated with intolerance. If other Debian developers decide to publish their political score as determined by the Political Compass quiz then I’m sure that we’ll find that most political beliefs are represented, and I’m sure that most people will discover that someone who they like has political ideas that differ significantly from their own.

Some ideas for running a conference

January 16, 2007 | etbe

Firstly for smooth running of the presentations it would be ideal if laptops were provided for displaying all presentations (obviously this wouldn’t work for live software demos but it would work well for the slide-show
presentations). Such laptops need to be tested with the presentation files that will be used for the talks (or pre-release versions that are produced in the same formats). It’s a common problem that the laptops owned by the speakers will have problems connecting to the projectors used at the conference which can waste time and give a low quality display. Another common problem is that laptops owned by the conference often have different versions of the software used for the slides which renders them differently, the classic example of this is OpenOffice 1.x and 2.x which render presentations differently such that using the wrong one results in some text being off-screen.

The easy solution to this is for the conference organizers to provide laptops that have multiple boot options for different distributions. Any laptop manufactured in the last 8 years will have enough disk space for the
latest release of Debian and the last few Fedora releases. As such machines won’t be on a public network there’s no need to apply security updates and therefore a machine can be used at conferences in successive years, a 400MHz laptop with 384M of RAM is quite adequate for this purpose while also being so small that it will sell cheaply.

A slightly better solution would be to have laptops running Xen. It’s not difficult to set up Xephyr in fullscreen mode to connect to a Xen image, you could have several Xen instances running with NFS file sharing so that the speaker could quickly test out several distributions to determine which one gives the best display of their notes. This would also allow speakers to bring their own Xen images.

This is especially important if you want to run lightning talks, when there is only 5 minutes allocated for a talk you can’t afford to waste 2 minutes in setting up a presentation!

In other news Dean Wilson gave my talk yesterday a positive review.

some random Linux tips

January 10, 2007 | etbe

echo 1 > /proc/sys/vm/block_dump
The above command sets a sysctl to cause the kernel to log all disk writes. Below is a sample of the output from it. Beware that there is a lot of data.
Jan 10 09:05:53 aeon kernel: kjournald(1048): WRITE block XXX152 on dm-6
Jan 10 09:05:53 aeon kernel: kjournald(1048): WRITE block XXX160 on dm-6
Jan 10 09:05:53 aeon kernel: kjournald(1048): WRITE block XXX168 on dm-6
Jan 10 09:05:54 aeon kernel: kpowersave(5671): READ block XXX384 on dm-7
Jan 10 09:05:54 aeon kernel: kpowersave(5671): READ block XXX400 on dm-7
Jan 10 09:05:54 aeon kernel: kpowersave(5671): READ block XXX408 on dm-7
Jan 10 09:05:54 aeon kernel: bash(5803): dirtied inode XXXXXX1943 (block_dump) on proc
Prefixing a bash command with ‘ ‘ will prevent a ! operator from running it. For example if you had just entered the command ” ls -al /” then “!l” would not repeat it but would instead match the preceeding command that started with a ‘l’. On SLES-10 a preceeding space also makes the command not appear in
the history while on Debian/etch it does (both run Bash 3.1).

LD_PRELOAD=/lib/libmemusage.so ls > /dev/null

The above LD_PRELOAD will cause a dump to stderr of data about all memory allocations performed by the program in question. Below is a sample of the output.

Memory usage summary: heap total: 28543, heap peak: 20135, stack peak: 9844
         total calls   total memory   failed calls
 malloc|         85          28543              0
realloc|         11              0              0   (in place: 11, dec: 11)
 calloc|          0              0              0
   free|         21          12107
Histogram for block sizes:
    0-15             29  30% ==================================================
   16-31              5   5% ========
   32-47             10  10% =================
   48-63             14  14% ========================
   64-79              4   4% ======
   80-95              1   1% =
   96-111            20  20% ==================================
  112-127             2   2% ===
  208-223             1   1% =
  352-367             4   4% ======
  384-399             1   1% =
  480-495             1   1% =
 1536-1551            1   1% =
 4096-4111            1   1% =
 4112-4127            1   1% =
12800-12815           1   1% =

cooling

January 8, 2007 | etbe

Recently there has been some really hot weather in Melbourne that made me search for alternate methods of cooling.

The first and easiest method I discovered is to keep a 2L bottle of water in my car. After it’s been parked in the sun on a hot day I pour the water over the windows. The energy required to evaporate water is 2500 Joules per gram, this means that the 500ml that probably evaporates from my car (I guess that 1.5L is split on the ground) would remove 1.25MJ of energy.from my car – this makes a significant difference to the effectiveness of the air-conditioning (the glass windows being the largest hot mass that can easily conduct heat into the cabin).

It would be good if car designers could incorporate this feature. Every car has a system to spray water on the wind-screen to wash it, if that could be activated without the wipers then it would cool the car significantly. Hatch-back cars have the same on the rear window, and it would not be difficult at the design stage to implement the same for the side windows too.

The next thing I have experimented with is storing some ice in a room that can’t be reached by my home air-conditioning system. Melting ice absorbes 333 Joules per gram. An adult who is not doing any physical activity will produce about 100W of heat, that is 360KJ per hour. Melting a kilo of ice will abrorb 333KJ per hour, if the amount of energy absorbed when the melt-water approaches room temperature is factored in then a kilo of ice comes close to absorbing the heat energy of an adult at rest. Therefore 10Kg of ice stored in your bedroom will prevent you from heating it by your body heat during the course of a night.

In some quick testing I found that 10Kg of ice in three medium sized containers would make a small room up to two degrees cooler than the rest of the house. The ice buckets also have water condense on them. In a future experiement I will measure the amount of condensation and try and estimate the decrease in the humidity. Lower humidity makes a room feel cooler as sweat will evaporate more easily. Ice costs me $3 per 5Kg bag, so for $6 I can make a hot night significantly more bearable. In a typical year there are about 20 unbearably hot nights in Melbourne. So for $120 I can make one room cooler on the worst days of summer
without the annoying noise of an air-conditioner (the choice of not sleeping due to heat or not sleeping due to noise sucks).

The density of dry air at 0C and a pressure of 101.325 kPa is 1.293 g/L.

A small bedroom might have an area of 3M*3M and be 2.5M high giving a volume of 22.5M^3 == 22,500L. 22,500 * 1.293 = 29092.500g of air.

One Joule can raise the temperature of one gram of cool dry air by 1C.

Therefore when a kilo of ice melts it would be able to cool the air in such a room by more than 10 degrees C! The results I observe are much smaller than that, obviously the walls, floor, ceiling, and furnishings in the room also have some thermal energy, and as the insulation is not perfect some heat will get in from other rooms and from outside the house.

If you have something important to do the next day then spending $6 or $12 on ice the night before is probably a good investment. It might even be possible to get your employer to pay for it, I’m sure that paying for ice would provide better benefits in employee productivity than many things that companies spend money on.

Xephyr

January 7, 2007 | etbe

As part of my work on Xen I’ve been playing with Xephyr (a replacement for Xnest). My plan is to use Xen instances for running different versions of desktop environments. You can’t just ssh -X to a Xen image and run things. One problem is that some programs such as Firefox do strange things to try and ensure that you only have one instance running. Another problem is with security, the X11 security extensions don’t seem to do much good. A quick test indicates that a ssh -X session can’t copy the window contents of a ssh -Y session, but can copy the contents of all windows run in the KDE environment. So this extension to X (and the matching ssh support) seem to do little good.

One thing I want to do is to have a Xen image for running Firefox with risky extenstions such as Flash and keep it separate from my main desktop for security and managability.

Xephyr :1 -auth ~/.Xauth-Xephyr -reset -terminate -screen 1280×1024

My plan is to use a command such as the above to run the virtual screen. That means to have a screen resolution of 1280×1024, to terminate the X server when the last client exits (both the -reset and the -terminate options are required for this), to be display :1 and listen with TCP (the default), and to use an authority file named ~/.Xauth-Xephyr.

xauth generate :1 .

The first problem is how to generate the auth file, the xauth utility is documented as doing it via the above command. But this really connects to a running X server and copies the auth data from it.

The solution (as pointed out to me by Dr. Brian May) is to be found in the startx script which solves this problem. The way to do it is to use the add :1 . $COOKIE command in xauth to create the auth file used by the X server, and to generate the cookie with the mcookie program.

In ~/.ssh/config:
Host server
SendEnv DISPLAY

In /etc/ssh/sshd_config:
AcceptEnv DISPLAY

The next requirement is to tell the remote machine (which incidentally doesn’t need to be a Xen virtual machine, it can be any untrusted host that contains X applications you want to run) which display to use. The first thing to do is to ssh to the machine in question and run the xauth program to add the same cookie as is used for the X server. Then the DISPLAY environment variable can be sent across the link by setting the ~/.ssh/config file at the client end to have the above settings (where server is the name of the host we will connect to via SSH) and in the sshd_config file on the server have the line AcceptEnv DISPLAY to accept the DISPLAY environment variable. It would have been a little easier to configure if I had added the auth entry to the main ~/.Xauthority file and used the command DISPLAY=:1 ssh -X server, this would be the desired configuration when operating over an untrusted network. But when talking to a local Xen instance it gives better performance to not encrypt the X data.

The following script will generate an xauth entry, run a 1280×1024 resolution Xephyr session, and connect to the root account on machine server and run the twm window manager. Xephyr will exit when all X applications end. Note that you probably want to use passwordless authentication on the server as typing a password twice to start the session would be a drag.

#!/bin/sh

COOKIE=`mcookie`
FILE=~/.Xauth-Xephyr
rm -f $FILE
#echo “add 10.1.0.1:1 . $COOKIE” | xauth
ssh root@server “echo \”add 10.1.0.1:1 . $COOKIE\” | xauth”
echo “add :1 . $COOKIE” | xauth -f $FILE
Xephyr :1 -auth $FILE -reset -terminate -screen 1280×1024 $* &
DISPLAY=10.1.0.1:1 ssh root@server twm
wait

core files

January 5, 2007 | etbe

The issue of core file management has come up for discussion again in the SE Linux list.

I believe that there are two essential security requirements for managing core files, one is that the complete security context of the crashing process is stored (to the greatest possible extent), and the other is that processes with different security contexts be prevented from discovering that a process dumped core (when attacking a daemon it would be helpful to know when you made one of it’s processes dump core).

The core file will have the same UID and GID as the process that crashed. It’s impossible to maintain the complete security context of the crashing process in this manner as Unix permissions support multiple supplementary groups and Unix filesystems only support one GID. So the supplementary groups are lost.

There is also a sysctl kernel.core_pattern which specifies the name of the core file. This supports a number of modifiers, EG the value “core.%p.%u.%g” would give a file named “core.PID.UID.GID“. It would be good to have a modification to the kernel code in question to allow the SE Linux context to be included in this (maybe %z).

To preserve the SE Linux context of the crashing process with current kernel code we need to have a unique type for each process that dumps core, this merely requires that each domain have an automatic transition rule for creating files in the directory chosen for core dumps. In the default configuration we have core files dumped in the current directory of the process. This may be /tmp or some other common location which allows an attacker to discover which process is dumping core (due to the directory being world readable) and in the case of SE Linux there may be multiple domains that are permitted to create files in /tmp with the same context which gets in the way of using such a common directory for core files.

The traditional Unix functionality is to have core files dumped in the current directory. Obviously we can’t break this by default. But for systems where security is desired I believe that the correct thing to do is to use a directory such as /var/core for core files, this can be easily achieved by creating the directory as mode 1733 (so that any user can create core files but no-one but the sys-admin can read them) and then setting the core_pattern sysctl to specify that all core files go in that directory. The next improvement is to have a poly-instantiated directory for /var/core such that each login user has their own version. That way the user in question could see the core files created by their own processes while system core files and core files for other users would be in different directories. Poly-instantation is easier to implement for core files than it is for /tmp (and the other directories for which it is desirable) because there is much less access to such a directory. When things operate correctly core files are not generated, and users never need to access each other’s core files directly (they are mode 0600 so this isn’t possible anyway).

This area will require a moderate amount of coding before it works in the ideal manner. I aim to briefly describe the issues only in this post.

monitors for developers

January 4, 2007 | etbe

Michael Davies recently blogged that all developers should have big screens. This news has been around for a while, the most publicity for the idea came from Microsoft Research where they did a study showing that for certain tasks a 50% performance increase could be gained from a larger monitor.

If you consider that a good software developer will get paid about $100K and it’s widely regarded that in a corporate environment the entire costs for a worker (including management, office space, etc) is double their base salary then you consider each developer to be worth $200K per annum. Therefore larger and more monitors could potentially give a benefit in excess of $100K per annum (we assume that the value provided by the developer is greater than their salary – apart from the dot-com boom that’s usually the case).

It’s quite obvious that you can get a really great monitor configuration for significantly less than $100K and that it will remain quite current for significantly more than a year (monitor technology advances comparatively slowly so a good monitor should last at least four years).

Some time ago I researched this matter for a client. I convinced all the managers in my area, I convinced a bunch of colleagues to buy bigger monitors for their homes (and bought one myself), but unfortunately senior management saw it as a waste of money. I was trying to convince them that people who were being paid >$100,000 should be each assigned a $400 monitor. Sadly they believed that spending the equivalent of less than a day’s wages per employee was not justified.

If I was in a management position I would allocate an amount of money for each developer to spend on hardware or conferences at their own discretion. I would make that amont of money a percentage of the salary of each employee, and I would also allow them to assign some of their share to a colleague if they had a good reason for it (EG if a new hire needed something expensive that would exceed their budget for the first year). I think that people who are good programmers are in the best position to judge what can best be done to improve their own productivity, and that allowing them to make their own choices is good for morale.

On a more technical level I have a problem with getting a big monitor. I do most of my work on a laptop because I travel a lot. I don’t travel as much as I did while living in Europe but still do a lot of coding in many strange places. I started writing my Postal benchmark in the hotel restaurant of a Bastion hotel in Utrecht during one of the worst recorded storms in Europe (the restaurant had huge windows and it was inspirational for coding). I wrote the first version of my ZCAV benchmark in Denver airport while waiting for a friend.

What I need is a good way of moving open windows from my laptop to a big external display and then back again. I don’t want to logout before moving my machine. With a Macintosh this is quite possible (I’m using a OS/X machine while working for a client and the only thing that has impressed me is the monitor support). With Linux things aren’t so easy, it’s supposed to be possible but I haven’t heard any unqualified success stories yet.

I guess I could try setting up XDMCP between my laptop and a desktop machine with some big displays and logout before moving.

Any suggestions?

Update, here are some comments from the original version of this post as Blogspot:

Anonymous said:
You want xrandr. The newest version, 1.2 (needed on client, server, and drivers), will allow you to extend your X session onto a new display and back off of it without reconfiguring anything. However, even the current version can switch you onto and off of an external monitor if you set it up in advance. Just set up the appropriate Virtual area, and the right set of modes to include and exclude the external monitor. When you have it set up right, you will see in xrandr’s list of modes both the mode to use the internal display only, with its resolution, and the mode to use both displays, with the combined resolution. For instance, I have a 1400×1050 internal LCD and a 1680×1050 external LCD, so I see modes for 1400×1050, 1680×1050, and 3080×1050.
Praveen Kumar said:
What you are saying is absolutely true. I have experienced the improved productivity myself. I would recommend people running dual screen setup on their laptop. I am running dual screen setup where my laptop LCD runs a screen at 1024×768 and an external monitor (21″) runs other screen at 1600×1200 using xinerama. It paid well so far.
Anonymous said:
I’m using “synergy” at home. I’ve 2 PCs and use both with just one keyboard/mouse, and I usually put my laptop between both screens, and an exit-hook of dhclient automatically starts synergy on my laptop if I’m at home so this third screen is just inserted between the two others..
Anonymous said:
Because the DPI varies so greatly between displays (little high-res laptop vs. external so large that pixels are sizable), I think it’s necessary to configure each monitor with its own DPI in X. This requires a multiple X screen setup, xinerama won’t do.
Berge Schwebs BjÃ¸rlo said:
You could take a look at Xdmx (http://dmx.sourceforge.net). It’s a multihead Xinerama-like X-server. Which basically means you can connect X-servers on different machines together and have xinerama-over-network. Pretty neat. People have been using it for making seriously large screens (http://www.evl.uic.edu/cavern/lambdavision/).
Please give a shout if you get it working (-:
Lionel Porcheron said:
I have experienced xrandr (which integrates quite well with Gnome) recentely and it works well. The only drawback is when you want to switch from “extend screen” with an external monitor to “mirror screen” for a presentation: you need to edit xorg.conf actually (If I am correct). Otherwise, it works quite well.
SÃ¸ren Hansen said:
You might want to take a peek at xmove. It’s kind of like screen, but for X.

Xen shared storage

January 3, 2007 | etbe

disk = [ ‘phy:/dev/vg/xen1,hda,w’, ‘phy:/dev/vg/xen1-swap,hdb,w’, ‘phy:/dev/vg/xen1-drbd,hdc,w’, ‘phy:/dev/vg/san,hdd,w!’ ]

For some work that I am doing I am trying to simulate a cluster that uses fiber channel SAN storage (among other things). The above is the disk line I’m using for one of my cluster nodes, hda and hdb are the root and swap disks for a cluster node, hdc is a DRBD store (DRBD allows a RAID-1 to be run across the cluster nodes via TCP), and hdd is a SAN volume. The important thing to note is the “w!” mode for the device, this means write access is granted even in situations whre Xen thinks it’s unwise (IE it’s being used by another Xen node or is mounted on the dom0). I’ve briefly tested this by making a filesystem on /dev/hdd on one node, copying data to it, then umounting it and mounting it on another node to read the data.

There are some filesystems that support having multiple nodes mounting the same device at the same time, these include CXFS, GFS, and probably some others. It would be possible to run one of those filesystems across nodes of a Xen cluster. However that isn’t my aim at this time. I merely want to have one active node mount the filesystem while the others are on standby.

One thing that needs to be solved for Xen clusters is fencing. When a node of a cluster is misbehaving it needs to be denied access to the hardware in case it recovers some hours later and starts writing to a device that is now being used by another node. AFAIK the only way of doing this is via the xm destroy command. Probably the only way of doing this is to have a cluster node ssh to the dom0 and then run a setuid program that calls xm destroy.

etbe – Russell Coker

Linux, politics, and other interesting things

Tag Archives: Best Posts