etbe - Russell Coker

Redirecting Output from a Running Process

Someone asked on a mailing list how to redirect output from a running process. They had a program which had been running for a long period of time without having stdout redirected to a file. They wanted to logout (to move the laptop that was used for the ssh session) but not kill the process (or lose output).

Most responses were of the form “you should have used screen or nohup” which is all very well if you had planned to logout and leave it running (or even planned to have it run for a long time).

Fortunately it is quite possible to redirect output of a running process. I will use cat as a trivial example but the same technique will work for most programs that do simple IO (of course programs that do terminal IO may be more tricky – but you could always redirect from the tty device of a ssh session to the tty device of a screen session).

Firstly I run the command “cat > foo1” in one session and test that data from stdin is copied to the file. Then in another session I redirect the output:

Firstly find the PID of the process:
$ ps aux|grep cat
rjc 6760 0.0 0.0 1580 376 pts/5 S+ 15:31 0:00 cat

Now check the file handles it has open:
$ ls -l /proc/6760/fd
total 3
lrwx—— 1 rjc rjc 64 Feb 27 15:32 0 -> /dev/pts/5
l-wx—— 1 rjc rjc 64 Feb 27 15:32 1 -> /tmp/foo1
lrwx—— 1 rjc rjc 64 Feb 27 15:32 2 -> /dev/pts/5

Now run GDB:
$ gdb -p 6760 /bin/cat
GNU gdb 6.4.90-debian
Copyright (C) 2006 Free Software Foundation, Inc
[lots more license stuff snipped]
Attaching to program: /bin/cat, process 6760
[snip other stuff that’s not interesting now]
(gdb) p close(1)
$1 = 0
(gdb) p creat(“/tmp/foo3”, 0600)
$2 = 1
(gdb) q
The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: /bin/cat, process 6760

The “p” command in GDB will print the value of an expression, an expression can be a function to call, it can be a system call… So I execute a close() system call and pass file handle 1, then I execute a creat() system call to open a new file. The result of the creat() was 1 which means that it replaced the previous file handle. If I wanted to use the same file for stdout and stderr or if I wanted to replace a file handle with some other number then I would need to call the dup2() system call to achieve that result.

For this example I chose to use creat() instead of open() because there are fewer parameter. The C macros for the flags are not usable from GDB (it doesn’t use C headers) so I would have to read header files to discover this – it’s not that hard to do so but would take more time. Note that 0600 is the octal permission for the owner having read/write access and the group and others having no access. It would also work to use 0 for that parameter and run chmod on the file later on.

After that I verify the result:
ls -l /proc/6760/fd/
total 3
lrwx—— 1 rjc rjc 64 2008-02-27 15:32 0 -> /dev/pts/5
l-wx—— 1 rjc rjc 64 2008-02-27 15:32 1 -> /tmp/foo3 <====
lrwx—— 1 rjc rjc 64 2008-02-27 15:32 2 -> /dev/pts/5

Typing more data in to cat results in the file /tmp/foo3 being appended to.

Update: If you want to close the original session you need to close all file handles for it, open a new device that can be the controlling tty, and then call setsid().

Future Versions of Windows

There is currently a lot of speculation about the future of Windows following the massive failure of Vista in the market.

One theory that is being discussed is that Microsoft will cease kernel development and adopt a Unix kernel in the same way that Apple adopted a BSD based kernel.

I predict that MS in it’s current incarnation (*) will never do that. Having an OS kernel that enables easy porting of code to/from other platforms is entirely against their business model which relies on incompatibility to lock customers in. Whatever kernel MS use, it has to be incompatible in some ways with everything else. One easy way of achieving this would be to have a shared object (DLL) interface published and have the interface between the libc and other libraries and the kernel be undocumented and ever-changing (simply renumbering the system calls on every minor version increment would be a good start). The DLL interface could then have the complex APIs that MS loves to force on their victims (see Stewart Smith’s post about getting a file size in Windows for an example of the horror [1]).

The advantage of this approach would be that MS could cease developing an OS kernel (something that they were never much good at) and concentrate on owning the proprietary DLLs. There would be nothing stopping them from using a Linux kernel for this, as long as they release all source to the kernel they use (including the patch to renumber the system calls) they would be within the terms of the GPL.

My specific prediction is that some time between Jan 2011 and Dec 2016 Microsoft will release systems with the majority of the kernel code coming from BSD or Linux as their primary desktop and server operating systems.

Could people who disagree please make specific predictions for the future (including dates and actions) so that we can determine who was most accurate.

(*) For future incarnations of Microsoft after chapter 11 or being split in the way that AT&T was there seems no possibility to predict their actions.

[1] http://www.flamingspork.com/blog/2008/02/11/getting-a-file-size-on-windows/

Chilled Memory Attacks

In 1996 Peter Gutmann wrote a paper titled “Secure Deletion of Data from Magnetic and Solid-State Memory” [1]. In that paper he mentions the fact that the contents of RAM last longer at lower temperatures and suggests that data could be retained for weeks at a temperature of -60C or lower (while 140C causes rapid data loss). The paper also addresses issues of data recovery from hard drives, but given that adequate CPU power for encryption is available recovering data from a disk shouldn’t be an issue unless thee attacker can get the key to decrypt it or crack the algorithm – so disk recovery is not a hot issue at the moment.

Recently some researchers at Princeton University have published a paper describing in detail how to chill RAM to make it keep its data after a power cycle and even after being installed in a different computer [2]. This attracted a lot of attention, while Peter’s paper described the theoretical concept (in great detail) the Princeton group showed how to implement the attack using materials that are commonly available.

Most of the analysis of this misses some fundamental points. Any suggestion that you can wipe the RAM on power failure or on boot misses the point entirely. If an attacker can chill a DIMM and then remove it from the system then there is no chance for it to be wiped. Maybe if you had security on the PC case to detect case opening (some servers have a basic version of this) such things would do a little good, but it shouldn’t be difficult to bypass in most cases.

Another common flawed analysis is to suggest that this is no big deal because sniffing the memory bus has been possible for years. While it has always been possible for government agencies and companies who design motherboards to sniff the bus, for most potential attackers it has been overly difficult.

When considering the effectiveness of a security system you should first consider what your threat model is. Who is going to attack you and what resources will they be willing and able to devote to the attack? An organisation that is prepared to use expensive equipment and highly trained people to break your encryption probably has other methods of gaining access to your secret data that are easier and cheaper.

The research from Princeton suggests that I could perform such attacks with my spare time and with equipment that is very cheap. I’ve been idly considering doing this to an old PC just for fun! Therefore I have to assume that everyone who has the same amount of skill and money as me can potentially compromise my data if they capture one of my machines.

It is still most likely that if anyone steals my laptop they will want to sell it and use the money to buy drugs. I don’t think that I have any data that is anywhere near valuable enough to justify a targeted mugging. But my procedures (in terms of changing passwords etc) in the case of my laptop being stolen now need to be scaled up due to the ease in which data might be compromised.

The best way of dealing with this would be to have the decryption keys locked inside the CPU (stored in registers or memory that’s locked in the CPU cache). The possibility of getting a modern CPU to operate at any temperature approaching -60C is laughable, and the CPU is a well contained package that can operate on its own and is difficult to attack. This would make things significantly more difficult for an attacker while requiring little effort (in fact it might be possible to lock data in the CPU cache already in which case a software change is all that is required).

Update: A comment by Mike made a good point about CPU cooling. Toms Hardware performed an overclocking experiment (from 3.2GHz to 5.25GHz) and used liquid nitrogen cooling [3]. It might be possible to cool a CPU core to -60C in a reasonably small amount of time. But I still believe that it would raise the bar enough to make it worth doing.

Update2: Thanks Jaime for the spelling advice.

EeePC for a Teenager

I was asked for advice from a non-technical person about buying a laptop for their 15yo daughter. They were looking to spend $200 or $300.

I suggested that stretching their budget to $500 for a new EeePC would be a better option than the risks associated with a second-hand laptop (laptops are all different in their hardware).

I also suggested that waiting a few months to see what happens with the new EeePCs might be a good idea. There are rumours about the development of new EeePC models with bigger screens, this would be a benefit for those who can afford it, and for those who can’t the current model may drop in price. They took that suggestion very well and I suspect that they weren’t planning to purchase in the near future anyway.

I asked for advice on the LUV mailing list [1] for further suggestions that I could offer.

One suggestion was that larger laptops of a more traditional design have been driven down in price recently and might be worth considering. I will have to mention it, but suggesting that someone who wants to spend $200 should spend $500 seemed enough of a stretch, suggesting that they spend $700 or more might not be taken well. Of course there is the weight issue that Matt raised in a blog post [2]. The last thing I want to do is to convince people not to buy computers for their children!

A complaint was raised about the size of the keyboard in the EeePC (personally I wouldn’t want to use anything smaller than the T series Thinkpad I’m using now). In a quick google search the best information I could find about average finger sizes was the Danforth Diamond blog [3] which says that in the US women average a ring size of 6 while men average 10. According to the ring sizing page on the same site [4] that means ring finger diameter of 16.5 and 19.8mm respectively. So it seems likely that given the same level of typing skill an average woman could happily use a keyboard that’s 83% the size of the keyboard that an average man uses. Of course there’s a lot of variation between individuals, for example my fingers are thinner than average and I prefer a full-sized keyboard.

One thing I recommend when considering a keyboard (or a laptop) is to type your name a few times quickly. Typing something about a quick brown fox will take some thought, but typing your own name requires no thought so you can concentrate on the feel of the keyboard. Using this method it is very easy to go to a store and determine whether a laptop is likely to suit you.

The screen resolution and size is a significant issue, one EeePC owner described it as “too small to do anything useful“. Rumour has it that the next version will have a screen that’s a significant increase in size and resolution.

A final interesting comment was about the social aspect of owning a laptop. The suggestion was made that a teenager who owned the smallest (and possibly cheapest) laptop in class might have other kids tease them about it. I will be interested to see comments on this issue, it had never occurred to me that the laptop use in schools would be high enough for the ownership of any type of laptop to be a cause for harassment. Also I’m not sure which would be considered as being of lower status, an old second-hand laptop or a new low-end laptop. I’m pretty sure that owning either would be considered to have higher status than having no laptop.

A quick scan of the GraysOnline.com.au auction site suggests that a second-hand Pentium-M laptop (a few years old) could be purchased for about $550 and a Core 2 Duo based laptop might go for about $700. So it seems unlikely that for less than $500 you could buy something suitable for a novice user (something that will just work with recent software). I’m not planning on offering the full IT support package that I offer to my relatives so I can only recommend things that can be self-supported.

Of course as the people doing the buying seem to have no definite plans for the use of the computer then anything they buy will fail in some way to meet the requirements that they discover. :-#

Update: fixed a bad link.

Giving Away Hardware

For the last few years I have been actively seeking free hardware to give to members of my local LUG. Whenever a friend or business associate mentions that they are upgrading or replacing computers I enquire what they plan to do with the old ones and request that the old gear be given to me if there are no other plans for it. There is a moderate amount of hardware that I use for my own purposes, but the free hardware that is available is often in excess of my requirements and also sometimes just unsuitable for my use (I am happy to install a second-hand IBM or HP machine for a client but I won’t install a white-box machine).

One organisation that I sometimes give computers to is Computerbank [1]. The purpose of Computerbank is to take donations of old machines, fix them and install Linux, and then sell them for extremely low prices to people who can’t afford new machines. It’s been a while since I gave them any computers because for a long time the minimum specs on machines that they were willing to accept were higher than the machines that I obtained.

Generally I offer my old hardware to the mailing list of Linux Users of Victoria [2]. I offer not only working systems but also broken systems and other things that might be useless to most people – but are greatly desired by the small minority who can use them. One member of that list wants PC power supplies for repairing other electronic devices, so I collect batches of machines that are broken but appear to have working PSUs and give them to him. I once received a box of free two-button mice. I offered them to LUV members expecting that many people would want one or two for test machines. No-one wanted them for use as mice but one guy wanted all of them to use their sensors in robotics projects.

One thing that impresses me is the community spirit demonstrated. Often I will offer some free machines and the first response will be something like “I’d like to take that machine apart for the bits, but if someone wants a complete system please give it to them instead”. There aren’t many occasions when you see someone suggesting that they may not be the most deserving recipient for something that is free!

My aims in this effort are to help random Linux users in my area, and to help the environment by reducing the amount of land-fill. My efforts aren’t going to make a significant impact on the environmental situation, but they do make a significant impact on the availability of hardware for members of the Linux community – which seems to be of particular interest to people who want cheap machines for their children or grand-children.

I encourage other people to do similar things.

One thing that impressed me was the organisation of used hardware gifts at LCA. Near the start of the conference hardware was given away to anyone who put their hand up. At the end of the conference more hardware was given away (I expect it was mostly by delegates who lived locally). It would be good if this idea (which worked so well) was spread to other conferences.

[1] http://vic.computerbank.org.au/
[2] http://www.luv.asn.au/

Watching While Waiting

Over the past four years every visit to a doctor or hospital has involved some time spent in a waiting room, in the case of hospital visits it has often been more than an hour waiting. Each of those waiting rooms has had a selection of bad magazines and a TV. If I ever visit a hospital after 8PM then I might have a chance of seeing something that I want to watch on TV, but otherwise it’s all awful.

Fortunately there are better things that they could do. It would not be difficult to get a basic PC and install all the content from ted.com (and possibly other sites with quality free content) and have some sort of video juke-box program running. TED not only has lectures about arts and science but also performances by musicians and dancers. The chance that a random person can find something they enjoy in TED’s archives seems significantly greater than the chance of finding something worth watching in the daytime-soap TV segment.

But TED is far from the only option available, there are quite a number of free video sites and the ones that include advertising in the video stream would probably be more than happy to have their content copied on to juke boxes.

Service Videos

I just read interesting blog post about Lenovo service information [1]. They have huge documents about how to service their machines as well as apparently having videos in flash format to show how to perform various tasks.

The first thing I’d like to see is other companies following this example. I clearly recall one time when I had an IBM 2RU server apart and I couldn’t work out how to get the power cable for the tape drive installed, a video of how to do so would surely have saved me some time then, I was working for a law firm and had a small outage window – not a situation where I wanted to waste any time.

The next thing I would like to see is Lenovo not using Flash for this, but instead allow download of MP4 and/or OGG video. When doing a job in a tight time window with a demanding client I would like to have all the relevant documentation (including videos) on my laptop before I visit their site.

[1] http://lenovoblogs.com/connections/?p=17

Athlon Memory Problems

I had an old Compaq Athlon 1GHz system that seemed to be broken. It would display random things on the screen from the BIOS and fail the boot, it looked like a motherboard problem. Fortunately before I gave it away (I give away all my broken machines to members of my local LUG who want spare parts) I remembered that the machine uses main memory for video. I removed and reinstalled one of the DIMMs and it then worked perfectly, presumably it was making a poor contact.

The next task was to put it back in service, I had a bunch of spare RAM and I decided to upgrade it to 768M (it previously had 384M). With the new RAM it would fail to boot, sometimes it would give a kernel message “soft lockup detected“, and sometimes it would just hang. The hangs typically occurred when running the initramfs (the first non-kernel code). I tested this with a Fedora install (what the machine used to run when it had 384M), a Debian install (the aim was to use a hard drive with Debian installed for the new purpose of the machine), and with CentOS and Debian installation CDs. The CentOS and Debian installation CDs got a little further but still crashed. Memtest86+ however reported all the RAM as good.

I then tried reducing the RAM to 256M which worked perfectly. It seemed that increasing RAM above 384M would cause the problem.

The next thing I tried was the mem= kernel option. As a first experiment I tried mem=400m which worked perfectly. I then tried mem=759m which allowed the boot process to proceed a little further but it still crashed. The system supposedly has 760M available (8M used for video). Further testing revealed that mem=750m seemed to work well. Being cautious (and not short of RAM) I have configured the machine to use mem=740m, and it is now working well.

Oracle Unbreakable Linux

Matt Bottrell writes about the Oracle Linux offerings presented at LCA 2008 [1]

The one thing that Oracle does which I really object to is the “unbreakable” part of their advertising. They have pictures of penguins in armour and the only reasonable assumption is that their system is more secure in some way. As far as I am aware they offer no security features other than those which are available in Red Hat Enterprise Linux, CentOS, and Fedora. The unbreakable claims were also made before Oracle even had their own Linux distribution, which gave them even less reason for the claims.

If someone is going to be given credit for making Linux unbreakable then the contributors list for the SE Linux project [2] is one possible starting point. Another possibility is that credit could be given to Red Hat for introducing so many security features to the mainstream Linux users before any other distribution.

In terms of improving the security of databases it’s probably best to give credit to Kaigai Kohei and the PostgreSQL team for Security Enhanced PostgreSQL [3]. I believe that NEC also deserves some credit for sponsoring Kaigai’s work, I am not sure whether NEC directly sponsored his recent work on SE-PostgreSQL but they certainly sponsored his past work (and are credited on the NSA web site for this).

Oracle’s Linux distribution is based on CentOS and/or Red Hat Enterprise Linux (RHEL). The situation with RHEL is that the source is freely available to everyone but binaries are only available to people who pay for support. CentOS is a free recompile of RHEL and a good choice of a distribution if you want a server with long-term support and don’t want to pay Red Hat (I run many servers on CentOS).

While Matt gets most things right in his post there is one statement that I believe to be wrong, he writes “One of the craziest statements I heard during the talk was that Oracle will only support their products running under a VM if it’s within Oracle VM“. My knowledge of Xen causes me to have great concerns about reliability. My conversations with MySQL people about how intensive database servers are and how they can reveal bugs in the OS and hardware are backed up by my own experience in benchmarking systems. Therefore I think it’s quite reasonable to decline to support software running under someone else’s Xen build in the same way as you might refuse to support software running under a different kernel version (for exactly the same reasons).

Matt however goes on to make some very reasonable requests of Oracle. The demand for native packages of Oracle is significant, I can’t imagine official Debian package support appearing in the near future, but RPM support for RHEL etc would make things easier for everyone (including Oracle).

A better installation process for Oracle would also be a good thing. My observation is that most Oracle installations are not used for intensive work and use database features that are a sub-set of what MySQL offers. I’ve seen a few Oracle installations which have no more than three tables! The installation and management of Oracle is a significant cost factor. For example I used to work for a company that employed a full-time Oracle DBA for a database with only a few tables and very small amounts of access (he spent most of his time watching videos of fights and car crashes that he downloaded from the net). Adding one extra salary for a database is a significant expense (although the huge Oracle license fees may make it seem insignificant).

Laptop vs Book Weight

Matt Bottrell wrote an interesting and informative post about laptops for school kids [1]. His conclusion is that based on technical features the OLPC machine is best suited for primary school children and one of the ASUS EeePC, the Intel Classmate, and the Everex Cloudbook would be best suited for high-school students.

The Asus EeePC [2] is a good option, it runs a variant of Debian and the Debian Eeepc Team are active in getting full Debian support for it [3].

The Intel Classmate [4] has a choice of Windows XP, Mandriva, and Metasys Classmate. The web page says that it’s designed “for primary students (ages 5-14)“, so I think that Matt made a mistake in listing this as a possibility for high-schools, of course when running Mandriva it could have software installed for any age group but the hardware design may be better suited to younger children.

The Everex Cloudbook [5] runs the GOS Rocket [6] OS which seems to be an Ubuntu variant with an Enlightenment based GUI and a configuration aimed at using Google services (blogger, gmail, etc). Configuring Ubuntu to suit your needs is easy enough (it’s based on Debian). Note that Matt did not mention where one might purchase a Cloudbook in Australia and I don’t recall seeing one on any of my many window-shopping expeditions to Australian consumer electronics stores, while the EeePC is widely available (except when sold out). But I’m sure that if the government wanted to place an order for a couple of million units then Everex would ramp up production quickly enough.

Matt made one statement that I strongly disagree with, he wrote “A traditional notebook is far too heavy for high-school kids to lug around“.

To test this theory I searched for some high-school text books and a set of scales. A year 11 Maths A text book from ~1988 weighed 600g and the pair of year 12 Maths A and Maths B texts weighed 1.6Kg. When I was at high-school the day was divided into seven “periods”, some classes took two periods so four different classes which required text books (or other books) was typical. Carrying 3Kg of books to school would not be uncommon for year 12 students. The Lenovo T series (advertised as “premier performance” and the model I personally prefer) is listed as having a starting weight of 2.1Kg (which presumably doesn’t include the power pack). My Thinkpad T series (from about 2004) weighs about 2.4Kg according to my kitchen scales and has a battery weighing just over 400g.

My practice for a long time was to own a spare power pack for my Thinkpad so that I could leave it at work (saving 400g when travelling to and from work). I have also had the practice of buying a spare battery when I buy a Thinkpad (you need a spare battery for a long trip). So if I had really wanted to save weight I could have left a battery at work and reduced by travel weight by another 400g (with the cost being that I couldn’t use it when on a train or bus).

A spare power pack is not overly expensive. In the usual case students would only need a battery when at school (it’s a little known fact that Thinkpads work perfectly without a battery plugged in). So if a student had a power pack at home as well as one at school and if they left their battery at school and they owned one of the latest Thinkpad T series (listed with a starting weight of 2.1Kg) then their travel weight might be about 1.7Kg. If the majority of school texts could be stored on their laptop then the result of using a Thinkpad T series would be a significant weight reduction! If the students were using a Thinkpad X series (more expensive so maybe not a good option) then the list weight is 1.57Kg and the travel weight might be as low as 1.3Kg (at a rough estimate).

The EeePC offers significant benefits for school use, it is light, cheap (children tend to break or lose things more frequently than adults so you should budget for buying two of anything that they use), and having no hard drive (flash storage) it should cope well with being dropped. The screen on the EeePC is unreasonably small buy Asus could release a new model with a bigger screen (they may do this in the future anyway or a government contract could encourage them to do it sooner).

I agree that the EeePC or the Everex Cloudbook is probably the best option for high-school students, but I can’t agree with any claim about a traditional laptop being too heavy, the only reason for excluding a traditional laptop is that those new ultra-lights are better.

Another reason that might be cited for not using laptops is the cost. While prices of $1000 or more for a traditional laptop are rather expensive, the $500 for an EeePC is not that expensive – and the government could surely negotiate a better deal, I would be surprised if they couldn’t get the price down to $350 by some bargaining and by removing the middle-man. A careful child could use the same laptop for the entire duration of high-school and their parents would incur less expense than they currently would spend on text books.

As for the current lack of electronic text books. Currently when the education department selects a book it’s a license to print money for the author and publisher. All that the education department has to do is to declare that they will do a deal with the first company to release their books under a creative commons license. The idea would be that an author (or publishing company) would get paid a fixed sum of money for a CC release of a text book which would then be available for use by anyone anywhere in the world. World-wide free distribution would be no loss to the author (each country tends to have unique books anyway) but would be a good act of charity from our government to developing countries.

Once books were available under a creative commons license (without the “no modifications” clause) they could be freely improved by anyone. Improving text books for younger students could be a good school project.

Update:

Thanks to Steve Walsh for pointing out that the Classmate can run Linux. It’s a pity that he didn’t link to my post so that his readers could see what he was referring to. I take it as a good sign of the quality of my posts that such small errors get pointed out.

etbe – Russell Coker

Archives

Categories

Redirecting Output from a Running Process

Future Versions of Windows

Chilled Memory Attacks

EeePC for a Teenager

Giving Away Hardware

Watching While Waiting

Service Videos

Athlon Memory Problems

Oracle Unbreakable Linux

Laptop vs Book Weight

Archives

Email and RSS

Archives

Categories

Tags

Archives

Email and RSS