3

Xen and Security

I have previously posted about the difference between using a chroot and using SE Linux [1].

Theo de Raadt claims that virtualisation does not provide security benefits [2] based on the idea that the Xen hypervisor may have security related bugs.

From my understanding of Xen a successful exploit of a Xen system with a Dom0 that is strictly used for running the DomU’s would usually start by gaining local root on one of the DomU instances. From there it is possible to launch an attack on the Xen Dom0. One example of this is the recent Xen exploit (CVE-2007-4993) [3] where hostile data in a grub.conf in a DomU could be used to execute privileged commands in the Dom0. Another possibility would be to gain root access to a DomU and then exploit a bug in the Xen API to take over the hypervisor (I am not aware of an example of this being implemented). A final possibility is available when using QEMU code to provide virtual hardware where an attacker could exploit QEMU bugs, an example of this is CVE-2007-0998 where a local user in a guest VM could read arbitrary files in the host [4] – it’s not clear from the advisory what level of access is required to exploit it (DomU-user, DomU-root, or remote VNC access). VNC is different from other virtual hardware in that the sys-admin of the virtual machine (who might be untrusted) needs to access it. Virtual block devices etc are only accessed by the DomU and Xen manages the back-end.

The best reference in regard to these issues seems to be Tavis Ormandy’s paper about hostile virtualised environments [5]. Tavis found some vulnerabilities in the QEMU hardware emulation, and as QEMU code is used for a fully virtualised Xen installation it seems likely that Xen has some vulnerabilities in this regard. I think that it is generally recommended that for best security you don’t run fully virtualised systems.

The remote-console type management tools are another potential avenue of attack for virtualised servers in the case where multiple users run virtual machines on the same host (hardware). I don’t think that this is an inherent weakness of virtualisation systems. When security is most important you have one sys-admin running all virtual machines – which incidentally seems to be the case for most implementations of Xen at the moment (although for management not security reasons). In ISP hosting type environments I doubt that a remote console system based on managing Xen DomU’s is going to be inherently any less secure than a typical remote console system for managing multiple discrete computers or blades.

I have just scanned the Xen hypervisor source, the file include/asm-x86/hypercall.h has 18 entries for AMD64 and 17 for i386 while include/xen/hypercall.h has 18 entries. So it seems that there are 35 or 36 entry points to call the hypervisor, compared to 296 system calls on the i386 version of Linux (which includes the sys_socketcall system call which expands to many system calls). This seems to be one clear indication that the Linux kernel is inherently more complex (and therefore likely to have a higher incidence of security flaws) than the Xen hypervisor.

Theo’s main claim seems to be that Xen is written by people who aren’t OpenBSD developers and who therefore aren’t able to write secure code. While I don’t agree with his strong position I have to note the fact that OpenBSD seems to have a better security history than any other multi-user kernel for which data is available. But consider a system running Xen with Linux in Dom0 and multiple para-virtualised OpenBSD DomU’s. If the Linux Dom0 has OpenSSH as the only service being run then the risk of compromise would be from OpenSSH, IP based bugs in the Linux kernel (either through the IP address used for SSH connections or for routing/bridging to the OpenBSD instances), and from someone who has cracked root on one of the OpenBSD instances and is attacking the hypervisor directly.

Given that OpenSSH comes from the OpenBSD project it seems that the above scenario would only add the additional risk of an IP based Linux kernel attack. While a root compromise of an OpenBSD instance (consider that a typical OpenBSD system will run a lot of software that doesn’t come from the OpenBSD project – much of which won’t have a great security history) would only lose that instance unless the attacker can also exploit the hypervisor (which would be a much more difficult task than merely cracking some random daemon running as root that the sys-admin is forced to install). Is the benefit of having only one instance of OpenBSD cracked due to a bad daemon enough to outweigh the risk of a Linux IP stack?

I’m sure that the OpenBSD people would consider that a better option would be OpenBSD in the Dom0 and in the DomU. In which case the risk of damage from a root compromise due to one badly written daemon that didn’t come from OpenBSD is limited to a single DomU unless the attacker also compromises the hypervisor. When working as a sys-admin I have been forced by management to install some daemons as root which were great risks to the security of the system, if I had the ability to install them in separate DomU’s I would have been able to significantly improve the security of the system.

Another flaw in Theo’s position is that he seems to consider running a virtual machine as the replacement of multiple machines – which would be an obvious decrease in security. However in many cases the situation is that no more or less hardware is purchased, it is just used differently. If instead of a single server running several complex applications you have a Xen server running multiple DomU’s which each have a single application then things become much simpler and more secure. Upgrades can be performed on one DomU at a time which decreases the scope of failure (which often means that you only need one business unit to sign-off on the upgrade) and upgrades can be performed on an LVM snapshot (and rolled back with ease if they don’t succeed). A major problem with computer security is when managers fear problems caused by upgrades and prohibit their staff from applying security fixes. This combined with the fact that on a multiple DomU installation one application can be compromised without immediate loss of the others (which run in different DomU’s and require further effort by the attacker for a Xen compromise) provides a significant security benefit.

It would be nice for security if every application could run on separate hardware, but even with blades this is not economically viable – not even for the biggest companies.

I have converted several installations from a single overloaded and badly managed server to a Xen installation with multiple DomU’s. In all cases the DomU’s were easier to upgrade (and were upgraded more often) and the different applications and users were more isolated.

Finally there is the possibility of using virtualisation to monitor the integrity of the system, Bill Broadley’s presentation from the 2007 IT Security Symposium [6] provides some interesting ideas about what can be done. It seems that having a single OpenBSD DomU running under a hypervisor (maybe Xen audited by the OpenBSD people) with an OpenBSD Dom0 would offer some significant benefits over a single OpenBSD instance.

6

0wned a DVD Player

DVD player saying root

Above is a picture of a DVD player I saw on sale in Dick Smith Electronics [1] (a chain store that used to sell mostly electronics hobbyist gear but now mostly sells consumer electronics gear). I asked one of the staff why it said “root”, tests revealed that the DVD caused any player to display “root” once it was inserted. The DVD in question was from the $2 box (the DVDs that didn’t sell well at other stores) and for some reason had the string “root” in it’s title (or some other field that gets picked up by the player).

I wonder if an ex-employee of a movie company is laughing about millions of DVD players all around the world saying “root”.

Update: I’ve been told by private mail that “It means it’s displaying the “root” menu. (As opposed to the title menu or any submenu’s)” and “most just display ‘menu’ or similar“. So apparently every time my DVD player says “menu” the new ones from Dick Smith will say “root” (I have yet to test this theory).

1

Banking with an Infected Computer

Bruce Schneier summarised a series of articles about banking security [1]. He mentioned the fact that banks don’t seem to care about small losses and would rather just deal with the problem (presumably by increasing their fees to account for losses).

There are some other interesting bits in the article, for example banks are planning a strategy of securing transactions with an infected computer [2]! Now there are some possible solutions to this, for example if the bank issued a hardware device that allowed the customer to enter the account number, amount to transfer, destination account, and PIN number and then produced a cryptographically secure hash (based in part on a rolling code) that the user could type in.

The only way that you are going to do anything securely with an infected host is if everything is offloaded into an external device. In which case why not just do the Internet banking on the external device? It’s not difficult to make a hardware device that is small enough to carry everywhere, has a display, an input device, net access, and which is reasonably difficult to crack externally. Consider for example a typical mobile phone which has more RAM, CPU power, and storage than a low-end machine that was used for web browsing in 1996. Mobile phones have a good history of not being hacked remotely and are difficult for the owner to “unlock”. A locked-down mobile phone would be a good platform for Internet banking, it has wireless net access in most places (and with a Java application on the phone it could do banking by encrypted SMS). Being locked down to prevent the user from reconfiguring the software (or installing new software) will solve most of the security problems that plague Windows.

If when signing up for a new phone contract I was offered the possibility of getting a phone with secure banking software installed for a small extra fee then I would be very interested. Of course we would want some external auditing of the software development to make sure that it’s not like some of the stupid ideas that banks have implemented. Here is a classic example of banking stupidity [3]. They display a selected word and picture for the user when they login to try and prevent phishing (of course a proxy or a key-logger on the local machine will defeat that). They also ask for an extra password (of the simple challenge phrase variety) if you use a different IP address, of course as the typical broadband user doesn’t know when their IP address changes they wouldn’t know if their data was being proxied and dial-up users will enter it every time. A google search for “internet banking” picture password turns up a bunch of banks that implement such ideas.

3

My SE Linux Etch Repository

deb http://www.coker.com.au etch selinux

The above sources.list line has all the i386 packages needed for running SE Linux with strict policy on Etch as well as a couple of packages that are not strictly needed but which are really convenient (to solve the executable stack issue).

gpg --keyserver hkp://subkeys.pgp.net --recv-key F5C75256
gpg -a --export F5C75256 | apt-key add –

To use it without warnings you need to download and install my GPG key, the above two commands do this. You will of course have to verify my key in some way to make sure that it has not been replaced in a MITM attack.

The only thing missing is a change to /etc/init.d/udev to have a new script called /sbin/start_udev used to replace the make_extra_nodes function (so that the make_extra_nodes functionality can run in a different context). Of course a hostile init script could always exploit this to take over the more privileged domain, but I believe that running the init scripts in a confined domain does produce some minor benefits against minor bugs (as opposed to having the init scripts entirely owned).

I back-ported all the SE Linux libraries from unstable because the version in Etch doesn’t support removing roles from a user definition by the “semanage user -m” command (you can grant a user extra roles but not remove any roles). Trying to determine where in the libraries this bug occurred was too difficult.

Does anyone know of a good document on how to create repositories with apt-ftparchive? My current attempts are gross hacks but I’ve gone live anyway as the package data is good and the apt configuration basically works.

12

Insider Threats and Small Storage Devices

Danny Angus writes about the potential threat posed by small storage devices with large capacity [1]. His post was prompted by a BBC article about Hitachi’s plans for new hard drives [2], they are aiming for 4TB of data on a single drive by 2011 and a 1TB laptop drive. One thing I noticed about the article is that they made the false claim that current drives are limited to 1TB, the storage capacity is determined by the total surface area which is proportional to the square of the radius and the height of the drive (AFAIK there are no practical limits to the number of platters apart from the height of the drive). So if a 5.25 inch hard drive was to be manufactured with today’s technology it should get a capacity equivalent to at least three times the capacity of the larger 3.5 inch drive.

The reason that 5.25 inch drives are not manufactured is that for best performance you want multiple spindles so that multiple operations can be performed concurrently. Using 3.5 inch drives in servers allows the use of more disks for the same amount of space in the rack and the same amount of power. The latest trend is towards 2.5 inch (Small Form Factor AKA SFF) disks for servers to allow more drives for better performance. With 3.5 inch disks a 1U system was limited to 3 disks and a 2U system was often limited to 4 or 5 disks. But with 2.5 inch drives a 2U server can have 10 drives or more. I know of one hardware vendor that plans to entirely cease using 3.5 inch drives and claims that 2.5 inch disks will give better performance, capacity, and power use!

In regard to Danny’s claim (which is entirely correct) about the threat posed by insiders. I don’t believe that a laptop with 1TB of capacity is the threat. In a server room people notice where laptops get connected and there are often strictly enforced policies about connecting machines that don’t belong to the company. I believe that the greatest threat is posed by USB flash devices. For example let’s consider a database with customer name (~20B), birth-date (10B), address (~80B), phone number (~12B), card type (1B), card number (16B), card expiry (5B), and card CVV code (3B). That’s ~155 bytes per record in CSV or TSV format. If you have data for a million customers that’s 155M uncompressed and probably about 50M when compressed with gzip or WinZip (depending on which platform is being ripped). No-one even sells a USB flash device that is smaller than 50M, I recently bought a 2G flash device that was physically very small and cheap (it was in the bargain bin).

The next issue is, what data might be worth stealing that is large enough to not fit on a USB device? I guess that if you want to copy entire network file shares from a corporation then you would need more than the 16G that seems to be the maximum capacity of a USB device at the moment. Another theoretical possibility would be to copy the entire mail spool of a medium to large ISP. For the case of a corporate file server you could probably get the data at reasonable speed, 1TB of data would take 10,000 seconds or 2.8 hours to transfer at gigabit Ethernet speeds (if you max out a GigE link – it could be as much as five times that if the network is congested or if the server is slow). It’s doable, but it would be a rather tense three or more hours waiting by an illegally connected laptop. For the mail server of a large ISP there is often no chance of getting anywhere near line speed, it’s lots of small reads and seek performance is the bottleneck, such servers are usually running close to capacity (and trying to copy data fast would hurt performance and draw unwanted attention).

Another possibility might be to copy the storage of an Intranet search device. If a company has a Google appliance or similar device indexing much of their secret data then copying the indexes would be very useful. It would allow offline searches of the corporate data to prepare a list of files to retrieve later.

It would probably be more useful to get online access to the data from a remote site. I expect that an unethical person could sell remote access to someone who is out of range of extradition. All that would be required would be to intentionally leave a flaw in the security of the system. In most large corporations this could be done in a way that is impossible to prove. For example if management decrees that the Internet servers run some software that is known to be of low quality then a hostile insider could make configuration changes to increase the risk – it would look like an innocent mistake if the problem was ever discovered (the blame would entirely go to the buggy software and the person who recommended it).

A large part of the solution to this problem is to hire good employees. The common checks performed grudgingly by financial companies are grossly inadequate for this. Checking whether a potential employee has a criminal record does not prevent hiring criminals, it merely prevents hiring unsuccessful criminals and people who have not yet been tempted enough! The best way to assess whether HR people are being smart about this is to ask them for an estimate of how many criminals are employed by the company. If you have a company that’s not incredibly small then it’s inevitable that some criminals will be employed. Anyone who thinks that it is possible to avoid hiring criminals simply isn’t thinking about the issues. I may write more about this issue in a future post.

Another significant part of the solution to the problem is to grant minimum privileges to access data. Everyone should only be granted access to data that they need for their work so that the only people who can really compromise the company are senior managers and sys-admins, and for best security different departments or groups should have different sys-admin teams and separate server rooms. Of course this does increase the cost of doing business, and probably most managers would rather have it be cheap than secure.

AUUG 2007

Today was the final day of the AUUG 2007 conference [1].

Yesterday I gave a talk about SE Linux for about an hour (not sure exactly as I forgot to make an MP3). AUUG is well known for having conferences with very technical delegates and I wasn’t expecting an easy audience. At the start of my talk I asked for a show of hands as to who has used SE Linux before, about 1/3 of the delegates raised their hand. Someone requested that I poll the audience as to who had used SE Linux involuntarily, it wasn’t what I had planned to ask but it’s best to get these things out in the open so I asked the question. More people raised their hand as being unwilling users of SE Linux than those who had firstly admitted to using it!

A theme of the AUUG conference was quality, and I had planned to cover some of the ways that SE Linux improves the quality of code by making certain classes of bugs show up (EG file handle leaks) and by allowing the developers and sys-admins to know exactly what programs are doing. But I ended up explaining why you want to use SE Linux, the concepts of policy analysis tools (as compared to the absence of such tools for Unix permissions), the benefits of MAC and why SE Linux is worth using.

I believe that the talk did some good and conversations with delegates afterwards revealed that some of them had done some positive things with SE Linux.

Today I wore a T-shirt advertising the root password for my new SE Linux Play Machine [2] which will be online shortly (hopefully tomorrow) which got some interest (AFAIK I’m the first person to wear a root password on a T-shirt). When I have my play machine online I plan to wear the shirt whenever I visit an electronics store or any other location where geeks are likely to congregate. ;)

After the conference finished about 1/3 of the delegates went to Ginza Teppanyaki [3] for dinner. Some of the guys wanted to photograph me wearing my shirt.

Finally, the conference went pretty well, the delegates and speakers all seemed to enjoy themselves and learn some useful things. Congratulations to the AUUG conference organisers!

Execmem and SE Linux

Eddy writes about problems getting the game oolite to run under SE Linux [1].

Strangely after I fixed the shared object issue with libffcall1 (as described in my previous post [2]) it appeared to work for me.

Eddy asked how to allow one application to create write and executable memory regions without allowing such access for all programs. This can be done, in the targeted policy the type unconfined_execmem_exec_t triggers a transtion to the domain unconfined_execmem_t which permits the execstack and execmem operations. For example if the program /usr/bin/foo needs such access then the command chcon -t unconfined_execmem_exec_t /usr/bin/foo will change the type and the next time the program is launched from an unconfined session (a user login session, cron job, or daemon which is not constrained) then the change will apply.

1

Lintian and Executable Stacks

Debian has a program called Lintian that is used to search for common bugs in Debian packages. When it encounters a package with a shared object that requests an executable stack (as described in my previous post about executable stacks and shared objects [1]) it gives a warning such as the following:
W: liblzo1: shlib-with-executable-stack usr/lib/liblzo.so.1.0.0

Lintian is run automatically on Debian servers and has a web site at http://lintian.debian.org/. You can search the site for all packages which have such executable stacks [2].

Of all the packages listed I have only two installed on my system, liblzo1 and libsmpeg0, both of which I had already discovered and built new versions with the correct stack settings (I’ll publish an APT repository shortly). For the rest I am not sure whether they are really bugs. The ones that concern me are xserver-xorg-video-nsc (we don’t want a stack smashing attack on something as important as an X server) and the C libraries libuclibc0 and dietlibc which may cause many programs to run with an executable stack.

The above URL shows that libffcall1 [4] has this problem (as Eddy discovered [5]). Eddy filed Debian bug report 445895 [6] about this problem (I have just updated the bug report with a patch to make it work on i386).

Linda (an alternative to Lintian) does not currently warn about this. I have filed Debian bug report 445826 about this [3].

6

How SE Linux Prevents Local Root Exploits

In a comment on my previous post about SE Linux and worms/trojans [1] a user enquired about which methods of gaining local root are prevented by SE Linux.

A local exploit is one that can not be run remotely. An attack via TCP or UDP is generally considered a remote exploit – even though in some cases you might configure a daemon to only bind to localhost (in which case the TCP or UDP attack would only work locally). When compromising a machine it’s not uncommon for a local exploit to be used after a remote exploit or social engineering has been used to gain non-root privileges.

The two most common types of local root exploit seem to be those which attack the kernel and those which attack a SETUID process. For a non-SE Linux system it usually doesn’t matter much how the local exploit is run. But a SE Linux system in a default configuration will be running the Targeted policy that has almost no restrictions on a user shell session. So an attacker who wants to escalate their privileges from local user to local root on a typical SE Linux system has a significant benefit in starting from a user account instead of starting from a web server or other system process.

In the SE Linux model access is granted to a domain, and the domain which is used for a process is determined by the policy based on the domain of the parent process and the labelling of the executable. Some domains are not permitted to transition to any other domains, such as the domain dhcpd_t (used for a DHCP server). Other domains are only permitted to transition to a small set of domains, for example the domain httpd_t (used for a web server) can only transition to a small number of domains none of which has any significant privileges.

On a machine running without SE Linux a compromise of a DHCP server is game-over as the server runs as root. A compromise of a daemon such as Apache on a machine without SE Linux gives unrestricted access to run applications on the systems – if a SETUID-root program has a security flaw then you lose. The same bug in a SETUID program on a machine running SE Linux is not fatal because SE Linux will prevent the program from doing anything that it’s parent could not do – even if an attacker made Apache run a buggy SETUID program the broken program in question could do nothing other than what Apache is normally permitted to do.

A security flaw in a SETUID-root program on a SE Linux system can still be exploited by a local user (someone who has logged in) when running the Targeted policy. When running the Strict or MLS policies many such vulnerabilities will not be exploitable by local users (for example exploiting PING would only permit network access).

As a rule of thumb you should consider that a kernel security flaw will make it possible to bypass all other security features. However there are some situations where SE Linux can prevent local exploits. One example is a bug discovered in July 2006 which allowed the creation of SETUID files under /proc [2], the targeted policy of SE Linux prevented this from working. Another example is CVE-2003-0127 [3] (a kernel security flaw that was exploited by triggering a module load and then exploiting a race condition in the kernel module load process), the commonly used exploit for this did not work on a SE Linux system because the user process was not permitted the socket access used to trigger the module load (it is believed that an attacker could have written a version of the exploit to work on a SE Linux system – but AFAIK no-one ever did so).

8

Can SE Linux Stop a Linux Storm

Bruce Schneier has just written about the Storm Worm [1] which has apparently been quietly 0wning some Windows machines for most of this year (see the Wikipedia page for more information [2]).

I have just been asked whether SE Linux would stop such a worm from the Linux environment. SE Linux does prevent many possible methods of getting local root. If a user who does not have the root password (or is not going to enter it from a user session) has their account taken over by a hostile party then the attacker is not going to get local root (unless there is a kernel vulnerability). Without local root access the activities of the attacker can be seen by a user logged in on another account – processes will be seen by all user sessions if using the SE Linux targeted policy and files and processes can be seen by the sys-admin.

If while a user account is 0wned the user runs “su –” (or an equivalent command) then in theory at least the attacker can sniff this and gain local root access (whether enough users do this to make attackers feel that it’s worth their effort to write the code in question is something I couldn’t even guess about). If the user is clueless then the attacker could immediately display a dialog with some message that sound urgent and demand the root password – some users would give it. If the user is even moderately smart the attacker could fake the GUI dialogues for installing updated packages (which have been in Red Hat distributions for ages and have appeared in Debian more recently) and tell the user that they need to enter the root password to install an important security update (oh the irony).

In conclusion I think that if a user is ill-educated enough to want to run a program that was sent to them in email by a random person then I expect that the program would have a good chance of coercing them into giving it local root access if the user in question had the capability of doing so.

Even if a Linux trojan did not have local root access then it could still do a lot of damage. Any server operations that don’t require ports <1024 (which means most things other than running a web, DNS, or mail server) can still be performed and client access will always work (including sending email). The trojan would have access to all of the user’s data (which for a corporate desktop machine usually means a huge network share of secret documents).

If a trojan only attempts to perform actions that SE Linux permits (running programs from the user’s home directory, accessing servers for DNS, HTTP, IRC, SMTP, and other protocols – a reasonable set of options for a trojan) then the default configuration of SE Linux (targeted policy) won’t stop it or even log anything. This is not a problem with SE Linux, just a direct result of the fact that in every situation a trojan can perform all operations that the user can perform – and if the trojan only wants to receive commands via web and IRC servers and send spam via the user’s regular mail server then it will be a small sub-set of the permitted operations for the user!

If however the trojan tries more aggressive methods then SE Linux will log some AVC messages about access being denied. If the sys-admin has good procedures for analysing log files they will notice such things, understand what they mean, and be able to contain the damage. Also there have been at least two cases where SE Linux prevented local root exploits.

Finally, in answer to the original question: SE Linux will stop some of the more aggressive methods that trojans might use. But there are still plenty of things that a trojan could do to cause harm which won’t be stopped or audited by SE Linux policy. When Linux gets more market share among users with a small amount of skill and no competent person to do sys-admin work for them we will see some Linux trojans and more Linux worms. It will be interesting to see what methods the trojan authors decide to use.