etbe - Russell Coker

Better Social Networking

When advogato.org was still cool I signed up to it. It was an interesting research project in skill metrics (determining the rating of people’s coding skills by the votes of others and weighting the votes by the rating of each person), and it was nice to be rated Master soon after I joined. I still use it on occasion for the blog syndication feature (when I find a good blog on Advogato I add it to my Planet installation).

When orkut.com was really cool (when every time I had dinner with a group of people someone would ask if they could be an “orkut friend”) I signed up to it. It was interesting for a while but then most people got bored with it.

Now there is Facebook and MySpace for social networking for social purposes and LinkedIn.com for business related social networking. I periodically get invited to join those services but have not been interested in any other than LinkedIn. I can’t join LinkedIn because their mail server is regarded as a SPAM source by my mail server but their web server refuses to provide any information on why this is (the rejection was apparently long enough ago that it’s rolled off my logs).

The problem with all these services is that I am expected to sign up with each of them and go to a moderate amount of effort in writing up all the data in the various web pages. Writing it is a pain, keeping it up to date is more pain, and dealing with spam in “scrap-book” entries in Orkut is still an annoyance which I don’t want to multiply by four!

So far the only step I’ve seen towards addressing this issue is the XFN – XHTML Friends Network [1] project. But that seems to be of fairly limited scope (just referring to the friendship status of people in a <a href link).

I believe that the requirements for social networking are:

Personal data about each person, some of which may only be available to friends or friends of friends.
The user owns their own data, has full control over where it’s sent and the ability to request people who receive it to keep some parts of it secret.
Ability to send email to friends of friends (determined by the wishes of each friend and FOAF).
Ability to get a list of friends of friends.
Incorporation of a standard format for CVs (for business social networking).

I think that the only way to go is to have a standard XML format for storing all personal data (including financial, career, and CV data) that can be used on any web site. Then someone who wants to be involved in social networking could create an XML file for a static web server (or multiple files with different content and password protected access), or they could have a server-side script generate an XML file on the fly with a set of data that is appropriate for the reader. The resulting social network would be entirely distributed and anyone could write software to join in. This covers item 1 and part of item 2.

For sending email to friends of friends it would be good to avoid spam as much as possible. One way of doing would be requesting that friends publish a forwarding address on their own mail server in a manner similar to SRS [2]. SRS include the ability for such addresses to expire after a certain period of time (which would be convenient for this). In fact publishing SRS versions of friends email addresses would be a good option if you already use SPF [3] and SRS in your mail server. This covers item 3.

The XML format could include information on how far the recipient could transfer it. For example if my server sent an XML file to a recruiting agency with my CV it could state that they could distribute it without restriction (so that they can give it to hiring managers) with the possibility of some fields being restricted (EG not tell the hiring manager what I used to get paid). For my mobile phone number I could send it to my friends with a request that they not send it on. This covers part of item 2.

The URL for the friends file would of course be in the main XML file, and therefore you could have different friends lists published from different versions of your profile (EG the profile you send to recruiting agencies wouldn’t include drinking buddies etc). This completes the coverage of item 2.

Then to have a friends list you have a single XML file on a web server that has the public parts of the XML files from all your friends. This means that getting a list of friends of friends would involve getting a single XML file for each friend (if you have 100 friends and each friend has 50 unique friends on average then you do 100 HTTP operations instead of 5,000). Minimising the number of web transfer operations is essential for performance and for reliability in the face of unreliable web servers (there is no chance of having 5,000 random web servers for individuals all up and accessible at the same time). This covers item 4.

Item 5 is merely a nice thing to have which allows more easily replacing some of the recruiting infrastructure. As any such XML format will have several sections for arbitrary plain text (or maybe HTML) for describing various things the CV could of course be in HTML, but it would be good to have the data in XML.

I posted this in the “blog” category because blogs are the only commonly used systems where end users do anything related to XML files (the RSS and ATOM feeds are XML). A blog server could easily be extended to do these social networking systems.

As with a blog users could run their own social networking server (publishing their XML files) or they could use a service that is similar in concept to blogger which does it all for them (for the less technical users). Then an analogy to Planet, Technorati, etc in the blog space would be public aggregation services that compare people based on the number of friends they have etc, and attempts to map paths between people based on friends.

This could also include GPG [4] data such that signing someone’s GPG key would cause your server to automatically list them in some friend category. The XML format should also have a field for a GPG signature (one option would be to use a GPG sub-key to sign the files and have the sub-key owned by the server).

I don’t have any serious time to spend on implementing this at the moment. But if someone else starts coding such a project then I would certainly help test it, debug it, and contribute towards the XML design.

The Price of Food

If you live in a hotel for an extended period of time (which can provide significant career benefits – click on this link for details [1]) the issue of food price and availability is going to concern you.

If you are in a decent hotel you will have a fridge in your room that you can use for your own food. A recent trend downwards in hotel quality has been to use fridges that are stocked with over-priced drinks that have sensors and automatically bill you if you move any of the drinks. A good hotel will have a fridge that either has space for your own food/drink or which allows you to temporarily move their stuff out. If you are staying in a hotel for any period of time and the hotel is not run by robots then you should have the option to negotiate the removal of all the over-priced drinks to provide you space for your own food.

If you have such fridge space then you have good options for making sandwiches – which are cheap and healthy.

In UK hotels (which incidentally tend to not have a fridge in the room if they are affordable) the standard practice is to have breakfast included as part of the hotel fee. If you are flexible about your eating then you can eat a large breakfast and have a minimal lunch to reduce expenses.

Finally you have to consider how much you earn as an hourly rate (after tax) and compare it to the cost of food. For example if dinner at a cheap restaurant costs $10 and you earn $30 per hour after tax then you only need to save 20 minutes of your time by eating at the restaurant (as opposed to making a meal and washing the dishes) to make it economically viable.

I have previously written about the efficiency of work [2]. I think it’s reasonable to assume (in the absence of any formal studies on the topic) that when your efficiency of working decreases due to over-work your enjoyment of your leisure time is also reduced on a per-hour basis (in addition to having less leisure time). I know that some people enjoy cooking and consider it a leisure activity (my sister seems to be one of them [3]). But if cooking isn’t something you enjoy then you will probably feel that eating out is reducing the amount of “work” time and therefore increases the quality of your life and the quality of your work.

Finally for the time spent living in a hotel while searching for work (if you travel to another country without arranging employment first) the main financial factor is not how much you can save money on a per-day basis, but how quickly you can find work. The ability to accept a job offer from any region has the potential to significantly reduce the amount of time taken to find work and thus put you in a better financial position in the long-term. This benefit of living in hotels should significantly outweigh the extra expenses of eating out etc.

SE Linux in other Distributions

Recently a user has been asking about SE Linux support in MEPIS [1]. He seems to expect that as the distribution is based on Debian it should have the same SE Linux support as is in Debian.

The problem with derived distributions (which potentially applies to all variants of Debian, Fedora, and RHEL) is that the compilation options used may differ from what is required for SE Linux support.

If an application works in Debian then you can expect that it will work in all derived distributions. But SE Linux is not an application, it is a security extension to the OS which includes code in the kernel, login, cron, pam, sshd, logrotate, and others. For any one of these packages a maintainer of a derived distribution might decide to turn off features to save disk space or memory, or because they want to use features which don’t work well with them (due to functional differences or bugs). The maintainer of a derived distribution might even decide that they just don’t like a feature and disable it for that reason alone!

I believe that it is possible to use APT with multiple repositories and specify preferences for each repository. So it should be possible to use a source such as MEPIS for most packages but Debian (or my private repository of SE Linux back-ports [2]) for the packages which need SE Linux support.

That said, I am not sure why someone would want to use MEPIS with SE Linux. Currently the benefits of SE Linux are of most use for a server and MEPIS is a desktop focussed distribution. Debian works reasonably well for a desktop (it has worked well for me for most of the past 11 years), so it seems that Debian for a SE Linux desktop machine is a good choice and Debian is a better choice than MEPIS for a server.

Safe Banking by SMS?

Is it possible to secure Internet banking with SMS?

As secure tokens are too expensive ($10 or more in bulk) and considered to be too difficult to use by many (most?) customers banks have sought out other options. One option that has been implemented by the National Australia Bank and will soon be available from the Commonwealth Bank is SMS authentication of transfers.

The idea is that when you issue an online banking request you receive an SMS with a password and then have to enter that password to authenticate it. If you receive an unexpected password then you know you have been attacked. I wonder how much information is in the SMS, does it include the amount and where the money is to be transferred (in the case of a funds transfer – the operation most likely to be used by attackers)? If the full details are not included then an attacker could hijack an active session, get the user to enter the password, and then act as if the user entered the password incorrectly. The user would then request a new SMS and complete their desired transfer without realising that they just authorised a transfer to Russia…

If the full details are recorded will the user look at them? Online banking fraud often involves transferring the funds to an idiot in the same country as the victim. Then the idiot sends the money to the attacker in some other manner which is more difficult to track. I wonder whether an attacker could divert the funds transfer to one of the idiots in question and have the victim not realise that the wrong account number was used.

Another issue is that of SMS interception. Anyone who can hack the network of a phone company could steal money from any bank account in the country! For wealthy people there is also the possibility of stealing their mobile phone and making funds transfers before they report the theft. Another possibility is to register for a new phone company. Last time I changed phone companies it took about an hour for the new company to have the phone number and I don’t recall the phone company doing anything to verify that I owned the number in question. If an attacker had a credit card with the same name as the victim (names are not unique so this is not impossible or even inherently illegal) they could open a new phone service and steal the number. Someone who’s mobile phone stops working probably wouldn’t assume that it was part of a bank fraud scheme and act accordingly, in fact if they don’t use their mobile phone later it might be several days before someone contacts them in some other manner and mentions that they weren’t answering their mobile phone.

A final possibility is the situation where a mobile phone is connected to a computer. Devices that combine mobile phone and PDA functionality are becoming common. A trojan horse program that offered to do something useful when a mobile phone was connected to the PC via a USB cable might fool some users. All that would be required is a few minutes of the phone being connected if the attacker already has the password for online banking. Maybe they could even make it appear that the bank was demanding that the phone be connected to the PC – that should fool users who don’t understand how SMS authentication works.

It seems to me that SMS authentication is an improvement (it adds an external device which usually can’t be directly manipulated by the attacker) but is far from perfect security.

I previously wrote about the bad idea that you can bank with an infected computer [1]. SMS authentication is a good step towards making things more difficult for attackers (which is always a good idea) but doesn’t really secure the system. Also it costs 5 cents for each SMS, I expect that the banks will want their customers to pay for this – I would rather pay for a $10 token up-front.

[1] http://etbe.coker.com.au/2007/10/19/banking-with-an-infected-computer/

Software vs Hardware RAID

Should you use software or hardware RAID? Many people claim that Hardware RAID is needed for performance (which can be true) but then claim that it’s because of the CPU use of the RAID calculations.

Here is the data logged by the Linux kernel then the RAID-5 and RAID-6 drivers are loaded on a 1GHz Pentium-3 system:

raid5: automatically using best checksumming function: pIII_sse
pIII_sse : 2044.000 MB/sec
raid5: using function: pIII_sse (2044.000 MB/sec)
raid6: int32x1 269 MB/s
raid6: int32x2 316 MB/s
raid6: int32x4 308 MB/s
raid6: int32x8 281 MB/s
raid6: mmxx1 914 MB/s
raid6: mmxx2 1000 MB/s
raid6: sse1x1 800 MB/s
raid6: sse1x2 1046 MB/s
raid6: using algorithm sse1x2 (1046 MB/s)

There are few P3 systems that have enough IO capacity to support anywhere near 2000MB/s of disk IO and modern systems give even better CPU performance.

The fastest disks available can sustain about 80MB/s when performing contiguous disk IO (which incidentally is a fairly rare operation). So if you had ten fast disks performing contiguous IO then you might be using 800MB/s of disk IO bandwidth, but that would hardly stretch your CPU performance. It’s obvious that CPU performance of the XOR calculations for RAID-5 (and the slightly more complex calculations for RAID-6) is not a bottleneck.

Hardware RAID-5 often significantly outperforms software RAID-5 (in fact it should always outperform software RAID-5) even though in almost every case the RAID processor has significantly less CPU power than the main CPU. The benefit for hardware RAID-5 is in caching. A standard feature in a hardware RAID controller is a write-back disk cache in non-volatile RAM (RAM that has a battery backup and can typically keep it’s data for more than 24 hours without power). All RAID levels benefit from this but RAID-5 and RAID-6 gain particular benefits. In RAID-5 a small write (less than the stripe size) requires either that all the blocks other than the ones to be written are read or that the original content of the block to be written and the parity block are read – in either case writing less than a full stripe to a RAID-5 requires some reads. If the write-back cache can store the data for long enough that a second write is performed to the same stripe (EG to files being created in the same Inode block) then performance may double.

There is one situation where software RAID will give better performance (often significantly better performance), that is for low-end hardware RAID devices. I suspect that some hardware RAID vendors deliberately cripple the performance of low-end RAID devices (by using an extremely under-powered CPU among other things) to drive sales of the more expensive devices. In 2001 I benchmarked one hardware RAID controller as being able to only sustain 10MB/s for contiguous read and write operations (software RAID on lesser hardware would deliver 100MB/s or more). But for random synchronous writes the performance was great and that’s what mattered for the application in question.

Also there are reliability issues related to write-back caching. In a well designed system an update of an entire RAID-5 stripe (one block to each disk including the parity block) will first be performed to the cache and then the cache will be written back. If the power fails while the write is in progress then it will be attempted again when power is restored thus ensuring that all disks have the same data. With any RAID implementation without such a NVRAM cache a write to the entire stripe could be partially successful. This means that the parity block would not match the data! In such a situation the machine would probably work well (fsck would ensure that the filesystem state was consistent) until a disk failed. When the RAID-5 recovery procedure is used after a disk is failed it uses the parity block to re-generate the missing data, but if the parity doesn’t match then the re-generated data will be different. A disk failure may happen while the machine is online and this could potentially result in filesystem and/or database meta-data changing on a running system – this is a bad situation that most filesystems and databases will not handle well.

A further benefit of a well designed NVRAM cache is that it can be used on multiple systems. For their servers HP makes some guarantees about which replacement machines will accept the NVRAM module. So if you have a HP server running RAID-5 with an NVRAM cache then you could have the entire motherboard die, have HP support provide a replacement server, then when the replacement machine is booted with the old hard drives and NVRAM module installed the data in the write-back cache will be written! This is a significant feature for improving reliability in bad corner cases. NB I’m not saying that HP is better than all other RAID vendors in this regard, merely that I know what HP equipment will do and don’t know about the rest.

It would be good if there was a commodity standard for NVRAM on a PC motherboard. Perhaps a standard socket design that Intel could specify and that every motherboard manufacturer would eventually support. Then to implement such things on a typical PC all that would be required would be the NVRAM module, which while still being expensive would be significantly cheaper than current prices due to the increase in volume. If there was a significant quantity of PCs with such NVRAM (or which could be upgraded to it without excessive cost) then there would be an incentive for people to modify the Linux sotware RAID code to use it and thus give benefits for performance and reliability. Then it could be possible to install a NVRAM module and drives in a replacement server with Linux software RAID and have the data integrity preserved. But unless/until such things happen write-back caching that preserves the data integrity requires hardware RAID.

Another limitation of Linux software RAID is expanding RAID groups. A HP server that I work on had two disks in a RAID-1 array, one of my colleagues added an extra disk and made it a RAID-5, the hardware RAID device moved the data around as appropriate while the machine was running and the disk space was expanded without any down-time. Some similar things can be done with Linux, for example here is documentation on converting RAID-1 to RAID-5 with Linux software RAID [1]. But that conversion operation requires some down-time and is not something that’s officially supported, while converting RAID-1 to RAID-5 with HP hardware RAID is a standard supported feature.

[1] http://scott.wallace.sh/node/1521

More About Living in Hotels

In the past I have spent about 18 months living in hotels with a couple of months of breaks in between. I have previously written about it in terms of living in London hotels [1], but I have been asked for more generic advice.

Firstly the amount of possessions that you may have when living in hotels is seriously limited. For ease of travel you want to restrict yourself to one suitcase for checked luggage and one for carry-on luggage. Hotels often have short-term storage space for possessions of guests, so having a second suitcase of items that are not worth stealing (clothes and books) may be an option. But consumer electronics devices other than a single laptop computer are not an option.

I read an interesting blog post on ZenHabits.net titled Minimalist Fun: The 100 Things Challenge [2] which advocates counting and limiting the number of possessions you own. When living in hotels if I considered my books as one collection and my clothes as another (having never been interested in trendy clothes they count as utilitarian items for work or leisure not objects that I seek to own) and as my mobile phone was a tool for work and my computer gear was strictly limited to items that were needed for work (and thus “tools”) my only possessions were a digital camera and some bottles of liquor! The lack of ability to accumulate possessions may be considered as an advantage or a disadvantage depending on what your aims are.

If you are moving to another country for work there are three ways of doing it. The easiest is to be a permanent employee of a company that assigns you to work there – in which case they will probably pay to transport your stuff when you buy or rent a house. If you are a looking for new employment (either contract or permanent) in another country then you can either find the work before moving or after arriving there. Finding work before arriving in the country is difficult and generally only works for short-term contracts. So it’s most likely that you will either be looking for work immediately after arriving or after a short contract. In either case having better mobility increases your employment options – why restrict yourself to one city or region when you can choose from all jobs in an entire country or (in the case of the EU) half of a continent! The career benefits of being able to accept any job anywhere in the world at short notice are significant!

There are situations where an employer will pay hotel bills. One example was when I was working for a London based company and they assigned us to work at the other side of London. My colleagues complained and the company paid for hotel bills for everyone Sunday night to Thursday night inclusive as well as an extra hour of pay per day as compensation for the inconvenience. For me of course one hotel was as good as another so it just meant that my employer was covering 5/7 of my living expenses. Then I had a meeting with the hotel manager and pointed out that having me check out every Friday would be bad for them as the hotel was mostly empty on the weekend and suggested that they make me a deal for the other two days – I ended up paying something like one night of hotel fees per week! If I had rented an apartment I would have still been paying the full rent (which while less than 30 days hotel fees per month would have been considerably more than 4 or 5 days of hotel fees per month).

If you live in a hotel then there is always some sort of deal that can be arranged. Apart from certain busy days (such as around the Christmas and new-year time) they always want long-term guests and will be willing to reduce the price, give free dinner or drinks from the bar, etc.

The cost of living in a hotel at times such as Christmas may be as much as five times the regular rate. That is a further incentive to visit friends or relatives at Christmas. If you can’t visit your family (which may be difficult if they live on the other side of the world) then finding a friend who has a spare room might be an option.

Restorecon Equivalent for Unix Permissions

SE Linux has a utility named restorecon to set (or reset) the security context. This is useful for many reasons, corrupted filesystems, users removing files or changing the context in inappropriate ways, and for re-creating files from tar files or backup programs that don’t restore SE Linux contexts. It can also be used to report the files that have different contexts to that which would be set by restorecon to verify the contexts of files.

Restorecon determines the context from two sources of data, one is the policy that came with the system (including any policy modules from other sources which were loaded) and the other is the local file contexts that were created by semanage.

It’s a pity that there doesn’t seem to be an equivalent program for Unix permissions. rpm has a -V option to verify the files from a package and dpkg doesn’t seem to have an option to perform a similar operation (/var/lib/dpkg/info/* doesn’t seem to have the necessary data). But even on an RPM based system this isn’t possible because there is no way to add local files into the list.

I would like to be able to specify that an RPM system should have root:root as the owner and permission mode 0755 for all files matching /usr/local/bin/* and use a single command to check the RPM database as well as this extra data for the permissions of all files.

Does anyone know of any work in this area?

I’m going to file Debian and Fedora bug reports about this, but I would appreciate any comments first.

Update:

Here is an example of how this feature works in rpm:
# rpm -Vv nash
…….. /sbin/nash
…….. d /usr/share/man/man8/nash.8.gz
# chmod 700 /sbin/nash
# rpm -Vv nash
.M…… /sbin/nash
…….. d /usr/share/man/man8/nash.8.gz

The “M” character indicates that the permission mode of the file does not match the RPM. There is no way to automatically correct it (AFAIK) but at least we know that something changed. With Debian AFAIK it’s only possible to verify file checksums not the permission.

SecureCon Lecture

On Thursday at Secure Con [1] I gave a lecture about SE Linux that went according to plan, and they gave me a nice bottle of Penfolds Shiraz afterwards (thanks to the sponsors).

During my lecture I announced my plan to run the hands-on training session over the net. The idea is that the Debian and CentOS images from jailtime.org with minor modifications will be put online somewhere for anyone to download. Anyone can then run the images on their own Xen server, go through the exercises, and ask questions on IRC at the same time. If you are interested in such training then please indicate in a comment what times would be good for the IRC discussion. Note that I’m only available between 7AM and 10PM starts in time zone +1100 (that is 20:00 to 11:00UTC for the starting time), the finishing time would be two hours later – and it would be possible to do the training in multiple sessions.

One interesting thing was that at the end the moderator of the session offered a box of lollies to the first person who could tell him my user-name (which was included in ls output on one of the slides).

Afterwards I was in idle conversation with some delegates and the topic of the Mac Mini [2] machines came up. Those machines are smaller than the Cobalt Qube (that I have in the past lugged around for portable SE Linux demonstrations), quite powerful (1G of RAM with an 80G hard drive seems to be the minimum for buying new at the moment), and they have keyboard and video ports which is often more convenient than sys-admin by serial port. I am now patiently waiting for Intel-based Mac Mini’s to start selling cheaply on eBay. Such a machine with 1G of RAM would make a nice SE Linux demo machine, I could run at least 7 Xen DomU’s for different users! Of course a second-hand laptop would do just as well, but laptops seem to hold their value better than most other machines.

One thing that disappointed me was the small turn-out for the conference dinner. It seemed that as there was a gap in the program between the official end of the conference at 5PM and dinner at 6PM most people decided to go home. One thing to note for future events is that leaving gaps in this way is probably a bad idea. Maybe if they had said “drinks at the restaurant from 5PM and dinner at 6PM” then the turn-out would have been better.

SecureCon Tutorial

My SecureCon tutorial went quite badly today. After having network problems and having both the Xen servers crash for no apparent reason I had to give up and give an impromptu lecture.

The original plan had been to use two Xen servers which each had 15 instances and have the delegates go through a training program that involved installing SE Linux on Debian and CentOS and comparing the features of them for various tasks.

Instead I spent just over two hours talking about SE Linux without notes (the beamer didn’t like my laptop and the desktop it was connected to was locked). I did end up getting another desktop machine working later in the lecture to type some notes.

My plan now is to make all the files available for download, additionally make some instances available on one of my servers, and then run some training via IRC.

Xen for Training

I’m setting up a training environment based on Xen. The configuration will probably be of use to some people so I’m including it below the fold. Please let me know if you have any ideas for improvements.

The interface for the user has the following documentation:

sudo -u root xen-manage create centos|debian [permissive]
Create an image, the parameter debian or centos specifies which
distribution you want to use and the optional parameter permissive
specifies that you want to use Permissive mode (no SE Linux access controls
enforced).
Note that creating an image will leave you at it’s console. Press ^]
to escape from the console.
sudo -u root xen-manage list
Display the Xen formation on your DomU. Note that it doesn’t tell you whether
you are using Debian or CentOS, you have to access the console to do that.
sudo -u root xen-manage console
Access the console.
sudo -u root xen-manage destroy
Destroy your Xen image – if it’s crashed and you want to restart it.

Continue reading Xen for Training

etbe – Russell Coker

Archives

Categories

Better Social Networking

The Price of Food

SE Linux in other Distributions

Safe Banking by SMS?

Software vs Hardware RAID

More About Living in Hotels

Restorecon Equivalent for Unix Permissions

SecureCon Lecture

SecureCon Tutorial

Xen for Training

Archives

Email and RSS

Archives

Categories

Tags

Archives

Email and RSS