|
|
Flame has written some ideas about building Linux Powered Battle Droids and demonstrating them [1].
It seems to me that the biggest problem with having a battle of a similar nature to Robot Wars [2] is creating a safe arena. Getting an area the size of a basketball court fenced off with bullet-proof plastic is not going to be cheap.
The first solution to this problem that occurred to me was to have a battle held underwater. Water really slows down projectiles. The disadvantages of this are that it’s slightly more difficult to view. The viewing options are to using a web-cam, having a port-hole in the side of the pool (impossible if you don’t own the pool), and having a transparent viewing platform on the surface of the water (which would also be difficult and maybe expensive). Another disadvantage of submarine warfare is that having a droid spring a leak and quietly sink is not very dramatic.
It might be a better option to use aerial warfare via helicopters and balloons. The amount of weight that such craft can carry is not particularly great so there will be no real armor and fairly weak weapons. A motorbike helmet and a heavy coat should be all the protective equipment that is needed if the usual rules prohibiting projectile weapons from such contests are implemented.
A petrol powered model aeroplane could do some serious damage if it hit someone at full speed. But I don’t think that an indoor basketball court or any other enclosed arena that might be available would be large enough for fighter-planes. So it would be just balloons and helicopters.
Stewart Smith has written about the removal of a blog from Planet Linux Australia [1] due to publishing a list of URLs that the Australian government wants to censor.
The first point I want to make is that even if you had a list with thousands of entries that are not likely to offend anyone or incur any legal liability then it’s still not suitable for syndication on most Planet feeds. The correct thing to do is to have a paragraph describing the list and why people would want to read it and then use the MORE feature of your blog so that the rest isn’t in the RSS feed. If you use WordPress which seems to have the MORE function broken then that would mean hosting the list somewhere else.
In regard to the specific post, in a comment on Stewart’s post Matt suggests that the Planet software somehow filter out certain blog posts. I am not aware of any way of doing that apart from through code changes, Matt could submit some patches to allow that sort of thing.
One thing that would be really good would be to have an exclusion tag or category in a blog feed. So you for example you could have feed URLs such as /feed/lca which would be configured to list all posts without the tag not-lca. Another way for a blogger to do this would be to use Yahoo pipes [2]. The people who run a Planet should be prepared to take any feed URL. It would not be difficult for a blogger to create a pipe that excludes all items that have “NSFW” in the title (or any other possible way of listing them).
A final option is to have multiple blogs. I have a blog for documents that I regularly update [3]. Many of those documents had been plain HTML files edited with vi for years before I started blogging. But WordPress is a reasonable CMS and as I use it for blogging it made sense to use it for other documents too. WordPress has no good option for managing two types of documents, ones that are date-based (regular blog posts with the date in the URL) and non-date based (which change periodically and have different date stamps). There are WordPress pages, but the support for having moderate numbers of pages is not great. Also on my document blog I will often have articles appear new regularly as I change the date when updating them. Anyone is welcome to subscribe to the feed for my document blog if they are interested in seeing new versions of the documents, but I expect that most people don’t want to.
The Debian WordPress package (as of last time I used it) and my fork of the Debian WordPress package have great support for multiple blogs. There is WordPress-MU for bulk blog hosting, but that is only designed for people who want to run something like LiveJournal or Blogger. If you just want a few blogs for friends and relatives then the regular Debian WordPress package will do the job well.
Some bloggers maintain two blogs, one for public things and another for close friends and relatives (people who ARE interested in what they ate for breakfast). Having one blog for the NSFW material would be a reasonable thing to do for certain bloggers.
Finally while I doubt that someone who runs a Planet installation faces any legal liability, there is also the issue of a PR liability. From a PR perspective I think it’s best for the reputation of Linux users in Australia for certain things to not appear on Planet Linux Australia. That said it would be good if there was a process for removing and reinstating blogs that was publicly documented. There will obviously be many differences of opinion as to what is too risky to allow on the Planet so we should expect that from time to time feeds will be temporarily removed. When that happens what does a blogger have to do to be done to be syndicated again?
Update:
A comment has revealed a way of filtering out RSS feeds via the feed URLs used by wordpress. A URL such as /feed/cat=-X will give a feed of all articles that don’t contain category number X. Multiple categories can be specified when separated by commas. So this allows WordPress users to exclude their NSFW category from Planet Linux Australia.
A client is considering some options for serious deployment of some CPU intensive work. The options that are being considered include cloud computing (Amazon EC2 [1]), virtual machines (Slicehost [2] and Linode [3]), and purchasing servers to install in racks at various locations. I can’t disclose the criteria that will determine when each of those three options will be used (I expect that we will end up using all of them). But my research on the prices of various servers will hopefully be useful to someone.
For the server vendor I chose Dell. I believe that HP offers slightly better quality hardware than Dell, but they cost more and are more difficult to deal with (I can’t even get a price online). For this project I will be using a bunch of redundant servers (in a similar concept to the Google server array) so I’m not going to be overly bothered about losing a server occasionally – therefore the slight benefit that HP offers for reliability does not make up for the expense.
Dell has some 1RU servers that have two CPU sockets and allow eight CPU cores. It seems that the best value that Dell offers for a server without RAID (the entire server is redundant) is a PowerEdge SC1435 that has two Opteron 2352 quad-core CPUs running at 2.1GHz, 4G of RAM, a 1TB SATA disk, and a Broadcom PCIe Gig-e card for $3,816.50. That machine gives an option of 2.3GHz CPUs for an extra $621.50, I am not sure that increasing the clock speed by almost 10% for a 16% increase in system price is a good idea.
The second best option was a PowerEdge 1950 III that has two Xeon E5420 2.5GHz quad-core CPUs with 12M of cache, 4G of RAM and a 1TB SATA disk for $4,302.30. The Intel option has 3 years of support included while the AMD option included 1 year of support and needed at least an extra $990 for 3 years of support. So it seems that if 3 years of support is desired then the Intel based server becomes significantly cheaper and is probably a better option.
Dell’s 2RU and 4RU servers are of no interest if you want CPU performance. The 2RU servers only support two processors and the 4RU servers only support four processors. So it’s a ratio of 2 processors per RU for 1RU servers vs one processor per RU for 2RU and 4RU servers, and the 2RU and 4RU servers are a lot more expensive too.
I am investigating the Dell blade server. Blade servers are great for CPU density and good for management. The Dell blade enclosure M1000e takes 10RU of space and supports 16 half-height blades or 8 full-height blades. The Dell M905 blade supports four AMD quad-core processors for a total of 128 cores in 10RU, there are also half-height blades that support two quad-core processors for the same CPU density.
So in terms of CPU density it’s an average of 12.8 cores per RU for the blade server vs 8 cores per RU for 1RU servers. While I haven’t got a complete price yet, it seems that four CPUs suitable for the M905 will cost about as much as four 1RU servers. So the 1RU systems are definitely better value for money than the blade server. The difference is the management cost. N servers that have two CPUs will be more work than N/2 servers that have four CPUs, but on the other hand blade servers require some specialised skills to run them (which I don’t have) and that might also cause problems. I don’t think that blades will be part of this project.
When reading a magazine I often see an advert for a product that I want to buy (or recommend that a client buy). This is of course expected as the advertisers put a lot of effort into targeting their adverts to the people who read such magazines. However I often decide that I want to buy the product some weeks after reading the magazine.
Linux Journal [1] usually has adverts for good server hardware that supports Linux. Their web site is quite nice in many ways, it allows subscribers to read articles online and has an index to all back-issues. But it has no link that I could find for reading the adverts! I would like to see an index of all advertising that has ever been published so that I can read the adverts online. The ancient adverts are good for historical reference and the new ones are good for purchasing decisions.
Right now I would like to be suggesting that a client consider buying a number of servers from a company that advertises in Linux Journal, but I’m not because they don’t publish their adverts online!
To make matters worse the Linux Journal web site doesn’t include a suitable contact address for issues unrelated to subscriptions. I believe that a magazine needs to publish an email address for copyright infringement reports (I often notify magazines when someone rips off their content so that they can issue a DMCA take-down notice), an email address for technical problems (I often notice rendering errors and broken links in web pages and like to report them), and an email address for random stuff. I’m even happy to use a web-based form to submit my suggestions if that’s what makes them happy, but having no published way of notifying them is simply a mistake.
NB I do have some email addresses of LJ employees in my addressbook somewhere, I will send them email if no-one responds to this blog post. But I’m publishing this because it’s probably a mistake that other magazines make, and because any response from them is going to be too late for the report on hardware prices that I’m writing.
Update: Linux Journal does allow you to download a sample copy, that is a PDF of one complete issue that includes adverts. So I can see adverts from August 2008.
An article from 1999 suggested that car crashes caused a financial loss in OECD countries equivalent of 2% of their entire economies [1]. An article from the Sydney Morning Herald in 2001 gave a conservative estimate of the cost of a road fatality at $1.5 million [2], it also notes that due to different analysis methods American transport economists derived a figure of $5.5 million. $1.5 million in 2001 adjusted for CPI would be close to $2 million now.
Currently that $2M cost is an externality of the car industry. Most of it is paid by the government, IE we all pay for it through our taxes. This means that there is little financial incentive for drivers and car companies to make the roads safer. Many of the attempts to legislate road safety fail due to the legal system being unable to manage the rapidly changing range of vehicles on the market.
The insurance companies have very detailed analysis of the relative safety of vehicles, so it seems that the only sensible way of enforcing safe driving is through economic measures implemented via insurance.
I believe that for every person who is killed or seriously injured on the road a fine of $2M should be levied. Every driver should be compelled to have insurance to cover such fines (driving without insurance should be illegal).
Then the government could cease being involved in regulating what types of car someone can drive. If someone who is less than 25 years old can get insurance for a turbo-charged car then it probably means that a statistical analysis suggests that the combination of driver and vehicle is likely to be reasonably safe (EG there are many turbo-charged cars on the market that are not particularly fast).
Now this will increase the car insurance costs for everyone, but it will decrease the amount of general tax money that is spent on issues related to road fatalities, which would allow the income tax rates to be decreased. This means that any tax-payer who has a good driving record and who drives a type of car that tends not to be crashed could expect to save money overall. Any tax-payer who doesn’t drive a car would save even more money.
But the main point of this idea is to increase road safety by forcing bad cars and drivers off the road. Currently defective cars are only removed from the road if police notice something unsafe about them and cite them for being unroadworthy – this only happens if it’s a problem which can be observed from outside the vehicle (EG worn tires or broken lights). In some states elderly drivers have no requirement for periodic health checks to determine their ability to drive, I know of one case of a woman who was certified as legally blind, ordered a white cane, and then drove home afterwards! I’m sure that insurance companies would implement whatever tests are necessary to reduce the risk of being hit by multiple $2M fines from a single crash.
Bruce Schneier writes about the risks involving children abandoned in cars and cites an article about the tragic deaths of children in hot cars [1]. One unfortunate error that he made was to not cite the following from the end of the last page of the Washington post article he cited [2]:
In hyperthermia cases, he believes, the parents are demonized for much the same reasons. “We are vulnerable, but we don’t want to be reminded of that. We want to believe that the world is understandable and controllable and unthreatening, that if we follow the rules, we’ll be okay. So, when this kind of thing happens to other people, we need to put them in a different category from us. We don’t want to resemble them, and the fact that we might is too terrifying to deal with. So, they have to be monsters.”
I believe that similar thought processes are used in relation to many other situations, and that such thought processes prevent people from taking appropriate actions to minimise the risk. If someone considers that forgetting a child in the back seat to be an accident that could happen to anyone then they would be inclined to take action to minimise the risk (such as spending some money on a sensor). If however they consider such forgetfulness to be proof of being a “bad parent”, then as they are a “good parent” they would have to avoid buying a monitor. I’m surprised that Bruce didn’t draw an analogy between this and the forgetful losses of laptops and guns by people who work for law enforcement agencies (which he has written about before).
I wonder how expensive it would be to make a sensor for heart-rate, breathing, and temperature integrated with a GSM modem and a GPS? If it could be small enough to be attached to clothes then the child could wear it at all times.
If such a sensor was to detect a sign of a problem it wouldn’t matter whether the child was forgotten in a car, at day-care, or even being actively supervised. The data would be sent to the monitoring agency along with GPS data. The monitoring agency could then phone the parents. If the parents don’t answer or don’t know where the child is then the police could track down the GPS location. Probably most calls would be due to parents leaving a child too close to an air-conditioner or playing outside in the sun in summer which are unlikely to give a fatal result and a phone call would get a quick fix for what would only be a minor health problem.
If the device was marketed as monitoring for “sleep apnia” then parents could buy it without admitting to the possibility that they might do anything wrong. The causes of SIDS are a topic of ongoing research and parents can admit to being worried about their children suffering from it without admitting any possibility that they might make a mistake.
I am not aware of any Linux Users Group (LUG) being active in informing it’s members of how the policies of the various political parties compare with regard to free software and the other issues that are of interest to most members. I believe that this is a grave mistake.
Shortly before an election there are many social groups that send lists of questions to all the parties. They ask about the policies the parties have in regard to the issues that they care about, and helpfully mention the number of members that will receive the response. This of course doesn’t mean that every member of the group in question will cast their vote in the same way, merely that they will take note of the answers.
The committee members of the parties in question will then decide how to answer the questions and whether policy should be tweaked to allow answers that the lobby groups will like. So this process not only helps members of a group make informed voting decisions related to issues that they care about, but it also helps political parties choose policies that are least offensive to the group in question.
Here is a draft of a list of questions that I think should be asked of all political parties on behalf of Linux users:
- It is important for all citizens to access all government data without being forced to buy new software or hardware, open standards allow everyone to access the data with free software. Do you support the use of open standards for data on government web sites and other forms of electronic communication between government agencies and citizens?
- For long term archival of records it is important that file formats remain readable. The only effective way of doing this is to use open file formats that are implemented in free software. Do you support mandating that all data submitted to government agencies (by citizens or corporations) be in open file formats wherever possible?
- In these difficult economic times there is a great interest in keeping jobs in the country instead of sending money overseas. To what extent do you support the use of free software that is installed and managed by locals (keeping the money in the economy) instead of importing software at great taxpayer expense?
- Commercial software has a limited support period, after that time has elapsed there is no further support and systems become increasingly unreliable. Do you support mandating that all systems relating to the emergency services run on free software to allow quality long-term support by local citizens?
- There has been a lot of concern recently about the spread of child-porn. The best available evidence shows that insecure home PCs that run “Trojan Horse” programs are a key part of distributing it and other illegal material. Do you support the introduction of government programs to train parents in installing one of the more secure free operating systems on their home PC to protect their children?
This is just a rough draft. Obviously there needs to be local differences (EG don’t use point 3 in the US because MS brings money into the US economy).
Does anyone have any suggestions for other questions?
I have just been asked for advice about “secure filesystem” and decided to blog my answers.
The first issue is what is meant by “secure filesystem, that could either mean the ability to restrict file access (EG by supporting SE Linux security contexts and using SE Linux for file access control) or the ability to encrypt data in case the machine is stolen. For access control I recommend SE Linux of course. For encryption on a local machine I mostly use dm-crypt which is configured with the cryptsetup utility. I encrypt at the LVM logical volume level as it is common that there are some LVs that don’t need to be encrypted. For files that need extra encryption or files that are shared between machines I use GPG.
A question was asked about kernel vs user-space filesystem encryption. AES is in the kernel so there is no lack in terms of strong encryption there. Also performance is pretty good (in most cases the CPU is fast enough that the hard drive is the bottleneck). For fine grained encryption (such as some of the experimental filesystems that encrypt data separately for each user) user-space is probably the only way to go.
If you want servers to be “high-security level” and protected from “hackers or unauthorised people” then it’s difficult to offer any advice that is smaller than a text book. I suggest that if you have such questions then you should do one of two things. If you are running a corporate IT department then hire an expert who can help with determine your specific requirements and meet them. If you want to learn about computer security and run your own systems in the best way possible then read as much from the experts as possible.
If you are looking for a project to contribute to related to security then if you choose SE Linux I could offer some specific advice on things that need work. I suggest not deciding on whether to do “kernel level or user level” work up front, but decide first which area of security you want to work on and then select a project which fits – then you should be able to determine whether your skills are best suited to kernel or user space coding. As for whether developing a new filesystem is necessary, I will note that SE Linux works well on Ext3 and XFS, it has just become usable on JFFS2, and it will work on other newer filesystems in the near future. Adding SE Linux support to a filesystem is not a difficult task if the filesystem supports XATTRs. I believe that there is a lot of scope for other access control systems to be developed which use XATTRs for security labels.
I can’t advise on e-books. I generally don’t read books, I read blogs and papers. Anything that I read which I consider to be worth recommending will probably have a link from my blog.
I’m in the middle of migrating a mail server away from the Cyrus mail store [1]. Cyrus provides a POP and IMAP server, a local delivery agent (accepting mail via LMTP). It is widely believed that Cyrus will give better performance than other mail stores, but according to a review by linux-magazin.de Dovecot and Courier deliver comparable (and sometimes better) performance [2].
The biggest problem with Cyrus is that it is totally incompatible with the Unix way. This wouldn’t be a problem if it would just work and if it would display reasonable error messages when it failed, but it doesn’t. It often refuses to work as desired, gives no good explanation, and it’s data structures can’t be easily manipulated. Dovecot [3] and Courier [4] use the Maildir++ format [5] (as well as many other programs). I have set up a system with Courier Maildrop and Dovecot for the IMAP server [6] and it works well – it’s good to have a choice! But also Maildir++ is reasonably well documented and is an extension to the well known Maildir format. This means that it’s easy to manipulate things if necessary, I can use mv to rename folders and rm to remove them.
Cyrus starts with a database (Berkeley DB file) of all folders in all mailboxes. Therefore it is not possible to move a user from one back-end server to another by merely copying all the files across and changing the LDAP (or whatever else contains the primary authentication data) to point to the new one. It also makes it impossible to add or remove folders by using maildirmake or rm -rf. The defined way of creating, deleting, and modifying mailboxes is through IMAP. One of the problems with this is that copying a mailbox from one server to another requires writing a program to open IMAP connections to both servers at once (tar piped through netcat is much faster and easier). Also if you need to rename a mailbox that contains many gigabytes of mail then it will be a time consuming process (as opposed to a fraction of a second for mv).
Cyrus has a tendency to break while Dovecot is documented as being self-healing and Cyrus also seems to cope well in the fact of a corrupted mail store. Even manually repairing problems with Cyrus is a painful exercise. The Cyrus mail store is also badly designed – and it’s design was worse for older filesystems (which were common when it was first released) than it is for modern ones. The top level of a Cyrus maildir contains all the messages in the INBOX stored one per file, as well as three files containing Cyrus indexes and sub-directories for each of the sub-folders. So if I want to discover what sub-folders a mailbox has then I can run ls and wait for it to stat every file in the directory or I can use an IMAP client (which takes more configuration time). As opposed to a Maildir++ store where every file that contains a message is stored in a folder subdirectory named “new“, “cur“, or “tmp” which means that I can run ls on the main directory of the mail store and get a short (and quick) result. Using tools such as ls to investigate the operation of a server is standard practice for a sysadmin, it should work well!
A finall disadvantage of Cyrus seems to have many small and annoying bugs (such as the reconstruct program not correctly recursing the sub folders). I guess it’s because not many people use Cyrus that such things don’t get fixed.
One trivial advantage of Cyrus is that by default it splits users into different sub-directories for the first letter of the account name. Dovecot supports using a hash of the user-name this is better than splitting by first-letter for performance (it gives a more equal distribution) but will make it slightly more difficult to manipulate the mail store by script. Ext3 can give decent performance without a two level directory structure for as many as 31,998 sub-directories (the maximum that it will support) due to directory indexing and Linux caching of dentries. There may be some other advantages of Cyrus, but I can’t think of them at the moment.
Here is a script I wrote to convert Cyrus mail boxes to Maildir++. To make this usable for a different site would require substituting a different domain name for example.com (or writing extra code to handle multiple domains) and inserting commands to modify a database or directory with the new server name. There is no chance of directly using this script on another system, but it should give some ideas for people performing similar tasks.
Continue reading Why Cyrus Sucks
I have recently configured my mail server to use IMAP. I started doing this when I was attending Linux.conf.au so that I could read urgent mail using my EeePC while at the conference and then be able to deal with the more complex stuff using my laptop later on.
The next logical step is to have mail delivered to different folders in the IMAP account. While there are ways of doing this via the Subject and other header fields, my needs are not that great. All I need to do is to support user+extension@example.com going to a folder named extension in the user’s mail store. While changing my mail server I decided to install Postfixadmin at the same time.
My first attempt to use Maildrop was to put the following in the /etc/postfix/main.cf file:
mailbox_command = /usr/bin/maildrop -d mail -f “$SENDER” “$DOMAIN” “$USER” “$EXTENSION”
That seems to only work when you have local accounts, so I ended up setting fallback_transport = maildrop and then putting the following in /etc/postfix/master.cf:
maildrop unix – n n – – pipe flags=DRhu user=vmail argv=/usr/bin/maildrop -d vmail ${nexthop} ${user} ${extension}
Where vmail is a Unix account I created for storing mail. Then I added the following to /etc/postfix/main.cf. Some of these are probably redundant (such as the virtual_mailbox_base). The recipient limit is set to 1 because there are no command-line parameters for maildrop to support two recipients for the same message.
virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf
virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_gid_maps = static:2000
virtual_uid_maps = static:2000
virtual_mailbox_base = /mail
vmaildir_destination_recipient_limit = 1
virtual_transport = maildrop
maildrop_destination_recipient_limit = 1
The files /etc/postfix/mysql* all have fields user=, password=, hosts=, and dbname=. The queries in each of the files are as follows:
mysql_virtual_alias_maps.cf:query = SELECT goto FROM alias WHERE address='%s' AND active = 1
mysql_virtual_domains_maps.cf:query = SELECT domain FROM domain WHERE domain='%s'
mysql_virtual_mailbox_maps.cf:query = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1
The /etc/courier/maildroprc file has the following contents:
# log the maildrop actions
logfile "/var/log/maildrop.log"
#
# parameters 1, 2, and 3 are the domain, user, and extension
DOMAIN=tolower("$1")
USER=tolower("$2")
EXTENSION=tolower("$3")
DEFAULT="/mail/$DOMAIN/$USER"
#
# try making a backup (cc) copy of the mail but do not abort if it fails
exception {
cc "$DEFAULT/.backup/"
}
#
# try delivering to the extension folder but do not abort if it fails
exception {
if(length("$EXTENSION") != 0 && "$EXTENSION" ne "backup")
{
to "$DEFAULT/.$EXTENSION/"
}
}
#
# deliver to the main inbox if there is no folder matching the extension or if no extension is specified
to "$DEFAULT/"
Installing Postfixadmin [1] was another challenge entirely. One of the complications of this is that there is no Debian package for Lenny (it seems that there will be one in Squeeze – Lenny+1).
I found David Goodwin’s tutorial on installing Postfixadmin and lots of other things on Debian/Etch [2] to be a very useful resource. I look forward to seeing a Lenny version of that document.
Please let me know if you can think of any way to improve this.
|
|
