etbe - Russell Coker

Some Postfix Scripts for dealing with Outbound Spamming

I’ve just written some small scripts to help me manage spam emergencies on a mail server. I’ve been doing this thing with a bit of manual effort for a while, but after having done it once from a phone I want to optimise it a bit to reduce painful typing.

My observation is that when a system I run is used as an outbound mail relay for spamming I will notice this reasonably quickly by the queues getting big. Some portion of mail that is queued is delayed due to general network issues and anti-spam measures that neither accept nor reject mail cause a typical spam message to be more likely to be queued than a typical non-spam message. So I look for accounts that send large amounts of mail.

The biggest mail server I run (and the only one to have an outbound spam problem) is configured to not allow users to fake their sender address (IE use an address from gmail.com or another server) unless they pay extra. So the spam I deal with tends to have a valid origin email address that I can use.

Queue Counting

The following command provides a sorted list of the accounts that have the most mail in the queue. This usually means a maximum of 2 or 3 spammers at the end of the list:

The following function declaration makes the command get-top-sender find the sender with the largest number of queue entries (which has been a spammer every time I’ve checked such things) and assign their email address to the environment variable $TOP_SENDER. It has to be a shell function so that the environment variable will be set in the context of the shell and available to child processes.

function get-top-sender() { TOP_SENDER_LINE=$(mailq|grep ^[A-F0-9]|cut -c 42-80|sort |uniq -c|sort -n|tail -1) ; export TOP_SENDER=$(echo $TOP_SENDER_LINE|cut -f2 -d\ ) ; TOP_COUNT=$(echo $TOP_SENDER_LINE|cut -f1 -d\ ); echo "Top Sender is $TOP_SENDER with $TOP_COUNT messages" ; }

Viewing the Queue

The following script uses the first parameter or the environment variable $TOP_SENDER to specify the user who’s mail should be read. In the normal course of events user email won’t be read by the sysadmin, but if we are to determine whether a slew of email is spam or not there’s no other way. So far I haven’t seen a large number of queued messages not be spam, but I expect it’ll happen eventually and the user will be happy that I checked it instead of locking their account and deleting all the queued mail. For readers who don’t do sysadmin work, this is why we always have entries in the Terms of Service about the possibility of your email being read to fix technical problems or to investigate possible breaches of the ToS from your end.

#!/bin/bash
set -e
mkdir -p ~/tmp/$$
cd ~/tmp/$$
if [ "$1" != "" ]; then
TOP_SENDER="$1"
fi
if [ "$TOP_SENDER" = "" ]; then
echo "Specify the sender on the command-line or in \$TOP_SENDER"
exit 1
fi
for n in $(mailq|grep ^[A-F0-9].*$TOP_SENDER| cut -c 1-10) ; do
FILE=$(echo $n|sed -e "s/^$.$/\/var\/spool\/postfix\/deferred\/\1\/\1/" -e "s/ .*$//")
if [ -f "$FILE" ]; then
postcat "$FILE" > $n
else
echo "File $FILE missing, probably active"
fi
done
less *
rm -rf ~/tmp/$$

Note that this needs to be done with a single less command so that I can terminate it by pressing q and easily go to the next file with “:n“. It does waste some server resources by running postcat on all queue files relating to the user in question, but that doesn’t matter, servers are supposed to be powerful enough to cope with some inefficiency in processing uncommon operations.

The Final Command

After that I have a script that connects by ssh to all outbound mail relays and deletes bad messages from the queues and then connects to the database server to lock the account that was used for sending the mail. I’m not publishing it because it’s specific to the servers I run.

So when NAGIOS reports that the queue is too large I have one script to find the most likely culprit, one script to view the queued email from that account, and a final script to lock the account and purge the spam.

I’ve also added a reminder of the command names to /etc/motd as I don’t want to be running ls and set on a phone to discover command names that I’ve forgotten.

Any suggestions for improvements to this will be welcome.

Effective Computers for Schools

Sam Varghese has written an article that is very skeptical of the educational value of the OLPC project [1].

Are Laptops Any Good for Schools?

Sam cites an article in the New York Times by Winnie Hu about schools removing laptop programs due to a lack of success [2]. Winnie’s article gives an example of a school shutting a program because of getting new teachers who lacked computer skills and some examples of schools which had issues with the repair cost. The solution to that would be computers that are more robust and easier to use – by all accounts the OLPC systems are solidly constructed and easy to use!

It is claimed that using laptops doesn’t increase test scores. Using only test scores to compare educational methods is a way to lose, probably any school which does nothing other than try to increase test scores is worth avoiding. One example from Winnie’s article is a report by Mr. Warschauer of “students at a middle school in Yarmouth, Me., who used their laptops to create a Spanish book for poor children in Guatemala and debate Supreme Court cases found online“. What value can you place on having students develop books for poor children in other countries and debate Supreme Court cases? If this sort of program became popular then it would lead to the countries which do it becoming better places to live in future decades!

Mr. Warschauer also claims that “If the goal is to get kids up to basic standard levels, then maybe laptops are not the tool. But if the goal is to create the George Lucas and Steve Jobs of the future, then laptops are extremely useful“. I would go further than that, I think that providing the educational environment that involves international charity work and analysis of important court cases would lead to students having the skills to become good engineers, designers, and artists who work for people like George Lucas and Steve Jobs. No matter what you might do the vast majority of children will not grow up to be like George Lucas and Steve Jobs, there is a very limited number of positions for such people. But those people employ a huge number of creative people who do interesting and enjoyable work for good pay.

I don’t think that Winnie’s article supports criticism of the OLPC project. I don’t think that Winnie’s article even contains sufficient evidence to match the headline.

Playing Games

Sam also cites an article about the misuse of school computers for playing games [3]. Of course there are ways of limiting access to games based on time etc. For a Linux system you could have a root cron job that runs chmod on /usr/games at various times, and preventing users from easily running programs that they install isn’t difficult.

Not that games are all bad, the flash-based games on www.physicsgames.net do teach kids some things about physics – although I admit that much of that could be learned by playing with balls, skate-boards, etc.

The Problem with the OLPC Project

I think that the policy change to make OLPC systems only available to disadvantaged children was a mistake. The “give one get one” program was a great idea, maybe not the most effective way of getting funding but good for getting developers. While the new Sugar on a Stick project to put the OLPC GUI on a bootable USB device [4] it doesn’t compare well to having dedicated hardware IMHO.

Minimal Use of Computers

I have previously written about the weight of school bags and how laptops can alleviate the risk of health problems related to carrying heavy text books [5]. Since that time ebook readers and tablets have become incredibly cheap. Ebook readers are really light and the recent tablets are also very light (particularly the smaller ones). It seems to me that every school should at least be moving towards every student having an ebook reader and to use ebooks for all books that are part of the curriculum.

At the moment there are a bunch of tablets on sale for about $150, that includes tablets that can access the Internet via Wifi or via 3G – and a cheap 3G plan costs less than $150 per annum. If tablets were used for some of the computer tasks related to schools then a lot of the difficulty of repairing hardware would be removed. Tablets don’t offer as many options for messing up the software configuration and if the hardware breaks it’s easy to transfer the data onto a new tablet from the cloud – with a maximum cost of $150 to replace the hardware.

Also a cloud-based computing model could permit students to access all the same data from school and home while using desktop computers at both locations.

Current Success

Natalie Craig wrote an interesting article for The Age about “The Lab” – a drop-in computer center for kids on the Autism Spectrum run by Dale Linegar and Stefan Schutt [6]. The fact that kids who have communication difficulties can find it easier to communicate electronically should be fairly obvious to everyone in the free software community, but the creation of an organisation to support such kids is noteworthy – I think that such centers should be funded by the government and run in every city.

Salman Khan gave an interesting TED talk about his project the Khan Academy [7] which is an online video-based teaching system that evolved from some Youtube videos he produced to tutor his cousins. Salman pointed out that his cousins said that they would rather watch his videos than have him teach them in person, it seems that they liked being able to pause and rewind the talk as well as avoiding the pressure of human interaction. The Khan Academy has videos in Adobe flash format, but conveniently they have download links so those of us who don’t have the Flash plugin can still view the content [8].

One of the interesting things about the Khan Academy is that in school use the videos are being assigned to kids as homework and the class time is used for teaching children to do worked examples – what has traditionally been homework.

So it seems that there are real examples of special-needs kids and average kids benefiting from different types of electronic learning.

Conclusion

I think that almost everything about the education system is broken. I also think that in many ways the education system has reached a local maximum, so there is no small way of improving things, and adding computers without making any of the significant changes needed to fix the big problems probably can’t do a lot of good. In spite of this computers can provide some real benefits and I expect that those benefits can be better than the benefits of other potential ways of spending the money (IE it’s worth the opportunity cost). I also have no doubt that almost anything can give a negative result if done badly enough, so I don’t interpret examples of computer use failing in a school as anything other than evidence of a failing school.

While the Khan Academy has some results that seem very positive I think that’s only the first step of what needs to be done. It also seems to be based around users who have good Internet access which often isn’t the case in places where the OLPC is being deployed. This implies that we need to get systems like the Khan Academy designed to operate in a disconnected manner, maybe with a remote server that’s not connected to the Internet. This should be possible if funding is available.

Sam suggested that there needs to be a scientific study of the effectiveness of the OLPC. I think that every project which involves significant amounts of public funding should be researched to ensure that it’s an effective use of resources. 400,000 laptops is going to cost something more than $40,000,000 so it seems reasonable to devote a few person-years of research to determine how effective they are and which of the possible ways of using an OLPC will give the best results.

I am confident that a good study of the effectiveness of the OLPC would demonstrate that it provides real educational benefits when correctly incorporated into an educational program. I also expect that such a study would show some significant differences in the effectiveness of various ways of using them. But we shouldn’t be relying on confidence, we need some test results.

Radiation Poisoning

There have been ongoing news reports about the nuclear power plant problems in Japan following the earth-quake and tidal-wave. The Wikipedia page about the Fukushima I nuclear accidents seems to have the best current summary of the situation [1].

I think it’s worth noting that Potassium Iodide can be ingested to reduce the incidence of thyroid damage in the event of radiation leaks (which apparently usually contain radioactive iodine). KI can be ingested immediately after exposure and there’s apparently little harm in taking it before a time of risk (it can be good to take it for a couple of days before exposure). So getting some KI right now might be a good idea for people who live in the vicinity. Apparently the Japanese government have issued iodine tablets to the people closest to the disaster, but it’s probably worth spreading the word to people who are further away.

Leigh Krietsch Boerner gave a good explanation of how Iodine significantly reduces the risk of thyroid cancer [2].

The Nuclear War Survival Skills site has a good description of how to make your own KI solution [3]. This probably isn’t a viable option for anyone in Japan unless they can raid a chemistry lab as mail-order of chemicals will surely be too slow. This is probably something that’s best considered for future plans for anyone who lives near a nuclear reactor.

It would be good if airport pharmacies sold packs of KI tablets for the benefit of travelers who don’t live near a reactor but who are visiting a country that has them. It’s something I’d like to buy before my next trip to Japan.

Climate Action Now Rally Melbourne 12th March

This morning I attended the Climate Action Now rally to support a pollution tax [1]. The event was well attended, my personal estimate of the number of people there was there there was definitely more than 3,000 people, and maybe as many as 8,000 or more.

I spoke to an employee of the APS who was on guard duty, he described the event as “big” and said that they don’t usually have events that are so well attended. He gave a personal estimate of 5,000 people. While he wasn’t speaking on behalf of the APS he was presumably more skillful at making such estimations than most people. I expect that he also gave a conservative estimate, so it could have been a lot more than that.

There were some people using comedy to make political points. Below is a picture of the carbon monster, what you can’t see is the man who was handing out coal BBQ fuel for people to feed the monster, children appeared to enjoy that! Also there were three people dressed as Gollum representing the Liberal party (the Australian conservative party), I was unable to get all three in one picture.

Some vegans had a banner supporting the VeganEasy.org site [2]. Being vegetarian can make a significant impact on human environmental impact including on the amount of electricity used (from coal or other sources).

Here are pictures taken looking East over the main part of the rally, looking North towards the building at 3 Treasury place from the other side of the road, looking South over the lawns where people who didn’t like being crowded gathered, and looking West over part of the main crowd. The picture looking South shows another view of the Carbon Monster and his accomplice with a sign “haven’t you had enough dear?“.

When I first arrived there were two main sections to the rally. They weren’t discrete (there was no empty space between them) but they acted separately because there was no PA system that could make enough noise to be heard in both sections. At one stage the West section was chanting while there were speeches in the East section. Below is a picture looking out over the West section as people had just turned to go home. This area was entirely packed when I first arrived.

The rally was really a family event. There were some grey-haired people wearing t-shirts explaining that they wanted to create a better world for their grand-children and there were also some young children. At the end of the event some children used cardboard signs as toboggans to slide down a steep grassy bank near the main rally area – recycling!!! I deliberately chose my photos to try and avoid including children, but even so you can see a few in the background of some of them.

How Not to Park a Mercedes

Why is that Mercedes blocking so much of the road through the station car park?

It’s because it’s the second car in a 1 car parking spot!

Here’s a front view.

These were taken in the Coburg Station car park on Wednesday. The car park was about half empty, so the alternative to blocking part of the road and blocking in someone who was legally parked was to just park about 10 meters further away from the station.

Some people just shouldn’t be driving.

Links February 2011

Australia’s Department of Finance has mandated that the MS-Office document format should be the standard document format for all agencies [1]. Paul Wayper notes the fact that MS doesn’t plan to support it’s own standards and suggests ODF, also known as ISO/IEC standard 26300:2006. [2].

Psychology Today has an interesting blog post by Marnia Robinson about scientific reasons for avoiding porn addiction [3]. Her post links to a number of other posts on the topic on Psychology Today.

Psychology Today has an interesting post by Noam Shpancer titled “What Doesn’t Kill You Makes You Weaker” [4]. It seems to me that the belief to the contrary is due to people rationalising bad things that happened to them, “it’s not all bad as it made me stronger”.

Psychology Today has an interesting article by Noam Shpancer about the soldier as hero myth [5]. He argues that soldiers are really just employees, the risk to American soldiers in war zones really isn’t that great and that they are just pawns to the military-industrial complex.

Anders Ynnerman gave an interesting TED talk about visualising medical data which includes many startling 3D pictures of humans and animals that are being scanned [6]. He describes how this is useful in forensic analysis by the police and in better understanding biological processes.

Margarita Manterola gave an interesting Debconf talk titled “Making Debian Rule, again”, she has some interesting ideas for improving Debian [7]. She is most focussed on social problems and the questions and comments had some interesting ideas. The Center for Non-Violent Communication [8] was mentioned in the questions, apparently their success includes teaching better communication skills to violent criminals, so it seems that they can help with some big problems.

Benjamin Mako Hill wrote an interesting article about the ratio of writers to readers on Wikipedia [9]. While he makes some good points I’m wondering about the measure of an “editor”, there have been few months where I have made 5 edits, so I guess I’m just a reader. I probably should take more care in logging in when I make changes though, I’ve made more than a few small changes from random systems (client sites, Internet cafes, etc) without bothering to login. This would mess up the stats a bit.

The Reid Report has an interesting article about Bush being limited in his international travel due to the fact that he has admitted being a war criminal [10].

Understanding Prejudice is an interesting article that summarises a lot of psychological research [11]. It gives many ideas for anyone who wants to reduce prejudice.

Robin Harris wrote an interesting ZD Net article about vibrations and the impact on disk performance [12]. Apparently it’s very common to be able to get more performance from drive arrays if you dampen vibration, previously I believed that unless you had extreme vibration to cause catastrophic problems then nothing would go wrong. But it seems that even smaller amounts of vibration can cause a 30% performance loss or worse! There is work in progress on vibration-damping racks.

In a story published in The Australian about the NSW ambulance service being shut down due to a virus, Professor Caelli cites SE Linux as an example of a technology to make computers more virus resistant [13]. Professor Caelli also states that there should be a legal requirement to have backup systems for computers that have such grave consequences if they crash.

Sinclair Community College is offering a new class “Fundamentals of Linux Security”, it includes SE Linux [14].

Slate has an interesting article about Pickpockets [15]. Apparently they are becoming extinct in the US as law enforcement has prevented the “Fagins” from training younger people, and it’s only in Eastern Europe that the skills base remains.

Virgin Mobile CRM Upgrade Failure

I’ve recently got a new Xperia X10 Android phone for my with with Virgin mobile, it’s generally been working OK although I am having some issues [1], I’ll write another blog post soon about other problems I’ve discovered with the phone and how I’ve solved some of the previous ones.

I upgraded my wife’s phone first because I can’t be without Nagios SMS messages if things don’t work. So now that things are generally working I want to get myself an Xperia through Virgin (and have my wife’s phone get the Nagios SMS in the mean-time). But since last Friday the entire Virgin sales infrastructure has apparently been down. It started with just declining my attempts to purchase a new phone on a separate account, but when I decided to add a second phone to my wife’s account the web site told me that they are upgrading their CRM system and it should be fixed on the 22nd of Feb (yesterday). The web site is now saying that I should “check back in 2 hours for an update“, it’s been saying that for a couple of days now.

So for most of a week potential Virgin customers have been turned away. It could be that Virgin stores are processing sales on paper, but they offer some significant discounts for web sales – the plan I want is a $39 per month plan and I’ll get 3 months free for buying on the web. I’m not about to visit a store and lose $117! I’m sure that many people are losing confidence in Virgin and taking their business elsewhere. I have only just installed the “3G Watchdog” free Android app that monitors bandwidth use and automatically turns off 3G when the quota is reached. For the first few days of using the phone which were more data intensive than usual I had no monitoring and no way of using the Virgin web site to discover how much was used. If Virgin bill me extra for data use I’ll complain and demand that they alter the bill.

This is even worse for pre-paid customers who can’t add credit to their account while this happens!

Virgin state that they are “upgrading the hardware, operating system and database our platform uses to ensure that we can service our Members even better for years to come” [2]. I wonder how people get themselves into such a mess. I guess they didn’t have a decent test environment to allow testing the upgrade process before doing it on the live data, I can understand a routine small upgrade going wrong and corrupting data in a way that takes some time to resolve. But when everything is upgraded then everything should be tested, and tested before going live! One thing that Virgin could do to regain some credibility is to publish what went wrong and what they learned from it. I would be much more happy to trust my personal data and my business critical phone to a company that learns from it’s mistakes and publishes plans on how they do better than one that just does PR.

According to Whirlpool they did the same thing on the 25th of January, so they have had two outages of their billing/CRM system in two months! [3]. The Whirlpool thread has discussion about last month’s down-time and this month’s down-time.

On the up-side, Whirlpool user Kevin JD is a Virgin representative who advised customers to send email to telesales@virginmobile.com.au with a contact phone number if they have any issues. It’s good to see a company engaging with it’s customers.

Update: An hour after emailing the URL to this post to Virgin I got a call from a service representative. It wasn’t a very productive call as I already knew that their servers are down and they can’t do anything. But it’s good to know that they are very enthusiastic about making things better. One useful thing that I learned is that my wife’s service is probably on a pro-rated bandwidth quota. As she got the phone in the middle of the month we can only do 100M of data transfer not 200M before the end of the month, by my rough calculations I’ve downloaded well over 50M of data (maybe as much as 100M) from the Android marketplace. So it might be necessary to negotiate about the bill as soon as their CRM system works.

On Burning Platforms

Nokia is in the news for it’s CEO announcing that they have a “Burning Platform”, Jeff Waugh gives an interesting analysis of the situation [1].

What is a Platform for Phones?

This whole scenario makes me wonder about what a platform should mean in terms of mobile devices. Wikipedia says “A computing platform is some sort of hardware architecture and software framework (including application frameworks) that allows software to run. Typical platforms include a computer’s architecture, operating system, programming languages and related user interface (run-time system libraries or graphical user interface). [2]

I think that Nokia’s problem was that it didn’t leverage it’s strength in making great phone hardware. Nokia used to have a commanding share of the market due to simply making better phones. They had phones that were upwardly compatible (the same UI and used the same chargers), their phones were always very solidly constructed, and they had some good design features such as having the battery comprise part of the case – which allowed a replacement battery to be physically larger for longer battery life. Even now they have hardware features which are better than the rest – such as the N8 which is widely regarded as having the best camera of any phone. Nokia also did some smart things like releasing their N800 series of phones with more open software than most companies used, they supported running arbitrary Linux applications.

While they kept the platform as the API interface on a phone OS and maintained several different OSs (with multiple versions in production at any time) they made a less desirable platform for developers and a lot more development work for themselves.

Nokia fragmented their own market. There is no technical reason why they couldn’t manufacture a phone that is capable of running a choice of their own OS as well as Android (or alternatively their own OS or the Windows Phone OS). There may be technical issues that prevent designing a phone to run either Android or Windows phone, but they could have got two out of three options available.

On top of a hardware phone platform we could have multiple software platforms, Android, Windows, Meego, etc. The users could then decide which one to use.

A Hardware Platform for Phones

I think that it would be ideal if a phone company offered phones with a choice of OS and allowed the user to change the OS. The N8 is a great phone and the hardware is really appealing, I would have definitely bought one for my wife if it ran Android. Also the resale value of phones can be improved if they can be re-purposed. If someone knew that people on ebay were paying good rates for an old phone because it could be re-programmed then they would be more likely to buy it. As phones are essentially free for most people due to telco deals, a phone that has a resale value of $200 is a lot more desirable than one with a resale value of $100 if all other things are equal.

I would like to see the mobile phone as a platform just as the IBM PC compatible was a platform. Imagine if IBM had sued Compaq into oblivion when they first cloned the PC, if that had happened I don’t think that I would be using an IBM PC (Thinkpad) right now. The early IBM PCs really weren’t great systems for home use, for pretty much anything you might want to do the early PCs didn’t compete well with CP/M systems and systems from Commodore and Apple. It was only when cheap clones flooded the market that the “PC” platform took off. Modern PCs can boot from disks that were created 20 years ago. At every time during the PC’s history there has been a choice of OSs available that work with the standard BIOS boot loader and expect certain essential hardware (such as a BIOS interface for video, keyboard, and storage). This choice increased the market allowing economies of scale in production. It also allowed competition between vendors which forced the margins to be low and resulted in PCs becoming cheap – over the last 20 years every aspect of a PC apart from Intel CPUs and Microsoft Software has steadily dropped in price.

I would like to see phones designed with a common boot loader, a common core set of CPU functions (I believe that with ARM CPUs a lot is optional), and common interfaces to the touch-screen and the GSM system. The boot loader would ideally have support for booting multiple OSs, in a typical end-user case the store would provide the phone configured to quietly only boot one of the available OSs – but it would be trivial to offer the user a choice of OS at boot time for advanced users. The process of installing a custom ROM image in a phone is regarded as dangerous at the moment and most people who would like to do such things are hesitant due to the risk of “bricking” their phone. If booting the last good OS was an option then more people would try new OSs. With the current situation I am a little hesitant to re-program my new phones (and I’m braver about such things than most people).

One of the many problems with the current situation is that the phone vendor has to officially support every phone OS. One example of the problems with this is the Sony Ericsson Xperia X10 which is a fine piece of hardware that has been saddled with an older version of Android for it’s entire life, Sony has stated that they won’t produce any more updates and thus the demand for this phone is a lot lower than it might be. I believe that this is why the Xperia X10 is significantly cheaper than any other phone with similar features. Sony is losing money because they have a closed hardware platform and don’t support the recent versions of Android!

Ways of Booting the OS

It shouldn’t be a technical challenge to have support for basic touch-screen IO and 3G Internet access in the boot loader of the phone. Then an OS such as Android could be loaded from the network before being booted without needing an older version of Android running. 80386 systems with a few megs of RAM used to be able to boot from a LAN, a modern phone has much more powerful hardware and a network connection that’s at least as good as 10base2 networking so it should be able to do the same.

Modern phones have micro-SD cards for local storage. There’s no reason why the OS couldn’t be loaded on micro-SD card. Fixing the OS on a phone that was “bricked” could be achieved by putting the micro-SD card in a PC and then copying the files across – just as I often fix PC OS problems by installing the hard drive in a working system.

Changing the OS of a phone should be easier than changing a PC between Ubuntu and Fedora (the Linux distributions that seem to have the most usage by less technical users).

Conclusion

If Sony had made their platform the hardware then they could have spent their effort on what they do well (making great phone hardware) and delegated almost all of the work related to creating the OS to other companies (Google, MS, telcos, etc).

If the entire industry moved to a hardware platform for phones then the result would be lower prices and some thin margins for manufacturers. This would be good for users and OK for manufacturers. The current situation however is quite bad for users who can never get phones to do exactly what they want and not so good for phone manufacturers such as Nokia (and everyone else who does business with Microsoft) who find themselves on the wrong side of market forces. The phone as a hardware platform shouldn’t be too bad for Microsoft either, they did OK on the PC hardware platform.

Sony Ericsson Xperia X10i – First Experience

I have just got a new Xperia X10i for my wife. It’s with Virgin who use the Optus network. Optus apparently have better coverage than Three (who we have been using for 6.5 years) and Virgin offers the best deals – they have good phones on low monthly rates and do nice things like selling most phones unlocked (the Xperia is one that comes unlocked).

I chose the phone for use in system administration work. A large screen should be good for reading email and running a ssh client. Hopefully I will be able to use the phone’s Wifi to provide mobile net access to a laptop for any problem that can’t be solved on a tiny screen. My wife doesn’t do much sysadmin work nowadays, but it’ll save some effort if we can both use the same model of phone. Also according to Amobil.no the Xperia X10 is the second best phone for camera-phone functionality [1], and my wife would really like to take some good pictures with her phone (she loves the LG U990 Viewty that she has right now).

Screen

One major reason for choosing this phone is that it has the biggest and highest resolution of any phone that doesn’t cost a huge amount of money. It does well in this regard, the screen is very good to look at. However it is apparently not particularly sensitive and we still have the protective cover on the screen which makes the swipe to unlock operation quite difficult. My wife generally likes to keep the protective plastic on such screens for a while to prevent damage, but she may take this off sooner than on other devices.

Sony Foolishness

One reason that this phone is cheaper than any other phone with similar specs is that it’s made by Sony. Sony won’t provide updates to Android 2.2 or better and they have their own stupid applications installed in an intrusive manner. The main screen of the phone has a large icon for the Sony Timescape program. If you choose a background image of a person’s face then the Timescape logo is in the ideal location to obscure the person’s eyes. Generally it’s not desired to have the person in your phone background look like Zorro. Fortunately you can delete icons from the home screen, you just have to press and hold the icon until a trash-can appears at the bottom of the screen and then drag it to the trash.

Phone Book

The core phone functionality is fairly important for a mobile phone. But it seems that every modern phone has some serious failings in this regard. The Xperia comes with a phone book that is reasonably good for managing contacts and partially coalesces multiple entries with the same name – which is convenient as I somehow managed to import 3 copies of each entry in my wife’s phone book. The down-side is that the multiple entries are still stored, so when it comes time to update someone’s phone number my wife will have to separate the entries and then delete all but one of them, adding probably a minute of effort for each one. With 60 phone book entries that’s probably an hour of wasted time during the course of her phone use.

They really should do a time based analysis of phone use and optimise the UI for efficiently performing common tasks. I think that an ideal design would start off by asking the user to rate the importance of phone functionality, camera, facebook, twitter, etc and then give a UI that matches the important tasks. A device that has 384M of RAM, 1G of built-in Flash storage, and an 8G micro-SD card (which Virgin provided with the phone) should be able to offer a selection of UIs.

Camera

The configuration for the camera allows some settings to be changed easily and others have to be changed through the “advanced” settings which requires one more button press and a few extra taps on the screen. One real stupidity is to have the phone light be in the advanced settings, so if the room happens to be a little darker than expected you suddenly need to go to a lot of extra effort to turn on the light.

Finally it would be really good to be able to take a quick picture. I would like to be able to be able to press a combination of buttons and have it immediately take a picture regardless of what had been running before. Not press the main menu button, the camera icon (which I made a link from the home screen), wait a second for it to load, and then press the shutter button. Ideally I would like to press a combination of hardware buttons to have it immediately take a picture even when the screen is locked. I don’t want to wait for the screen unlock process, sometimes the perfect shot is only available for a second.

Wifi

It doesn’t support ad-hoc networks. This is really annoying for me as I don’t own an access point – and I don’t plan to buy one just for a phone. My laptop (Thinkpad T61 with Intel Corporation PRO/Wireless 3945ABG [Golan]) doesn’t seem to support running as an access point, and my EeePC 701 with an Atheros AR5001 has the same problem, here is the error message:

# iwconfig wlan0 mode master essid x
Error for wireless request “Set Mode” (8B06) :
SET failed on device wlan0 ; Invalid argument.

I have a cheap Realtec USB Wifi device that claims to support being an access point but just fails when I try it. So presumably it doesn’t really support it but the driver is buggy.

It seems that I may need to put a custom Android installation on it to get ad-hoc working.

Also it’s apparently necessary to root the Xperia to get Wifi tethering to work for providing net access to a laptop. So I guess I’ll hacking it soon.

Internet Access

The Virgin web site couldn’t help me set up the Internet access because I ported a phone number from another Telco. I had to use the phone number supplied by Virgin to login to their web site and the site then wanted to “sent” (probably SMS) the settings to my phone, but of course it couldn’t SMS to that number as I’ve got the Three number active. Presumably they were expecting that customers would setup Internet access before porting their old number.

I phoned the Virgin tech support and used the settings they suggested, but it didn’t work. Then I took the phone to a Virgin store and it started working after they rebooted it with no changes to settings. Just like Windows used to be. :(

Email Access – Default App

The email setup defaults to port 143 IMAP access, so even when a mail server provides access on port 993 for IMAPS and on port 995 for POP3S probably most users will not use any encryption because their phone doesn’t default to it. Also one annoying feature is that when I gave it the email address “test@coker.com.au” it decided to use “test” as the user-name for IMAP by default. It seems that a very large portion of the active email accounts nowadays are on systems that handle multiple domains and thus require the domain name in authentication, and a large portion of the remainder accept domain names even when they aren’t required. Besides it’s a lot easier to delete a domain name when it’s not needed than to type it in when it is.

The default hostname for sending and receiving mail is the domain name, if the MUA configuration program had checked the MX record it would have known that smtp.sws.net.au is a better default choice for the mail server. I’ve now started the practice of creating a CNAME entry named “m” in every domain I run, so when someone is setting up a mobile phone they just have to add “m.” to the start of the default hostname to use for sending and receiving mail. By doing this I can probably save an average user at least 30 seconds of typing, which is particularly important if I happen to be talking them through the process!

The MUA uses IMAP commands to load the text of the message without attachments and then loads attachments on demand, this will be handy for when people attach unreasonably large files. With IMAP caching and partial loads the core functionality seems OK, and some quick tests of standard but not commonly used IMAP functions such as noticing that mail has been deleted by another client seemed to work well. The performance of loading a new mailbox with 369 message was quite poor, I’m not sure how much of this is due to the network connection and how much is due to the client implementation. This is a real concern for me as I typically have about 7,000 messages in my IMAP account and 3,000 unread messages most of which are in two folders.

Another problem is the fact that there seems to be no way of searching in the list of messages, sorting/grouping by subject or sender, or showing unread messages. This makes the Xperia unsuitable for the task of reading mailing lists when on public transport. I would really like to be able to sort through a folder of mailing-list mail, delete messages that I don’t need to keep and flag ones that need attention when I have the resources of a desktop system when I’m on a tram or in other situations where a laptop can be inconvenient.

One undeniable bug I discovered (as opposed to the missing features I listed) is that when a message has a line with a URL followed by many spaces it won’t load correctly in the web browser, the spaces will be treated as part of the URL. I’ve changed the default signature in my email to have a description followed by a URL to avoid triggering this bug. Presumably other Android phones are bug for bug compatible. :(

Moxier Mail

There is a complete set of email and calendaring applications preloaded from Moxier.com. The Moxier mail client uses company.com as an example domain name (they should use example.com – does no-one read the RFCs?). It’s nice that they verify the SSL certificate and display the details when it’s not signed.

Unfortunately Moxier doesn’t support anything other than Exchange, and I can’t delete it either as it’s part of the base set of applications from Sony.

SMS

The SMS application is supposedly “conversation” based. This doesn’t seem like a benefit to me, it’s a pity that there is no option to make it act more like every other SMS implementation I’ve used over the last 12 years.

Battery

The battery is described as having a capacity of 1500mAh, for comparison a AA rechargeable batteries have capacities ranging from 1800mAh to 2800mAh. My first mobile phone had a rechargeable battery that took the same space as 4*AA batteries and the phone was designed to run on AA batteries if the main battery wasn’t charged. I would like to have an Android phone that had a battery pack of similar size to at least 2*AA batteries, and 4*AA would be OK too.

The Xperia battery was reported as being half discharged after playing with the phone for less than an hour. Basically when doing anything with the phone that is remotely intensive I need to have it plugged in. Unless of course I’m using one of the star-chart applications…

Lights

There are two white leds in the gaps between the keys for Settings, Main-Menu, and Back on the front. They are quite bright and annoying to look into at night.

SSH

SSH was one of the reasons for getting an Android phone, but it was one of the last things I worked on because everything else was so much fun. I’m currently using ConnectBot, it supports storing ssh host keys, generating a key-pair from random screen touches, locking and unlocking the key-pair (with a pass-phrase) and pasting the public key to an open window. All the basic functions just work although it’s rather inconvenient to enter the CTRL keys.

The next thing to investigate is a small and light Bluetooth keyboard. Using an external hardware keyboard will save me from the pain of entering CTRL keys (which is even worse than the pain of entering passwords that contain digits and mixed case). It will also allow me to type a lot faster and use the entire screen for the ssh session.

Current Bug Summary

Wifi doesn’t do ad-hoc mode or tethering.

MUA doesn’t correctly handle URLs in messages if many spaces follow the URL and is also unsuitable for mailing list mail.

Sometimes requires a reboot to make Internet access settings take affect.

Camera light can’t be easily turned on and off.

This isn’t too bad.

Conclusion

Most things do what I want. Once I get a MUA working with all my email, Wifi doing what I want, and a Bluetooth keyboard it’ll be a fairly ideal mobile sysadmin terminal. I’ll probably order one for myself this weekend.

[1] http://www.amobil.no/artikler/norges_beste_kameramobil/79081

Mplayer, Squeeze, and SE Linux on i386

I’ve just updated my SE Linux repository for Squeeze to better support running mplayer on the i386 architecture, below is the APT sources.list line:

deb http://www.coker.com.au squeeze selinux

The first issue is a bug in the compilation of the SDL libraries which makes them request an executable stack (bug #613535). Recompiling the libraries on my system caused this bug to go away, so it must be some issue with the compilation process. I have previously summarised the execstack issue, but we haven’t solved this yet [1].

The next issue is the fact that the ffmpeg libraries require execmod access (see my previous post for the details of the execmod issue [2]. The execmod issue with ffmpeg is pretty much the same as it was when I first wrote about the issue in 2008 [3]

Finally the allow_execmem boolean needs to be set on i386 with the command “setsebool -P allow_execmem 1” to allow libGL the access it needs. This is an issue I haven’t been able to solve, I don’t know why libGL needs write and execute access to memory, I posted to the SE Linux list about this some time ago but didn’t get any good answers [4]. Any suggestions would be appreciated.

Archives

Categories

Tags

Queue Counting

Viewing the Queue

The Final Command

Are Laptops Any Good for Schools?

Playing Games

The Problem with the OLPC Project

Minimal Use of Computers

Current Success

Conclusion

What is a Platform for Phones?

A Hardware Platform for Phones

Ways of Booting the OS

Conclusion

Screen

Sony Foolishness

Phone Book

Camera

Wifi

Internet Access

Email Access – Default App

Moxier Mail

SMS

Battery

Lights

SSH

Current Bug Summary

Conclusion

Archives

Email and RSS