The main sources of information used when hiring someone are their CV, the interview, and references.
The CV is written by the applicant or sometimes for the applicant. Naturally it says only good things, if a CV notes no skill in a particular area then it may be used to exclude an employee from consideration. But the trend is towards including a reference to anything that you touch, so someone who lists DBA experience may merely have done a couple of CREATE TABLE operations.
The interview is a good test of people skills but is often of little value in assessing technical skills. The interviewer asks questions such as “do you know technology X” and the applicant says “I know that really well“. If the company is hiring another person with similar skills to current employees then they can have their current employees sit in on the interview and ask difficult technical questions, but for unknown reasons managers often don’t take that option and get no advice from their technical people. Also if the company is hiring someone with specialised skills (EG they are about to implement a new application and want to hire their first employee to work on it) then it may be impossible for them to assess the technical merit of answers. Probably the best use of the interview is to match answers with the CV, if the applicant doesn’t appear to know the contents of their own CV then they should be rejected.
The biggest problem with interviews is when the questions are all of the form “do you know X“. Someone who really knows it will say “yes” as will someone who doesn’t know enough to realise the limits of their knowledge – and such ignorant people vastly outnumber the skillful people. The real problem is that the people who are moderately skillful will lose out. If someone asks me about my MySQL skills I will tell them that I’m not really good at it. Sure I’ve run replicated servers with tens of thousands of users running 24*7, but that doesn’t mean I’m really good at it – probably most people who will claim to be great at MySQL without qualification would have less experience than me.
Reference checks rely on an unknown person saying good things about the applicant. For starters there is the issue of the number of references which may not be representative of their employment history – EG the applicant could use as a reference the one manager who didn’t sack them.
The next issue is that there is little incentive for the referee to be honest, most people are aware of instances where someone once worked for a friend and can rely on good references for the rest of their career. If a reference is inaccurate then there is no realistic opportunity for redress.
Finally every reference check that I am aware of (checks where I have been the referee or the applicant) has involved the applicant giving the phone number of the referee to the hiring manager! The phone could be owned by a friend or relative of the applicant, so logically a good reference that is based on trusting the applicant to supply the phone number only proves that the applicant is either good or really bad. To make a reference check prove something the recruiter would at a minimum have to phone the number listed in the white-pages for the corporation that used to employ the applicant, asks to speak to the manager of the relevant department, and then gets a reference. Calling a mobile phone number that is supplied by the applicant (which seems to be the standard practice) is essentially trusting the applicant – and trust is the root cause of most security problems!
Really most of this ends up as trusting the applicant to provide honest evidence that they are trustworthy and believing that the applicant’s technical knowledge is good enough to be correct when they say that their technical knowledge is good. It can fail spectacularly when someone isn’t trustworthy enough to provide honest evidence of their integrity or when someone doesn’t have the skills needed to know that their skills are lacking.
As an aside, even if the reference is given accurately and in good faith it may still be misinterpreted. The fact that telephone references are exclusively relied on exacerbates this problem. Ideally references would be in writing with some way of proving their authenticity (maybe using phone verification of the accuracy of the written document).
So how can we solve this? Some people believe that career based social networking software will solve the problems, but as usual I think that software doesn’t magically solve human problems. The first challenge when trying to use social networking to solve the problem is to find someone on your friends list who has relevant knowledge, this may be viable in a small industry (EG when someone from bank A applies for work as bank B in the same city). The next issue is that of false “friends“. I’m sure that I’m not the only person who has been pressured to add people as friends on social networking sites, the non-computer social interactions really don’t prepare people for saying “no you are not my friend” (apart from high-school I guess). With professional social networking sites there are further issues, if you are working on a client site and a manager demands that they be listed as one of your friends then what are you going to do?
So it seems to me that the social networking sites are at best a helper for the gossip network. If you think that a friend of a friend from a social networking site might be able to help you then you first ask your friend if the person in question is really a friend, and if so are they one of the shifty pseudo-friends you only hang out with because their company pays good money. But the problem with the gossip network is that it’s mostly secret and is therefore subject to settling vendettas, I’ve heard of senior managers going out of their way to spread false stories about former employees to settle scores.
The best solution I can think of is for someone who has a reputation to publicly stake it on the accuracy of their references. If I’m going to give a reference then I would be happy to do so via a GPG signed email or a blog post. This doesn’t mean that my references will always be correct, but it would show that I try to give good references.