No Image Available

Does psychometric testing work? By Sarah Fletcher

pp_default1

Sarah Fletcher looks at whether psychometric testing improves the efficiency and quality of the recruitment process and how to implement it within a company. handshake


The purpose of psychometrics

There has been a long history of debate over the validity of psychometric testing. Royal Mail has fuelled this by announcing that the 125,000 applications it receives each year will be vetted using psychometric testing. The purpose is to identify the most unsuitable applicants, reducing the volume of candidates to a manageable size to take through to the next stage of recruitment. Richard Alberg, Chief Executive of the organisation selected to deliver the programme PSL, says the strategy “is more about what we’re trying to avoid than what we’re looking for; it’s who should definitely be eliminated from consideration.”

“It is more about what we’re trying to avoid than what we’re looking for; it’s who should definitely be eliminated from consideration.”

Richard Alberg, Chief Executive, PSL

Psychometric tests assess ability or personality to determine which candidates will be most suited to the role; and the importance of each is weighted according to criteria set by the recruiting company. For example, Alberg says that an employee working in the sorting office would need a calm temperament and be comfortable with doing repetitive work for long periods of time, whereas those hired to deal with the public require stronger communication and problem solving skills. Obviously the employee will need to work well with other staff so the psychometric test should also assess whether the applicant will fit into the existing team.

How to implement and evaluate psychometric testing

As it is important to hire individuals that suit the company culture, an organisation using psychometric testing should assess its criteria each time it recruits to keep the balance of personalities and skills it needs. Managing Consultant and psychometric software provider Peter Duckitt notes: “Benchmarks should be checked every time they are used … It’s not unusual for organisations to change the benchmark according to the specific need of a particular assignment. For instance, if you have recruited six people into a department who are all introvert, you might decide that the next one should be an extravert to maintain some sort of team balance.”

The benchmark used by Royal Mail reflects this concern. The company makes sure its team contains a mix of genders and cultural groups by deselecting only the worst performing thirty to forty five per cent of applicants. Alberg identifies these as “conservative cut-off scores” as this figure is chosen to ensure that the candidates that go through to the next recruitment stage have different skills and are from a variety of backgrounds. He explains that this is because a certain gender or group will perform best on a particular ability test, so if only the highest performing ten per cent pass to the next stage, the employees hired will provide only a very narrow skills set.

As it is also important to recruit candidates of a certain ability, Duckitt explains how to set the criteria accordingly: “If, say, working with numbers is not important to the position, then the sten range [a distribution scale of one to ten commonly used in psychometric testing] could be painted very wide (say sten two to sten eight for example) so that most people would fit into the range – but if working with numbers is important then the sten range would be made much narrower, say sten seven to nine, so far fewer people would score at that level. A sten range is created for each of the scales in the benchmark.

“The fit is measured as a percentage score. Provided you have a reasonable benchmark, the evidence is that an 80 per cent fit or greater will be good for the role. We hardly ever find a 100 per cent fit across 20 different parameters.”

As the job specification or qualities required by the company can alter over time, this data should be regularly reviewed. Richard Marsh, Managing Director of HR Management assessment provider Profiles International, suggests that the criteria supplied by the psychometric software is valid “for at least two to five years” but “when forming new job patterns we recommend a minimum of three to six months.” Alberg recommends a shorter review period of every two months whilst the project is being rolled out, extending to an annual assessment when it is established.

Does it work?

Can the candidate cheat? Marsh claims that the depth and extensiveness of the psychometric test secures it against candidates seeking to misrepresent themselves: “Because we are talking about an assessment of over 330 questions in six parts where the candidate needs to be consistent, the Distortion Scale [which charts the applicant’s information on a graph] quickly picks up on “bad data” [information which is inconsistent with the general pattern of information the candidate has given] and the client is coached on what to do in these instances.” Kevin Kerrigan, Managing Director of SHL, recommends the interview as a way of “authenticating that the individual could have produced that result” in the psychometric test.

This suggests that the psychometric test is not a foolproof or altogether objective method of assessment. If the test requires someone to interpret the results, they are vulnerable to the human error they were designed to eliminate. The risk, Marsh says, is that “one person’s liveliness is another’s distractibility”, and this distinction can only be made by interviewing the candidate.

Many psychometric providers seem aware of this problem, offering solutions ranging from personal consultants to written guidelines on how to interpret the results. Does this suggest insecurity that interviewers are not able or perceptive enough to read an applicant’s character? “Not at all,” says Marsh. “That is part of the challenge. Some people are more perceptive than others and will pick up on different (real or perceived) traits.” To help reduce the impact of human error, it is common practice to use a “placement report” “for the interviewer to use as a guide for their own questions”, in order to reduce “the effect of poor decision making down the line.”

Whilst a psychometric software provider can offer consultations and post-test support to better equip the recruitment process, this is only part of the solution. Kerrigan points out that psychometric tests help to reduce the margin of human error so that the interview process will be more successful: “A candidate shortlist of higher quality and better fit will improve the quality of the interview”. He admits, however, that this isn’t necessarily enough as SHL advise that all hiring managers are trained in competency-based interviewing techniques. It seems, then, that psychometric testing can only work as part of a wider recruitment strategy.

A number of providers acknowledge that psychometric testing does not provide a complete picture. Nigel Evans, Principal Occupational Psychologist of psychometric software provider Consulting Tools Ltd, admits that it can only “predict” good performance on the basis of “probability”. It is, he says, “never an exact science”. This is echoed by Duckitt: “Psychometrics should always be used in conjunction with other processes such as interviews and screening application forms. They are statistical tools and have a margin of error, therefore no psychometric measure should be used on its own for selection.”

“Psychometrics should always be used in conjunction with other processes such as interviews and screening application forms. They are statistical tools and have a margin of error, therefore no psychometric measure should be used on its own for selection.”

Nigel Evans, Principal Occupational Psychologist, Consulting Tools

Implicitly, the interview remains the backbone of employment strategies: he says the interview is necessary in cases in which CV and psychometric test provide noticeably different conclusions; applied to determine why a candidate may have a desirable career history but their psychometric profile challenges this apparent fit. This throws up a crucial issue for the future of psychometric tools: if recruiters are almost falling back upon the interview as the only really reliable source of information about the candidate, what’s the point in alternative assessment methods?

Conclusion

As psychometric testing is not entirely reliable, it should only form part of the recruitment process. It cannot replace the interview, but as the Royal Mail case study shows, it is a useful way of streamlining applicants to ease the burden of interviewing hundreds of thousands of candidates.

BPS psychologist Colin Parker explains why the company cannot afford to make a mistake. He looks at it in terms of return on investment: “In my mind the argument for the use of psychometrics is quite simple. Say you are recruiting someone on a salary of say £20,000 and they stay, on average, for five years. Then you are saying that on average the person is worth £25,000 and to be of benefit to the company they must contribute more than £25,000 worth of value.

“I am sure that you have worked in organisations where some of your colleagues are worth much more than their salary while others are not up to scratch. It is fairly easy to get rid of poor performers but much more difficult and expensive to get rid of someone who is slightly below average. Taking my example above, one person could easily be worth £30,000 and another £20,000; over five years that is a difference of £50,000.” If psychometric testing can identify these poor performers in the initial stages of the recruitment process, it effectively generates a year-on-year return for the company. Although the interview will always be a subjective process, psychometrics can ensure the least suitable applicants have already been removed. Taken as a part of the overall recruitment process, then, its contribution is valuable.

2 Responses

  1. Does Psychometric Testing work? – It all depends!
    As an Occupational Psychologist, I am concerned about the continued misunderstanding and misuse of ‘psychometric’ tests. Used properly and selectively they can be a significant improvement on traditional selection techniques.

    The purpose of using effective psychometric instrumentation is to improve the management of people through better:-

    •Selection of the right people,
    •Retention of valuable staff,
    •Development of their capabilities.

    How do we achieve this?
    We can achieve this by improving on the reliability and validity of the assessment of an individual’s attributes either for selection or development by the use of well designed and validated psychometric assessment techniques.

    How good are the various techniques?
    Worldwide research to evaluate the effectiveness of different assessment techniques shows their effectiveness to vary widely. The table below summarises these differences and the graph illustrates the impact of different levels of selection validity on the prediction of job performance.

    Method Validity ‘r’( -1.0 to +1.0)

    1. Work samples 0.55 – 0.65
    2. Cognitive Ability tests 0.50 – 0.65
    3. Assessment Centres 0.35 – 0.70
    4. Personality tests 0.40
    5. Bio-data 0.4
    6. Structured Interviews 0.30 – 0.6
    7. Unstructured Interviews 0.15 – 0.2
    8. Graphology 0.1
    9. Astrology -0.1
    (Yes! Believe it or not there are a number of major city firms that use things like graphology.)
    All of this presupposes a number of things:-
    •That the users are adequately trained in the administration.
    •That there is a clear analysis of the job into specific competencies and attributes,
    •That the tests actually measure these attributes,
    •That the tests have adequate reliability and validity data to justify their use.
    In my experience, many of these conditions are not met, particularly where so called ‘personality’ tests are wrongly used as a formal part of the selection process.
    I also find that many senior managers resist the proposition of needing training in interview techniques and still hold to their cherished belief that they “can judge the candidate in the first couple of minutes”.
    Used appropriately there are great benefits to be gained in accuracy and cost in the selection and development process.
    Bernard J M Stewart,
    Consultant Occupational Psychologist
    Integrated Performance Development

  2. Testing is a form of generating ROI
    If the testing process can provide the level of ROI on human capital that is assumed within this article then why aren’t more organisations using testing for recruitment?

No Image Available
Newsletter

Get the latest from HRZone

Subscribe to expert insights on how to create a better workplace for both your business and its people.

 

Thank you.