Author Profile Picture

Paul Barrett

Cognadev UK & SA Ltd

Chief Research Scientist

Read more about Paul Barrett

Data-scraping social media for personality assessment: will this benefit HR?


Michal Kosinski, one of the leaders in personality assessment using social network activity linguistic analysis, has been making a series of interesting claims about the effectiveness of his approach.

He has said that “with a mere ten ‘likes’ as input, his model could appraise a person’s character better than an average co-worker. With seventy, it could ‘know’ a subject better than a friend; with 150 likes, better than their parents. With 300 likes, Kosinski’s machine could predict a subject’s behavior better than their partner. With even more likes it could exceed what a person thinks they know about themselves.”

This has big implications for recruiters and HR practitioners tasked with identifying talent for their organisations.

Personality assessment based upon Facebook and Twitter ‘digital footprints’ is now a reality, in use, and deployed in earnest by at least one organisation, Cambridge Analytica.

They are selling the profiles to politicians, marketeers, and commercial entities wishing to use them for profiling applications. It is now also possible to buy an off-the-shelf PredictionAPI service which an organisation can implement directly to ‘score’ social media activity.

By doing this, individuals can be categorised using the OCEAN (openness, conscientiousness, extraversion, agreeableness, neuroticism) framework for personality assessments.

These developments are interesting for HR practitioners because recent evidence indicates that automated assessment systems like this possess the capability to forecast a range of variables, in some cases to a greater degree than a brief self report personality questionnnaire which is commony used.

Potentially, these new tools offer huge benefits to large organisations, as they remove the need for HR and applicants to engage with personality questionnaire assessments. Candidates simply have to provide sufficient information that their digital footprint can be uniquely identified.

Once identified, the tool ‘scores’ the personality attributes for a candidate, runs these against relevant HR or even “Big Data Analytics”-created job role target profiles, and reports back to HR if a candidate’s scores pass the screening process.

Otherwise the system would simply say ‘thanks, but no bananas today’ to the candidate.

The advantages for HR in terms of potential efficiency gains and cost reduction are obvious.

Such a system provides a seamless and automated ‘pre-screening’ process, handling all candidates autonomously and only signalling to HR when individuals meet the biodata (education, experience, qualifications, age, gender, geographic area) and personality filters.

Although HR departments are already using various ATS systems (Applicant Tracking Systems), these still require significant human intervention whereas when using data scraped from social media data, virtually all the functionality can be automated and finding candidates for a job role can now be left to an autonomous system.

But are these tools sufficiently accurate to warrant widespread adoption?

According to newly published research by Kosinski et al, social media profiling has an equivalent level of accuracy to self-reported personality attributes.

Statistically, the validity coefficients used by personality test publishers are just slightly lower than the digitally-acquired personality assessments, suggesting they can exceed prediction of external criteria over and above self-report assessment.

However, since these correlations are relatively small (as are the validity coefficients for personality attributes being predictive of job performance), digitally acquired personality assessments are really only suited to pre-screening.

Even so, this is still significant for busy HR departments tasked with converting huge candidate lists into more manageable numbers. For instance, some airlines report having to process well in excess of 15,000 applications for cabin crew positions each month, so reducing that number to a more manageable quantity with less initial effort is potentially of significant financial and logistical benefit to an organisation.

Autonomous data scraping clearly has implications for some candidates with either no or very limited digital footprints, as these individuals may eventually become excluded from segments of the job market. In addition, when hiring for high-stakes employees, e.g. high autonomy leadership roles, far more detailed and powerful information about a candidate’s psychology than a few “self-report questionnaire-attribute” scores is required.

This is why test publishers who specialise in deep psychological assessments remain unaffected by the new Big Data drive to predict personality from people’s digital footprints.

Long term, these developments will most likely see a polarisation of the test market, with rough ‘n’ ready scraped data based assessments satisfying the high volume filtering requirements and smaller, specialist players filling the market niche for high-stakes, powerful and specialised assessments.

Cognadev specialises in the development of psychological assessment and development methodologies, to empower HR practitioners, recruiters, organisational psychologists and consultants to identify, understand and optimise talent.

Author Profile Picture
Paul Barrett

Chief Research Scientist

Read more from Paul Barrett