This article was co-authored by Martin Reddington and Elmira Bakhshalian, Founder and Research and Development Manager respectively at Martin Reddington Associates.
There is a burgeoning interest in the measurement of workforce engagement and performance – both from academic and practitioner perspectives. The old ways of doing an engagement survey every year or two to extract data to produce an engagement score are progressively giving way to approaches that rely on more sophisticated use of automation and analytics.
HR and OD professionals and operational managers are ‘wising up’ to these developments and expecting more insightful analysis.
One of the most exciting developments is natural language processing. Using complex algorithms these tools can automatically analyse free text responses to produce thematic, sentiment-weighted outputs. An example is shown below:
[CLICK TO ENLARGE]
Source: (Developing a New Employment Deal for Local Government, 2017)
This graph shows the analysis of approx 20,000 individual free text responses, in answer to three questions:
- What’s the best thing about working for your organisation?
- How do you work with your line manager to get something done?
- What’s the biggest tension you are faced with in the workplace?
The blue columns represent the frequency of detection of the different categories of comments, shown along the horizontal axis.
The vertical axis denotes the sentiment scale – the coloured dots identifying aggregate sentiment – positive (green), neutral (grey) and negative (red) – within each theme.
Examples of these different types of comments are set out below and we have shown the contrasting perceptions of highly engaged and highly disengaged groups:
Organisational Support:
Highly Engaged: “My line manager is very supportive and empowers me to make decisions and if I need help always supports me or pulls in the rest of the team.”
Highly Disengaged: “Lack of support when new procedures are implemented- managers seem to introduce new priorities without putting any support, training, and time in place for us. Managers constantly ‘reacting’ rather than planning.”
Creating Workable Solutions:
Highly Engaged: “Through conversational practice: talking through an issue and coming up with suggestions on both sides for potential solutions.
Highly Disengaged: “Discussions on how to solve problems are encouraged, but are not actually actioned or followed through by management, so services are rarely (if ever) improved.”
Lack of Resources
Highly Engaged: “We are trying to deliver more to our service users with less resources, both monetary and people.”
Highly Disengaged: “Number of staff has halved, more is expected of staff, no resources and more time spent on administration tasks rather than working with clients.”
Job Pressure:
Highly Engaged: “There is an energy and ambition across [the organisation], which is refreshing given the challenges we have had to overcome in the last decade, on top of the pressures of austerity.”
Highly Disengaged: “I am hugely frustrated by the pressures that are put on my team to provide a certain level of service, that result from being a hugely under-resourced department.”
Management:
Highly Engaged: “My managers take an active role in developing me personally and always takes suggestions on board.
Highly Disengaged: “Lack of immediate line management engagement in a balanced way – a way that takes seriously the competing demands of practitioner/ social work values/ principles on the one hand and managerialist “needs” on the other hand.”
Team:
Highly Engaged: “Being part of a successful team that meets profound current challenges, with a shared vision of future goals.”
Highly Disengaged: “Team is spread thinly over many centres so communication is patchy sometimes and it is hard to feel a sense of belonging.”
Until these new technologies came on the scene, it was usually a case of manually inspecting comments with a skilled eye to produce some broad themes. If large numbers of comments were involved, this was often a thankless task to be avoided. It is no coincidence that many so-called survey providers (whether internal or external) concentrated on statistical data collection and analysis, to avoid the potentially time-consuming and messy nature of free text or narrative analysis.
There is no excuse for these kinds of practices today. Furthermore, the free text responses add richness to other statistical information and create further insights.
So where is all this heading?
Well, as previously stated, the traditional survey is looking increasingly anachronistic – a ponderous ‘animal’ that is under constant and ever evolving threats to its survival.
The new approaches combine:
- High levels of rigour – by which we mean really good modelling underpinned by robust theory and not simply a long list of questions resembling a potpourri that has grown over the years by well meaning but often ill-informed suggestions.
- Highly engaging interface – use of mobile technology, for instance, that makes the experience of data collection easier and more rewarding for participants.
- Highly impactful reporting – sophisticated analysis that combines different types of data (quantitative and qualitative) in ways that really inform good quality decisions. A primary reliance on an engagement score is simply not good enough.
These features are only possible by using the best technology.
So our appeal to all decision makers looking to commission an investigation into workforce engagement and performance is to ask critical questions of any provider, whether internal or external, about their capabilities.
Do not view proof of ‘good practice’ simply in terms of a long list of clients or colourful reports with apparently vital scores in large boxes set out on the page.
Ask them about about the rigour of their model – what have they published in peer reviewed academic articles or reputable professional journals?
Ask about their technologies – just how do they analyse free text responses, for example, and how is this integrated with other data to create powerful insights you can act upon with confidence?
A technology-enabled future has the potential to revolutionise this field of enquiry – make sure you make the best of it.
One Response
Who and when did the idea
Who and when did the idea arrive that HR should manage employee engagement. This article has made the subject needlessly complex. Who are you addressing. There are very few HR leaders or organizations trained and positioned in large companies and empowered to make anything but the minimum changes, kind of like moving the deck chairs around thinking that will surely save the ship. Here’s what works every time, pays for itself in a few days, and HR is not involved, except they will receive quite a few arrows in the process, like every other group.
The CEO asks the employees a single question, the answers got to a third party reporting to the CEO, this the politics, culture, and silos are rendered mute. Acceptable suggestions are implemented within days, no committees, just executives making decisions and a person responsible for getting it done. This way employees see things changing while they can still submit initial or additional suggestions. Not convinced, in a single case this approach cut SG&A expenses by $300 million, capital by $200 million, inventory by $45 million and previously announced layoffs were reduced by 1800. The client’s break even on the cost of the program occurred in seven days, the benefit was 50 times the cost on a single year basis. Many sacred cows were killed or wounded, a corporate officer bully was sent home, many polices were changed or eliminated. All this from a single question from the CEO while the third party eliminated the typical blockers, politics, culture and the silos. It just can’t be more simple nor more effective than this. HR will have plenty to do, they just won’t be involved in the collecting or decision making, unless it’s an HR issue. So is the purpose of this site to improve employee engagement and all the good stuff that goes along with that? If so, you have to ask why thee are not a single case study describing the results in terms of improved business performance, not here or anywhere I can find. How can that be if EE is such a big deal, shouldn’t the results be foremost in every discussion? http://www.emgc.com