There’s so much rich insight coming out of the academic sector that HR professionals need to know. At Academics’ Corner we feature the best HR researchers that tell you what they’ve found and what you need to do differently on the back of the research. Get connected to the academic sector through Academics’ Corner and make sure you never miss another piece of key research again. If you’re an academic with a relevant story, please get in touch on academicscorner@hrzone.com.

This piece was co-written by Emma Donaldson-Feilder, Director, Affinity Health at Work.

In the last few years, the evidence-based management and practice movement has gained a large following with high profile researchers such as Rob Briner spearheading the associated models as a way to improve practice within the HR and occcupational psychology professions. In this piece we seek to suggest, as both academics and practitioners, that the prevailing models have limitations and that amendments are needed to ensure that there is two way collaboration between academics and practitioners

We believe that only through this amended approach can the HR agenda really be pushed forward.

The prevailing evidence-based practice model

The evidence-based practice model developed by Rob Briner and colleagues in 2009, which defines evidence-based management or evidence-based practice as a relatively new approach that is ‘about making decisions through the conscientious, explicit and judicious use of four sources of information’ (Briner et al 2009, p19).

The four sources of evidence the model suggests be used are:

1. Practitioners’ expertise and judgement – this could include asking managers about their own experience, assumptions and methodologies.
2. Context, local evidence, circumstances – this could include existing data held within the organisation or could be research/evidence collected in order to address the question/problem.
3. Evaluated external evidence – this could be published academic or practitioner research and data or case studies from similar problems in similar organisations.
4. Stakeholders’ preferences or values – the stakeholders will change depending on the problem/question being faced, but are likely to be experts and employers who have insights and evidence that might not be available through any of the other three sources of evidence.

As well as providing a more integrated approach, and a more structured and informed way to address the issues at hand, this evidence-based model attempts to overcome some of the limitations inherent in adopting a purely academic or purely practice-led methodology.

For example, academic research, by its nature, is designed to address very specific research questions in a way that is controlled, replicable and publishable. While this is desirable to achieve scientific rigour, it may limit the degree to which the research findings can be applied to real-world situations. Practitioner approaches, on the other hand, can be too focused on the particular circumstances of the situation and not build on the body of knowledge that exists in the field.

By drawing on the range of types of evidence, this evidence-based practice model enables users to take a broader perspective and bring together all the different elements that could contribute to a useful outcome.

In order to collect all these types of evidence, bring them together in a coherent way and produce solutions for both practitioners and organisations, the model suggests a methodology that involves the following steps: 

1. establishing clear questions to address;
2. data collection;
3. applying inclusion and exclusion criteria
4. data analysis
5. development and testing of outputs. 

The recommendation is that, from the research question or issue, one set of exclusion criteria should be developed in order to provide a quality check to all data gathered and develop solutions based on only the best possible evidence.

Not all forms of evidence are equal

While we applaud the model’s aims and principles, we feel that, in its current form, the approach does not function as intended when applied practically. Anecdotal evidence suggests that often outputs from evidence-based practice interventions include only academic data in the solutions. This is because only the academic data (rather than practitioner or stakeholder data) satisfies the quality criteria. As a result the approach becomes a production of systematic literature reviews in response to a question. 

Proponents of the approach have suggested that, once the data is analysed, the outputs can then be shared with practitioners and stakeholders in order to discuss feasibility and add a practical and contextual element.

However, this does not address the core issue of practitioner data having been excluded from the analysis itself. Our feeling is that, although the model in its conception suggests that a broader approach to a solution is necessary and beneficial (an assertion that we wholeheartedly support), the current methodology delivers an approach in which academic data is the only data judged to be of sufficiently high quality to be included within the analysis – and that therefore not all forms of evidence are equal.

It is true, as stated in the previous section, that academic literature is more controlled and replicable. It is this that makes academic literature publishable. However, it is this that also makes academic literature narrow. Real organisational problems are messy, with a plethora of contextual elements and considerations. Many issues and interventions in real organisational contexts cannot be studied in an ‘academic’ way as they cannot be controlled or conditions replicated. This means that academic literature does not have all the answers – and in fact does not address the same questions or issues as practitioner literature.

To develop really useful outcomes, practitioner evidence needs to be included in the process – which means that the exclusion criteria used need to allow for this.

An example…

During our recent research, which used an evidence-based practice approach (and won BPS Practitioner of the Year 2014), we were struck by how different the focus of academic literature was from that in practitioner literature/evidence. For instance, in seeking to understand how to develop positive manager behaviour, academic literature had focused almost entirely upon the impact of individual differences in the manager participants (for instance leader identity, learning orientation); by contrast, practitioner literature focused almost entirely on the management development methodology itself (how to develop the intervention and what should be included); and stakeholder input from organisations focused on the importance of the organisational context in which the managers worked. 

Due to the nature of the practitioner literature and the stakeholder views, there were differences in terms of ‘replicability and controllability’ of these three data sources: and academic literature tended to be of ‘higher quality’ as it was more replicable and controlled. So if we had chosen a single set of exclusion criteria and applied them to all the sources of data, we would have been left only with academic data.

This would have suggested that, in order to develop managers, we only need to focus on the managers themselves – rather than also the intervention and the organisational context. 

Clearly, in this situation, the other sources of evidence provided vital additional information and perspectives. We would argue that because the different sources of data approach the relevant issues in different ways, it is only by appreciating all the sources that we can develop a real, practical and useable solution.

A new way of collaborating and conducting research

Given all of the above, we suggest amending the evidence based practice model in order to produce practical and useful solutions for all parties. Our key suggestions for anyone wanting to take an evidence based approach would be to:

• Take a broader perspective on the utility of varied data sources – moving beyond an academic view of ‘quality’ to encompass both utility and practicality
• Develop exclusion criteria for each source of data that ensure that the best evidence is collected within each domain. For instance, for stakeholder views – which were largely opinion based – we chose to include themes only if they were mentioned by 2 or more independent parties from 2 or more organisations
• Ensure that in outputs, data sources are clear and transparent, to allow traceability

By taking a broader, more inclusive approach to evidence based practice, where both academic and practitioner inputs are recognised, we believe the HR agenda can be pushed forward. This approach will ensure that solutions for practitioners are evidence based, whilst also being more practical and usable. It will also ensure that academics are challenged to broaden their perspective on both their approach to analysis and the type of questions and issues that need to be answered within organisations.