Where is your organisation on the HR data science pyramid? And what can you gain by climbing higher?

A few years ago, I was in a project meeting with the head of HR at a global corporation. He said to me: “The reason we’re doing this project is that I don’t know how many people we employ, and I don’t know how much they earn.”

If that sounds familiar, you’re far from alone. This situation is surprisingly common in large organisations, many of which struggle with their use of data.

At the other end of the scale are organisations like HSBC, with which we’ve been working for several years. HSBC wants its HR operation to become truly data-driven, with analytics embedded as part of its everyday business operations.

Achieving this aspiration will enable line managers to call up key HR insights using natural, everyday language. It wants to gain the best possible information and AI-driven predictions to make better decisions.

Use cases in HR that add serious value

The results of this kind of work would help you answer many business-critical HR questions better. Questions such as:

• What percentage of my people are likely to leave in the next 12 months?
• What are the key reasons for leaving given by employees in exit interviews?
• What are the key features of the CVs of our top performers? 

It also allows organisations to answer much more robustly and effectively the question of ‘What works?’. Here are a few examples:

• Pay reviews. Do pay rises work to improve performance or retention? Which types of employees respond best to monetary rewards? Which responds better to something else? 
• Retention. How do you spot employees or categories of employees at high risk of going, leaving you with tricky gaps to fill? What indicators best identify a risk of leaving? Machine learning can bring a whole new dimension to this by finding indicators in text data (say employee survey responses) and structured numeric data (such as a satisfaction score).
• Recruitment. What does a candidate likely to perform very well in the long term look like? Of course, this too will need human input: an HR manager or line manager will have to give feedback after six months or a year on whether the hire was a success or not.
• Learning and Development. If an employee is struggling, what does the evidence suggest is most likely to bring about better performance? 
• Costs of rehire. Predicting the likely financial costs of losing a high-value employee. These are often much greater than might meet the eye. As well as recruitment expenses, there are costs that come with training and productivity loss as new people bed in.  

HR Data Science Pyramid

These examples show the power of reaching the top of a 4 -tiered structure that we call the HR Data Science Pyramid. Getting to the peak of the pyramid can be transformative. At the level of employee retention alone, we know it costs more than £30,000 to replace one person who leaves. There is huge value for early adopters here.

Level 1: Working towards a data foundation

HR operating at this level do not collect or collate data in a systematic way. They don’t use analytics and make the vast majority of business decisions using human intuition, hunches and gut feel.

Even if HR databases are used, the lack of systematic data processes can result in disparate business units structuring their data in different ways or using different terminology to describe the same thing. This can be a common problem even in larger organisations, where mergers and acquisitions may have left a firm with a legacy of different and incompatible HR systems.

As a result, it may not be possible to get reliable numbers for queries like headcount and payroll costs. Getting these numbers might be tricky, involving various inputs for disparate business units. There is often a significant lag-time in the data being reported. 

Level 2: Data foundation

Having a data foundation means having standardised, good quality HR data across the organisation – even if collated in relatively basic tools such as spreadsheets.

Data will be part of most decision-making processes but will be overridden in favour of human intuition if there is insufficient quantity, quality, and historical depth of data to allow reliable assessments based on evidence.

The critical component of quality data is that it is collected systematically, so it is up-to-date and comprehensive. The same fields and terminology are used for collecting the data for all employees, so you don’t categorise two people with essentially the same job differently.

For a rock-solid data foundation, organisations should collate all HR data from across the organisation in a coherent database or system that allows it to be viewed and analysed holistically. At this level, organisations will probably use basic analytical approaches that present data through a series of reports and dashboards.

Level 3: Exploratory data science 

Once organisations achieve this next level on Peak Indicators’ pyramid, they can start to produce all sorts of interesting and valuable insights and information, and data can become a core part of most decision-making processes.

Data collection and analysis becomes significantly more robust and efficient at this level, with dedicated departmental databases and specialist HR applications collating data systematically and automatically, 

We also start to see some limited use of algorithms alongside reports and dashboards, allowing HR organisations to begin profiling the data to answer questions about particular segments of their workforce, such as women, graduates, call-centre operators or junior managers. 

This level enables HR leaders to analyse their data in beneficial ways and start to answer questions like: “In the past three years, have the graduates appointed to this type of position been more or less likely to leave inside a year than non-graduates?”. 

Organisations engaged in exploratory data science can also start to integrate structured data (for example, gender or remuneration or frequency of promotion) with unstructured information such as staff sentiment surveys or exit interviews, and organise these into themes.

This allows them to get a rich picture of sentiments and the background to trends that analysis shows in structured data. It becomes possible to answer questions like: ‘What were the key causes of dissatisfaction among my best performing programmers who left the firm in the last year?’. 

Level 4: Data-driven

A data-driven organisation routinely uses high-quality data insights to make the best decisions across the board. Data from different departments and disparate sources is brought together in a single data architecture, allowing deeper and richer insights to be obtained. 

We are now into the realms of using machine learning and AI to help everyone in the organisation answer questions with ease and produce valuable predictions and insights. Here we see advanced analytics capable of predicting future outcomes and prescribing the most appropriate course of action across multiple processes.

At this level, data analysis to inform decision making is embedded in the culture of the organisation. Decisions from the boardroom to the shop floor are backed up with data insight. Insights generation is automated end-to-end, allowing ease of use. 

HR analysis can be obtained on a self-service basis by everyone in the organisation, including line managers, in real-time or close to it. That might look like line managers seeing their team with an indication of how likely that person is to leave beside them – or getting regular updates on team sentiment and the potential impact of that sentiment, allowing them to take effective and decisive action.