Hard, Soft or About Right?
Making judgements about people has been at the heart of management thinking for over 100 years. A landmark 1903 paper by Frederick Taylor focused upon how to identify “first class men” on the grounds that “a first class man can do between 2-4 times as much work as can be done by an average man”.
Since then a series of tools and techniques have been devised to help managers make judgements about their staff, and thereby improve individual and organisational performance. Some have sought to utilise new technology to improve these people systems and processes, with the result that sophisticated IT systems are now available to analyse data in order to drive promotion decisions, development initiatives and to inform leadership succession.
Too often however, users of talent or succession management systems forget that the most fundamental building block is the quality of judgement about the person – that is, the soundness of the assessments that are fed into the system.
The quality of an assessment can be measured in two ways: is it accurate and is it consistent? Most attempts at improving people decisions, such as introducing better psychometric tests and performance management training, tend to emphasise accuracy. However, consistency of measurement is equally important. If you can make the judgements about people in your organisation more consistent you will increase the likelihood of making them more accurate. This is because consistency (reliability) of measurement sets a ceiling on accuracy (validity) of measurement.
The table shown gives an illustration. It is possible that Daniel has seen a group of poorer leaders, Ravi has seen a large group of better leaders, and Eileen has seen a small group whose leadership skills are relatively well distributed. However, the balance of probability is that Daniel is too severe in his judgements, Ravi is too lenient, whereas Eileen is getting it about right. If this data was used to drive an organisation’s talent management system we would probably be overestimating the capability of some leaders and underestimating the capability of others. This will result in some poor senior appointments, and may lead to scarce development resources being invested in the wrong people.
In this situation (which is not atypical) the organisation would be wise to direct effort to ensure the three assessors’ judgements are more consistent with each other. Too often we see money wasted on the easier option – purchasing a software system to collate the data.
It is worth noting that although the above judgements may not be consistent with each other, all three may well be accurate in the relative rankings they have given to their people.
What simple things can an organisation do to tighten up their judgements?
- Look at the data you have on your people and assess whether there is a problem or not. Typically, a large proportion of the data will be too lenient, i.e. it will overestimate capability in some way.
- Help managers understand in more detail what good looks like. For example, behaviourally anchored rating scale (BARS) can help them make more consistent and appropriate judgements. Do this by building on the manager’s own experiences – good facilitators can do this in short workshops of around two hours.
- Ensure managers spend time discussing their people with each other, to encourage calibration.
- Give managers a common language to make judgements about the individuals in the organisation. This probably means a competency set that is consistently used, well understood and related to the day-to-day work of the organisation.
- Take a lesson from manufacturing – use real data to continuously improve the consistency of the judgements year on year.
In summary, if you want to make a significant improvement in the accuracy of your people decisions, start by trying to make them more consistent.