Wickland Westcott

Cross-Cultural Assessment

The benefits of using competencies to define expectations,measure capability and develop individuals are widely accepted and understood. While some fine details continue to be debated, most practitioners recognise the advantages of the simplicity and flexibility of using a competency-based approach to underpin their HR strategy. As a result, they are now more concerned with practical questions, such as how to optimise the use of their competencies and how to keep them up to date and relevant.

The past 10 years have seen a significant shift in the way in which many businesses organise themselves across international borders. Fuelled by recent improvements in the speed of communication and ease of travel, more and more organisations are looking to harmonise their operations across countries within international 'regions' or, indeed, globally. This has created a need for a human resource system that can be used across different cultures, leading to the development of generic competency frameworks.

Practical examples of the growing international use of competencies include the development of global recruitment processes and the growth of international fast-track development programmes in place of existing national graduate programmes.

Increasing numbers of employers have launched international development centres and 360-degree processes where managers from a diverse range of countries are benchmarked against a single competency-based definition of "what good performance looks like".

These applications often represent good practical uses of competencies based on aligning their requirements with organisations' objectives. However, few are being designed with a careful consideration of the impact of the value-laden nature of competencies on the validity of the data gathered.

As organisations struggle to realise the benefits of diversity and look to promote and celebrate local differences, this gives rise to two key questions:

• To what extent is it possible to define a single set of competencies for use across international borders?; and
• Can people from different cultures be reliably and accurately assessed against them?

To answer these important questions, Wickland Westcott recently concluded a research programme in partnership with global organisations such as Marconi, Nestle© and UBF Foodsolutions, the foodservice division of Unilever plc, supported by researchers from UMIST and the University of Sheffield's Institute of Work Psychology.

This work sought to examine the reliability and consistency of the use of competencies in cross-cultural assessment activities, and identify practical recommendations for maximising the validity of such initiatives in the future. A summary of the key findings and recommendations is provided below.

KEY RESEARCH FINDINGS

Our research project identified three prime issues relating to the cross-cultural use of an organisation's competencies.

First, clear links were found between cultural background and people's perceptions of what good performance looks like in terms of interpersonal and social competencies, for example, the commonly found competencies of "leadership" and "teamworking".

Second, a connection was also identified between cultural background and the demonstration of other less socially driven competencies, such as "creativity" and "proactivity".

And third, the research found that, while managers from Germany, the UK, Italy and the USA generally showed agreement about what constitutes effective behaviour, there were also some clear areas of disagreement. Although it should, therefore, be possible to design competencies that have cross-cultural validity, subtle important differences of perception concerning individual behaviours frequently mean that direct comparisons between candidates from specific countries are subject to systematic bias.

WHAT DO THE FINDINGS MEAN?

The findings from our studies suggest that when using the assessment centre technique, systematic differences should be expected between the behaviour of people from different countries, as they strive to meet different mental models of what good performance looks like.

Building on this understanding and the specific details behind these findings, we then tested a variety of approaches to control the impact of cultural differences in a third study. From this study, we were able to identify a series of five practical steps (see box 1) that together form a robust method for the reliable assessment of people from different cultures against competencies.

PRACTICAL RECOMMENDATIONS

Step one

The first step in the accurate cross-cultural assessment of competencies is not to assume that any competency is universally applicable just because a company has included it in its corporate competency framework.

Designers and users of competency-based assessment processes need to define precisely how and why each competency is relevant in order to avoid significant misinterpretations of the meaning of assessments made, either by those being assessed or the people making the assessment.

It is essential to develop a thorough understanding of the demands of the target jobs, and an appreciation of how the competencies are supported or conflict with candidates' own cultural values. This understanding should be developed in consultation with local line managers and HR professionals, and informed by academic research into cultural differences, such as the work of Geert Hofstede and Fons Trompenaars.

It is critical for those involved in designing assessment processes and interpreting competency-based feedback to appreciate exactly how the behaviours are required so that they can answer any challenges concerning the validity of the assessment.
This raises an interesting practical consideration for those organisations that define competencies at a relatively high level and allow individual countries to interpret and apply the competencies in a locally sensitive way. This approach works well when the competencies are being applied within individual countries. But when attempts are then made to assess people from multiple countries or cultures (for example, as part of an international development programme) agreement must be reached concerning what each competency means and why it is important before any assessments
are undertaken.

Step two

The second step is to ensure that recognised best-practice design principles are followed when developing an assessment centre. This includes tailoring the content of exercises to maintain face validity. In other words, the exercise must be culturally acceptable as valid and legitimate by the participants, regardless of their cultural background. Content validity must also be maintained by ensuring that candidates are set realistic case studies and activities. Critically, the issues that candidates are asked to deal with must genuinely reflect the full nature of the target job.

Clear instructions about each exercise must be prepared, and the centre should be designed to include opportunities to check participants' understanding of what they are expected to do.

Instead of leaving the centre's assessors to make the connection between individual exercises and the relevant competencies from the organisation's framework, clear links must be made to ensure that there are no cultural misunderstandings between assessors.

And thorough training must be given to all those who will be involved in the collection and analysis of assessment data. In terms of assessment centres, this includes the assessors themselves and any additional personnel who act as interviewers or observers. In other situations, such as mainstream selection interviewing, all the interviewers must be trained, as must the facilitators who are involved in assessment via 360-degree processes.

Step three

Our third recommendation is that psychometric personality measures add particular value to international and cross-cultural assessments.

Most assessment and development centres include psychometric tests to good effect. But when running international assessments, they add additional value by providing standardised benchmarks that can be used to double check the validity of data gathered about individual competencies. It is essential that personality questionnaires are used in the candidate's first language, and that only questionnaires that have been developed using local norm groups are employed.

We have found personality measures to be particularly useful when reviewing data that is the subject of some debate concerning its underlying cause or relevance within specific cultural situations.

For example, if reviewing development centre data from energetic and enthusiastic participants who come from relatively open, expressive cultures, it is valuable to understand how typical their approach is when compared with their own local norm group, as well as against fixed competency profiles. This is especially important when reviewing poor performance against competency frameworks, and can enable assessors to understand the likely impact of development areas within participant's own and other cultures. This will help them to create culturally specific development plans.

Step four

When selecting assessors and facilitators, it is essential that they are representative of the participants going through the process. This does not mean that participants should only be assessed by people from their own or similar cultures. Rather, there are often significant benefits from getting the perspective of assessors from a range of different cultures. However, assessment centres should include at least one assessor who has a detailed understanding of the culture and typical business practice within each of the participants' countries.

Drawing on the assessors' familiarity with the underlying values and mindsets of different participants increases the quality of interpretations by helping to identify those behaviours that may indicate development areas that are culturally driven and those that are not. This level of analysis can prove particularly valuable when looking to identify the best way of developing high potential employees, either before or after they are moved between countries.

Step five

The final step is about communication, in particular preparing participants both for the nature of the assessment method and the expectations that organisations have of them in terms of the competencies assessed.

The accuracy of even the most culturally sensitive assessment processes will be undermined if those being assessed have had little exposure to the type of activities they are to undertake. One of the unexpected findings of our research was the uncovering of clear evidence of the influence of familiarity with the process (or practice benefits) on levels of participant confidence and performance. People's exposure to different assessment methods differs even more widely internationally than it does nationally.

Practice leaflets (along the lines of those advocated for preparing candidates for psychometric tests), along with comprehensive briefings, represent the minimum standard of assistance that organisations should provide. Beyond that, practice 'events' should be considered, where participants have the opportunity to explore the competencies within example exercises and activities. These events can add particular value where people with vastly differing levels of familiarity with
assessment processes are to be compared with each other.

TANGIBLE BENEFITS

Early reactions to this research have been positive. Where Wickland Westcott's five steps have been used to modify existing
assessment processes, the results show an increase in client and candidate satisfaction. They have also resulted in the identification of clearer, more precise and "hard-edged" evaluations in situations where assessors had previously felt uncomfortable. Additionally, to accurately assess applicants/participants working in the UK but who originate from outside it, we are increasingly applying these principles in the design of assessment processes for use primarily within the UK itself.

The five steps identified in this article present a useful starting point, but further work will be required to continue to develop our ability to strike the right balance when assessing people within cross-cultural situations. 

Share This Thought Leadership Article

Recipient Name

Recipient Email Address

Your Name

Message

Verification Code

Enter the letters as they are shown in the image