This post was co-authored by Sophia Zhao and Kelly Hannum.
360° assessment is widely used by organizations in the field of leadership development. It is also growing fast in China. The advantage of such a data collecting method is that a person can receive a holistic picture of his own behavior and performance, instead of information from a single source. To better understand how this method may, or may not, work in China, we reviewed relevant research. There are many factors that may potentially impact the accuracy of 360° assessments, but we focused specifically on how cultural factors impact 360° assessments in China.
Overall the 360° assessment method works well in China, but we also found evidence that culture has an impact on 360° ratings. The main areas to pay attention to include:
Self-rating Bias. Inflated self-ratings are more common in individualist cultures where personal achievement is emphasized. Consistent with the emphasis on modesty, people in collectivist cultures, like China, are more likely to rate themselves relatively low. The low self-rating is called the modesty effect.
Other-rating Bias. In a collectivist culture like China, relationship and harmony is emphasized; individuals are concerned about preserving others’ face. Hence, raters are more likely to avoid giving negative feedback, and their judgment is likely to focus on the positive traits instead of the negative ones. In a research paper published in the Journal of Applied Psychology, the authors found that individuals who hold collectivist values are more likely to provide more favorable ratings (creating something called a leniency effect) and also to provide ratings that do not differentiate the various dimensions of ratees’ behaviors (creating something called a halo effect).
Moreover, in a culture that emphasizes power distance, the degree of leniency and halo effects also varies depending on the rater’s hierarchical position. In a high power distance culture like China, subordinates may feel particularly uncomfortable rating their bosses and fear the consequences of providing negative feedback. Subordinates are mostly likely to provide favorable feedback (leniency effect) on all the dimensions of behaviors (halo effect).
Self-other discrepancy. Self-other discrepancy refers to the inconsistency between self and other ratings in 360° assessment. Pioneer studies conducted in the US found that higher self-other discrepancies was related to lower leadership performance.
CCL researchers have examined this issue using data collected from thousands of leaders around the world. Overall the findings were mixed. There was no evidence that individualism/collectivism impacts self-other discrepancy. But assertiveness, the extent to which a culture emphasizes direct communication, may have an impact. In highly assertive cultures where people prefer direct communication, leaders have more feedback information on how they are perceived by their colleagues, and hence the self and other ratings tend to be consistent.
CCL researchers also examined the impact of power distance, and the results depend on the relationship between the rater and the ratee. In a high power distance culture like China, the self and peer ratings maybe consistent because leaders are most likely to seek feedback from peers. In contrast, the self and subordinate rating discrepancy may be high because subordinates may not have the opportunity to interact with the leaders and observe their behavior. There is no evidence of how power distance impacts self and boss ratings.
As mentioned previously, culture is not the only factor that influences self ratings, other ratings, and self-other rating discrepancy. In a review paper published in Leadership Quarterly, the authors summarize that the factors affecting 360° ratings include:
a) Self’s biographical characteristics, personality and individual characteristics, and job relevant experiences
b) Rater’s cognitive processes, motivation, and rater–ratee interactions and expectations
c) Contextual factors.
360° results are influenced by a variety of factors; hence, it is important to consider those when initiating a 360° assessment process and interpreting results.
The tips below are important for those using 360° assessments in China, as well as in other countries.
Communicate Purpose. Research indicates that employees’ perception of how the ratings will be used has an impact on what information they provide. Therefore,
- Before implementing a 360° assessment, make the purpose of the assessment clear. For example, if the results are going to only be used for development purposes, make sure everyone understands that.
- After the assessment, help leaders receiving the feedback interpret and apply their results. Without proper interpretation, 360° assessment may be useless, and can even be harmful.
Interpret the information.
- Consider the factors, such as the cultural context, that may influence self and other ratings, as well as the difference between them, when interpreting results.
- When raters are from different cultures, the ratings should be interpreted with special care. Cultural factors affecting the rating should be taken into consideration.
Do not over generalize or overemphasize.
- Be aware that there are many factors that potentially influence ratings. Take results seriously, but don’t accept everything at face value; think critically about results and what the significance of them may be.
Assessment results provide valuable data, but should be interpreted in the context of other information in order to be most helpful. Keep an open mind and create open communication channels to understand the raters, ratees, and the context.
How do you use and interpret 360° assessment in your organization? What are the barriers to using it effectively and how do you optimize the tool?