Cross-Cultural Research
More and more Australian research companies are conducting research internationally. For over five years now, Forethought has been conducting research across 20 different countries in 16 different languages. These examples are only two cases of what has become, in this global economy, a burgeoning trend to market across cultures. Consequently, it is becoming increasingly more important for Australian firms to better understand how cultural differences may influence response behaviour measured by scales.
Research has found that Asian cultures were more likely to choose the mid-point of a rating scale than non-Asian cultures. Without understanding this, such results may be incorrectly interpreted to reflect actual performance level. In some studies, Asian countries and Australia are combined into a single reporting region. The lower means are compared with North American and European regions’ means with the assumption that the lower scores are due to lower performance.
In examples where the mid-point was removed, the responses were more varied and did not tend to centre so much around the middle region of the scale. That is, the Westernised type of response categories used, may affect the questionnaire results derived from people of different cultures.
Another study found that Asian cultures were significantly less likely to choose extreme end point categories than the Western cultures, regardless of odd or even numbered scales. Suggested influences on these choices in Asian cultures include Confucian philosophy which teaches moderation and to avoid extremes; one should not stand out from the crowd but choose the ‘Middle Way’. Also, the cultural characteristic of ‘saving face’, both one’s own and the ‘face’ of others; this is achieved by avoiding disagreements or conflicts.
Distinctions are often made between the individualistic (ie. North American) and collectivist (ie. Japanese, Chinese) societies. Preferences in the use of the scale values has been found to be related to the degree of endorsement of these two societal types. It was shown that using a 7-point likert scale (odd-numbered and so has mid-point), Asian students are significantly more likely to choose mid-point responses than American students. Also, American students are significantly more likely to choose extreme end point values than Asian students. With each cultural group, endorsement of individualism was positively correlated with the use of extreme values and negatively correlated with the use of mid-points.
Despite all of these significant differences, none of them resulted in any significant differences across the actual mean rating scores. This was determined by comparing the cross cultural differences in mean ratings before and after controlling for the differences in response styles. This controlling was achieved by reducing the 7 point scales to 3 and 2 point scales. It has been shown that this removal of extreme end point and mid-point values is equivalent to having used these reduced scales in the actual data collection. A comparison of the mean ratings before and after this modification then, showed no significant differences existed. The conclusion that follows is that differences in average ratings across cultures must be due to factors other than the scale structure and related differences in response style.
So consequently, no evidence exists to date that proves the differences in response style with respect to avoiding extreme values or favouring mid-points, provides a meaningful explanation of cross cultural differences found between Asian and the American students.
Intuitively, the explanation must lie broadly with the different cultural influences that we are all exposed to in our upbringing. But how can we take these differences into account when comparing say, satisfaction ratings for performance from one culture to the next?
One suggestion is to introduce the use of a calibration question. This is discussed in the article entitled "Calibration Questions: Interpreting Respondent Ratings", also available in Think Tank. Its purpose is to measure the respondents’ base level of expectation so that their subsequent ratings can then be compared to this and the difference calculated. In effect, this serves to cancel out any cultural influences on the ratings leaving the ‘net rating’ more reflective of actual performance score.
Calculating this difference provides an indicative benchmark of the size of the differences in ratings that can be expected across the various cultures. This can then be taken into account when making the various comparisons of averages.
This is a simple measure to take in order to address the potential bias that may occur in cross cultural surveys. Whilst not the only solution, importantly, it attempts to acknowledge and account for the ever increasing presence of cultural differences in research today.
