We generally perceive that culture is reflected in art, music and social events, so it may sound odd that culture manifests in figures as well. After all, numbers are supposedly universal; the value of a given digit does not change regardless of where you are in the globe. So, how is it possible that culture can influence them?

The next time you use a ride-hailing app, think about the ratings the drivers are given. In Pakistan, it is rare for a driver to have a rating of less than 4.5 out of five. In isolation, this score seems an excellent rating; after all, it equates to 90%! However, users of the app are aware that compared to the ratings of other drivers on the same app, it falls in the lower bracket, as a good score is considered to be over 4.7. Within a space of merely half a rating point, Pakistani users rank a driver from poor to good; in other words, the majority of the population uses just a tenth of the scale!

Pakistanis tend to exhibit what insight professionals term Extreme Response Style; we tend to use the extremes of a rating scale. This presents a problem when one tries to measure emotive variables such as satisfaction or recommendations. Our population will generally give very high scores at the positive end of any rating metric. This, in turn, paints a rosy picture, although it does not mean that there is no room for improvement for a given product or service.

Interestingly, such cultural biases occur across the globe. In Japan, a score of six out of 10 is considered to be very high while the same score in Pakistan would be perceived as average. According to an analysis conducted by the global research agency Ipsos, Latin American countries exhibit similar tendencies. However, European and East Asian countries tend to use the middle of the scale, resulting in much lower aggregate scores, with Singapore residents considered to be the stingiest in terms of handing out high scores.

Why do we behave in this way? After a ride ends, drivers tend to ask for a good rating and most of the time we indulge them with the intention of helping them meet their targets. Moreover, as a developing market, most Pakistani consumers have very basic expectations which, if fulfilled by a product or service, will result in high satisfaction scores.

An advertiser from a media agency once complained that Pakistanis lie when asked for their opinions; personally, I feel that we are a very ‘obliging’ society. Although these cultural attitudes are here to stay, there are ways they can be taken into consideration to ensure the numbers are interpreted correctly.

it is critical to benchmark any rating metric, such as the Net Promoter Score (NPS) so that it is more relevant to a particular society. For the Pakistani market, which exhibits an Extreme Response Style, it would be a good idea to make the NPS scale more stringent by pushing it to the right by decreasing the threshold of the ‘promoter’ score and increasing it for the ‘detractor’ score in order to normalise overly-optimistic responses.

The research methodology of any study plays an important part in determining cultural bias: face-to-face interviews have a higher rate of extreme responses as opposed to indirect techniques such as a telephone call or the internet. This does not mean that either methodology is incorrect; simply that one should expect a more skewed response in certain cases.

The presence of the interviewer (especially in qualitative studies) tends to make respondents more obliging. Respondents may claim to use products or services made by a certain brand that are not offered by that particular brand or claim to be aware of the latest trends or technologies in order to avoid giving the impression of seeming ‘foolish’. This is referred to as Response Set Bias, where respondents want to project a certain image by providing an edited response that is socially more desirable.

Similarly, it is critical to never look at scale-based results in absolute terms; instead, focus on the trend. In the case of driver ratings, a percentile score should be given which compares the scores of all drivers and lets the user know what percentage of drivers have that same score or less. A driver with a seemingly high rating of 4.6 may fall in the 25th percentile – meaning that 75% of the other drivers have a higher rating. Most advertising testing models tend to have norms and benchmarks for this purpose, although it is important to keep in mind how the benchmark was established.

It is also a good idea to pose questions about specific information areas along with overall satisfaction-based scales. For instance, although a consumer may be satisfied with the overall performance of their car, they may be less satisfied with its safety features, fuel economy or resale value – and this highlights potential improvement areas. Furthermore, the use of worded scales further reduces bias by categorising a certain sentiment. However, the scales should be limited otherwise they tend to create confusion unless the target respondent is very specific.

Most importantly, it is critical to benchmark any rating metric, such as the Net Promoter Score (NPS) so that it is more relevant to a particular society. For the Pakistani market, which exhibits an Extreme Response Style, it would be a good idea to make the NPS scale more stringent by pushing it to the right by decreasing the threshold of the ‘promoter’ score and increasing it for the ‘detractor’ score in order to normalise overly-optimistic responses.

In the original NPS scale, a respondent who gives a score of nine or 10 is considered a ‘promoter’. In Pakistan, only those who give a score of 10 should be considered as such, those given a score of nine should be considered ‘passive’. Similarly, a detractor is someone who gives a score from zero to six in the original scale; in Pakistan, the threshold should be increased to seven. The reverse could be applied for countries such as Japan and Singapore to ensure low scores are not necessarily interpreted in a negative sense.

The context is critical behind any form of analysis, especially behavioural and psychometric and can vary from industry to industry. It is important to recognise that giving a high satisfaction score does not equate to greater positivity or vice versa. Stakeholders should guard against the notion that nothing can be done to improve customer loyalty in a country with traditionally low scores, or that it is acceptable to be complacent in countries where loyalty scores are high.

Figures can be extremely misleading if looked at in absolute terms considering that emotive variables are very subjective. This is how human behaviour is reflected in numbers: a rating of seven for you might mean a five for someone else, as the criteria for satisfaction is different for each person.

Hence, tackling cultural bias in research is critical to ensure that actionable insights are not lost in translation. The right approach can uncover risks and opportunities within the customer experience that are often hidden by cultural norms, allowing an organisation to better evaluate customer loyalty and satisfaction trends and set appropriate goals for improvement.

Ans Khurram is an insights professional working in the telecommunication industry in Pakistan. anskhurram@gmail.com