You just don’t seem to understand

1839502-the-radical-transformation-of-customer-service-rotatorIn 2017, the telecommunications industry and the Australian Communications and Media Authority will be re-assessing the customer information obligations framework for telecommunications companies – what is referred to as the Telecommunications Consumer Protection Code (the TCP code). The Australian telecommunications industry has indicated long-standing desire for more flexibility with fewer restrictions in the information provided on a mandatory basis to consumers. It has been contended that current mandatory consumer information requirements, particularly in terms of the amount of information that must be provided at point-of-sale, are not necessarily useful to consumers and result in substantial cost to the industry.

One concern of consumer advocates has been that important consumer protections could be lost in the absence of independent evidence-based information and research, to the detriment of both industry and its customers. Consumer information is fundamentally important, but it has to be carefully designed so that customers can understand what they’re buying, how to use the service, and how to constructively resolve issues into the future. In the past, with the development of the code and some other regulations, there has been an absence of sophisticated empirical evidence about the best way to effectively communicate this fundamental information in the telecommunications space.

Informed consent sits behind most legal agreements, but in reality, the notion of informed consent is usually measured by directly asking consumers whether they understand their obligations and rights under a contract. While this is indeed testing the reflective capacity of consumers in relation to their belief that they have understood something, it is arguable that it is not actually measuring whether the consumer has actually understood the agreement. In other words, a person may claim to understand the implications of their signing a contract, but may fail to appreciate the possible consequences until they are presented with a particular challenge arising from or related to the terms of the contract.

A knowledge test

In our recent research, funded by the Australian Communications Consumer Action Network (ACCAN) and Deakin University’s Centre for Employee and Consumer Wellbeing, we were interested in exploring if consumers actually “understood” what was contained in their telco agreements. In other words, instead of the usual, intuitively simple question, “Have you understood this agreement?”, we tested comprehension using a more scientific methodology.

We gave three groups different types of information that would be provided when they purchased a smartphone, then asked them a series of questions or posed a series of problems that might arise in the operation of their smartphone agreement. So, examples of the questions were, “How much would a 2-minute call cost on this plan?”, “How much data is included in this plan per month?”, and “Are calls to 1800 numbers, such as Centrelink, included in this plan?” All the responses had multiple-choice possibilities, so the test was arguably easier than it would be in real life (where we are not provided with multiple choices).

One third of those participating received what was an overview of the plan – the kind you might receive from a salesperson. Another third received this sales pitch and a standard Critical Information Summary or CIS (the two page document you are required by law to receive before you sign the agreement), and the final third received the sales pitch, the CIS, and a detailed 32-page terms and conditions document.

For authenticity, all of these forms of information were modified from actual telco documents or sales processes, but we invented a unique brand so that there wouldn’t be any halo effect from things such as brand loyalty. We also tested their authenticity on an expert panel of telecommunications representatives, consuer advocates, and the general public.

We attempted to make the process as easy as possible for participants, for example, all the participants could go back and read these documents throughout the experiment, so there was no deception or trickery. We measured their understanding of the agreement through these questions in the knowledge test 24 hours after they first received the documentation, and then 2 – 3 weeks after they had first received it (while providing them with the opportunity to go back and read the documentation at any time).

We also measured a few other self-assessed items such as perceived self-efficacy, financial literacy, the believability of the documentation, their satisfaction, understanding, and the relevance of the information that was provided. In other words, we were asking people, “Did this look/feel/smell like a normal telco document?” For a good representation, we selected 362 participants randomly from an online database of 350,000 Australians.

Confident, but Confounded

What we found was that most of our participants read these agreements (minimum time spent reading was 2 minutes for the sales pitch and up to 30 minutes for the full documentation), and felt pretty confident about their comprehension of them. In other words, they said that they had done what we do when we sign these documents, saying that they had “understood the agreements”. Sadly, the reality of their comprehension did not show that they had understood the agreement much at all. Indeed, they were quite confounded about what they could and couldn’t do.

To start with, across the entire study, the mean correct response to the knowledge test across all of the study was 42%, or 11 out of the 26 potential questions. Those participants in the study who fared the worst were those who undertook the intermediate and advanced questions.

More specifically, what we found was that the optimal amount of information to give consumers at point of purchase was a standard, clear, critical information summary. Providing a detailed terms and conditions document actually resulted in a negative outcome in terms of correct answers. In other words, too much information made respondents go backwards in terms of their understanding.

We were also really surprised to find an inverse or negative relationship between self-assessed understanding and the number of correct answers. So, for each point that people gave themselves in relation to their understanding of the contract, we found a 0.3 change in how many answers they got correct. The more confident our participants were in their abilities to understand a contract, the worse they did. That said, we did find the positive relationship between those who believed that the information was relevant to their needs, and correct answers. So it might actually be explained by the idea that, if the person believes the information is useful, they actually do quite well.

We also found that those with vocational qualifications did worse than any other educational levels (this group was also the most confident that they had understood the agreement). And people whose first language wasn’t English did worse than all others. Our models showed that non-English speakers got between 1 and 3 answers more incorrect than those whose primary spoken language was English.

In another part of the study, we found that key stakeholders underestimated and overestimated the capacity of consumers to solve these everyday problems. Regulators (those people responsible for determining consumer protection laws) had the most realistic expectations of consumers comprehension of their agreements. Consumer advocates underestimated consumers’ comprehension, and telecommunications representatives overestimated consumer capacity to understand the detail of their agreements.

What does this all mean?

The key finding from this research is that none of us are good predictors of what consumers understand or comprehend. Not even consumers themselves. We’re biased, and we shouldn’t rely on our own warped understanding of what we think we are capable of to make judgements about what we think others are capable of.

Which leads to a really strong argument that we regulators and telcos should be undertaking rigorous and neutral research to test the effectiveness of contracts, codes, and regulations in the context of what we want to achieve with those different types of codes and contracts, well before they are rolled out – a form of due-diligence in relation to consumer wellbeing. At present, regulators and the industry tend to rely on anecdote and self-assessed questionnaires when judging the effectiveness of regulations and codes.

And at a more practical and policy level, it is simply unrealistic to expect consumers to have anything more than a moderate understanding of even the easiest contractual elements of a telco agreement, and highly unrealistic to expect that people can respond to difficult or complex issues that might arise from the operation of that agreement.

An edited version of this piece was published in The Conversation and reported on in The Sydney Morning Herald, The Age and The Canberra Times.

This entry was posted in Consumer Behavior, Human Behavior and tagged , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s