Why This Matters
Statistics are a vital part of effectively evaluating information and yet, as the saying goes, “There are three kinds of lies: lies, damned lies, and statistics.” Misusing statistics is a key tool for manipulating opinion, and numbers are often used to mislead others in order to meet a particular agenda.
Torture numbers, and they’ll confess to anything.
– Gregg Easterbrook
To keep from being manipulated with numbers and statistics is not a matter of mathematical ability. Instead, it relies on learning basic methodology and knowing the ways in which people with an agenda use numbers to mislead. Daniel Levin sums up the issue perfectly:
Statistics Are Interpretations and Sometimes Numbers are Wrong
Statistics, because they are numbers, appear to us to be cold, hard facts. It seems that they represent facts given to us by nature and it’s just a matter of finding them. But it’s important to remember that people gather statistics. people choose what to count, how to go about counting, which of the resulting numbers they will share, and which words they will use to describe and interpret those numbers. Statistics are not facts. They are interpretations… Sometimes, the numbers are simply wrong.
– Daniel J. Levin, Weaponized Lies
Key Message
- Statistics are not facts. They are interpretations or, as Hector Macdonald would call them, partial truths.
- As a result, numbers can be used to hide or subvert the truth.
Effective Evaluation
Numbers Are Wonderful But They’re Often Misunderstood
Numbers are wonderful. They give us a clarity about our world that words often fail to provide. They allow us to compare things, to rate things, to measure change, to sum up a galaxy in a single figure… They are a universal language… But the problem with numbers is that so many of us misunderstand them so much of the time… Numbers ought by rights to be the most transparent form of communication in existence and therefore the hardest to abuse. Instead, we find competing truths co-opting numbers in every walk of life.
– Hector Macdonald, Truth: How the Many Sides to Every Story Shape Our Reality
When presented with numbers, Daniel Levin suggests the following:
- Do not accept a claim at face value. Do a quick plausibility check.
- Determine how the numbers were collected.
- Review the interpretation and graphic presentation.
Mistakes & Manipulation
The most common ways that people mistakenly or intentionally misuse numbers include:
- Cherry-picking statistics
- Mistaken or misleading reports of percentage changes and comparisons
- Choosing baseline units to mislead
- Misusing averages
- Making numbers look bigger or smaller
- Exaggerating or concealing trends
Misleading Reports
There are innumerable ways that numbers can be chosen to manipulate opinion, and such tactics are commonplace. (Daniel J. Levin, Weaponized Lies) Because this type of trickery is commonplace among organizations who wish to sway opinion and obtain funding, every large organization must know about these tactics. Whether or not they’re using deception in their communications is left for us to decipher. Here are a few examples of misleading reporting.
- Statistics may be properly created and reported, but later be mistakenly reported by others, due to a change in wording that inadvertently changes the meaning.
- A statistic may be reported not because it is relevant to the decision being made, but because it presents the desired message. For example, if a company wants to convince others it’s doing well, it could publish a report that its annual sales are increasing. But if the market is growing, they would be expected to have increasing sales. The appropriate statistic to know if they’re doing well would be their change in market share.
- A statistic can be so blatantly manipulated as to create an impression that has the same effect as having lied. The Advertising Standards Authority in the UK ruled that Colgate engaged in an unfair practice with its reports that “four out of five dentists recommend Colgate toothpaste.” Although the statement is true, the impression that dentists prefer Colgate above other brands is not true. The survey allowed dentists to choose more than one toothpaste and in fact their competitor was named as often as Colgate in the actual survey.
Percentages
Calculating Percentage Change
- Percentage change means the change from the baseline. The baseline is what we start with.
- For example, if interest rates rise from 3 percent to 4 percent, the baseline is 3 percent.
- In this example, the change (increase in this case) is 1 and the baseline is 3.
- Thus, the percentage change is 1/3 or 33 percent. A change from 3 percent to 4 percent is a 33 percent change.
- Percentage change may also be reported in terms of “percentage points” which is not the same as percent change because it’s not relative to the baseline. It’s simply the numerical change in numbers. In this case, there was a change of 1 percentage point.
- For clarity, percentage change should be reported in both percentage point difference and the relative change. Thus, if interest rates rise from 3 percent to 4 percent, that is an “increase of 1 percentage point, or 33 percent.”
- A 50 percent reduction in a $1,000 salary is $500. But a 50% increase does not restore the salary to its previous amount because the baseline has shifted from 1,000 to 500. Thus, a 50% increase from $500 is $250, making the new salary $750.
Watch Out For
- When comparing percentages, be sure the same baseline is being used.
Averages: One Testicle?
Be Vigilant with Averages
Be careful of averages and how they’re applied. One way that they can fool you is if the average combines samples from disparate populations. This can lead to absurd observations such as, “On average, humans have one testicle.”
– Daniel J. Levin, Weaponized Lies
Calculating Averages
- Average may be calculated as mean, median or mode.
- To calculate the mean: add up all the numbers, and then divide by how many numbers there are.
- To calculate the median: create a “distribution” by listing all the numbers from smallest to largest, and then choose the middle number.
- To calculate the mode: select the number that appears most often; if no number is repeated, then there is no mode.
Watch Out For
- Averages don’t tell anything about the range, which may be vital to the topic. As Levin notes, “the average annual temperature in Death Valley, California, is a comfortable 77 degrees F (25 degrees C). But the range can kill you, with temps ranging from 15 to 134 degrees F.”
- Because average can be calculated three different ways, then it may (not always, but often) have three different results. Thus, an easy way to manipulate the interpretation is to pick the one most favorable to the desired argument and then simply use the word “average” without clarifying the calculation behind it.
- There are endless ways to manipulate interpretations using averages. See Weaponized Lies for many clear examples.
Graphs
Creating Graphs
- Graphs are a visual representation of numbers, used to show patterns and trends in the data.
- They may include every data point, or they may summarize data.
Watch Out For
- Not labeling the graph’s axes is a simple way to lie with a graph.
- Another way to manipulate the numbers is to choose a particular start point of the y-axis. To unduly emphasize a difference, the y-axis will start near the lowest value being plotted, as opposed to starting at zero.
- Plotting things that are unrelated can trick viewers into thinking one thing caused something else when, in actuality, there’s no credible reason for that assertion.
- To see how effectively people manipulate graphs for particular agendas, see Weaponized Lies for many clear examples.
Sources & Resources
- Levin, Daniel J. — Weaponized Lies: How to Think Critically in the Post-Truth Era — 2017 book
- Macdonald, Hector — How the Many Sides to Every Story Shape Our Reality — 2018 book
