Review and Summary: The Data Detective

Don’t be fooled by its title; The Data Detective isn’t your typical technical statistics book. Instead, it’s a riveting guide on how to maintain a discerning attitude when bombarded with numbers. The original title of this book was How to Make the World Add Up: Ten Rules for Thinking Differently About Numbers. Tim Harford presents a compelling set of rules to arm you against being deceived by statistics. Packed with captivating real-world examples, it effortlessly draws readers in, making complex cases relatable and easy to grasp.

Summary

Rule #1: Search Your Feelings

We often find ways to dismiss evidence that we don’t like. And the opposite is true, too: when evidence seems to support our preconceptions, we are less likely to look too closely or flaws. The more extreme the emotional reaction, the harder it is to think straight.
We don’t need to become emotionless processors of numerical information—just noticing our emotions and taking them into account may often be enough to improve our judgement. Ask yourself: how does this information make me feel?
Our emotions are powerful. We can’t make them vanish, nor should we want to. But we can, and should, try to notice when they are clouding our judgment.
We can learn to control our emotions by: notice those emotions. When you see a statistical claim, pay attention to your own reaction If you feel certain emotions, pause for a moment.

Motivated reasoning

A wishful thinking.
Judging influenced by what we hoped would be true.
People with deeper expertise are better equipped to spot deception, but if they fall into the trap of motivated reasoning, they are able to muster more reasons to believe whatever they really wish to believe.

Rule #2: Ponder Your Personal Experience

What should we do when the numbers tell one story and day-to-day life tells us something different?

Statistical evidence can feel dry and thin. It doesn’t touch us in the same memorable and instinctive way as our personal experience. Yet we have to realize that our personal experience is limited.
- We must simply keep reminding ourselves what we’re learning and what we might be missing. In statistics, elsewhere, hard logic and personal impressions work best when they reinforce and correct each other. Ideally, we’ll find a way to combine the best of both.
If we don’t understand the statistics, we’re likely to be badly mistaken about the way the world is. It is all too easy to convince ourselves that whatever we’ve seen with our own eyes is the whole truth; it isn’t. And yet, if we understand only the statistics, we understand little. We need to be curious about the world that we see, hear, touch, and smell, as well as the world we can examine through a spreadsheet.

Conclusion: try to take both perspectives—the worm’s-eye view as well as the bird’s-eye view. They will usually show you something different, and they will sometimes pose a puzzle: How could both views be true? That should be the beginning of an investigation. Sometimes the statistics will be misleading, sometimes it will be our own eyes that deceive us, and sometimes the apparent contradiction can be resolved once we get a handle on what is happening.

Rule #3: Avoid Premature Enumeration

When we’re trying to understand a statistical claim, we need to start by asking ourselves what the claim actually means. Ask yourself if you really understand what the claim means. Ask what is being counted, what stories lie behind the statistics.

Rule #4: Step Back and Enjoy The View

However much news you choose to read, make sure you spend time looking for long-term, slow-paced information. You will notice things—good and bad—that others ignore.
Step back and look for information that can put the claim into context.
Look for something that will give you a sense of scale, such as comparing the situation in one country with the situation in other countries, or figuring out the cost per person of some proposed government expenditure.

Rule #5: Get The Backstory

Find science journalism that explains the facts, puts them in a proper context, and when necessary speaks truth to power. Ask yourself if the journalist reporting on the research has clearly explained what’s being measured. How large is the effect? Was this a surprise to other researchers?
Find second opinions: other academics and other specialists.

Rule #6: Ask Who is Missing?

What numbers are and aren’t collected, what is and what isn’t measured, and who is included or excluded are the result of all-too-human assumptions, preconceptions, and oversights.
Beware of sampling bias.

Rule #7: Demand Transparency When The Computer Says No

We shouldn’t be too eager to entrust our decisions to algorithms.
The problems is not the algorithms, or the big datasets. The problem is a lack of scrutiny, transparency, and debate.

Rule #8: Don’t Take Statistical Bedrock for Granted

There is nothing wrong with the idea that government should collect statistics to inform itself. But there is a risk that this view slips into a proprietorial sense of ownership. When politicians believe not only that they should be using statistics to run the country, but that those statistics are non of anyone else’s business, and that external scrutiny is a distraction. The facts are no longer the facts—they become the tools of the powerful.
Yet for all their problems and weaknesses, official statistics are still the closest we have to data bedrock.

Rule #9: Remember That Misinformation Can Be Beautiful, Too

Much of the data visualization that bombards us today is decoration at best, and distraction or even disinformation at worst.
When you are on the receiving end of beautiful visualizations:
- Check your emotional response first
- Check that you understand the basics behind the graph: what do the axes actually mean? do you understand what is being measured or counted? Do you have the context?
When you look at data visualization, you’ll do much better if you recognize that someone may well be trying to persuade you of something.

Rule #10: Keep An Open Mind

This book has argued that it is possible to gather and to analyze numbers in ways that help us understand the world. But it has also argued that very often we make mistakes not because the data aren’t available, but because we refuse to accept what they are telling us.