7 traps in data-driven decisions—and how to check them.
You launch a sales campaign. The dashboard turns upward. In the meeting, someone says, “It worked.” You helped design the campaign, so the explanation feels right. But what did the data establish, and what did you bring to it?
Cognitive bias is a systematic tendency in how you search for, interpret, remember, or judge information that can lead your conclusions away from the evidence. Confirmation bias is one example; it is not another name for the entire category.[1][2]
Mental shortcuts, or heuristics, help you make judgments without examining every possibility. They can be useful, but can also produce predictable errors.[1] A bad outcome alone does not prove bias: a reasonable decision can fail under uncertainty.
Our running example is a teaching scenario, not a reported campaign. Sales rose after launch. Before you credit the campaign, check whether prices, seasonal demand, or measurement changed. Timing alone does not identify the campaign’s effect.[8]
Observation
Recorded sales increased after launch.
Hypothesis
The campaign generated additional sales.
Decision
There is enough evidence to fund the next phase.
These seven cognitive bias examples follow the gaps between those statements. The research supports the mechanisms; the campaign examples and decision checks are applications of them.
Bias can enter before you calculate anything. Which reports do you open? Which reference point feels normal? Whose interpretation reaches you first?
You highlight the campaign’s strongest region and dismiss the weakest as “unusual.” In Wason’s rule-discovery experiment, participants often tested cases compatible with their hypothesis instead of effectively trying to rule it out.[2]
Check: Before opening another report, write what would weaken your claim. Then inspect that evidence with the same care you give favorable results. A contradictory result deserves investigation, not automatic belief.
Two routes through the same campaign evidence
How to read this: Follow the question to either route. The right branch makes your claim easier to challenge. It does not prove the campaign failed—or that every alternative explanation is equally likely.
Your launch forecast becomes the meeting’s reference point. Everyone adjusts it slightly instead of estimating from the available evidence. Anchoring experiments show that initial values can influence judgments even when they are uninformative.[1]
Check: Have reviewers estimate the campaign’s plausible effect independently before seeing the target. Then compare their assumptions. A useful baseline deserves weight; an arbitrary starting number does not.
A striking customer testimonial dominates your impression of the campaign. Availability is the tendency to judge frequency or probability by how easily examples come to mind.[1] The testimonial may be accurate while saying little about the typical customer.
Check: Inspect the relevant population and denominator. How many customers were eligible, exposed, and converted? Use memorable cases to generate questions, then check how widespread the pattern is.
A senior colleague calls the campaign a success. The next three reviewers agree—but all saw that first verdict. In a randomized experiment on a news aggregation site, prior ratings influenced subsequent ratings. Social feedback can shape the judgments it appears merely to summarize.[3]
Check: Collect written interpretations before discussion. Ask whether agreement comes from independent evidence or repeated exposure to one opinion. Agreement can be informative; its origin matters.
The campaign now has a story, an owner, and a budget. Revising the conclusion can feel harder than reaching it.
You are certain that more spending will reproduce the sales increase. Moore and Healy distinguish overestimating performance, overplacing yourself relative to others, and excessive precision in beliefs. These are related but different phenomena.[4]
Check: Record a plausible range and its assumptions, not just a best guess. Revisit predictions against outcomes across decisions. In this campaign, say which uncertainty remains: attribution, repeatability, or profitability.
After sales rise, you remember having expected the increase all along. Fischhoff found that knowing an outcome increased its judged prior likelihood; participants were largely unaware of that influence.[5]
Check: Save your prediction before launch, including alternatives and doubts. During the review, compare the result with that record. Judge the reasoning using what was available then, and separately ask what the outcome teaches now.
The campaign evidence is weak, but you argue that stopping would waste the work already done. Arkes and Blumer documented an increased tendency to continue an endeavor after investing money, effort, or time.[6]
Check: Ask whether you would fund the next phase today, given its future costs and expected benefits. Existing reusable assets and cancellation costs still matter. Irrecoverable past spending cannot, by itself, justify another round.
You cannot certify yourself as unbiased. You can make a decision easier to inspect, challenge, and revise.
“Be objective” is vague. In two experiments, asking people to consider the opposite reduced biased judgments more effectively than general instructions to be fair. That finding supports a specific challenge to your explanation, not a promise that a checklist removes bias.[7]
01 · WRITEState the claim
“The campaign caused additional sales.” Record the metric, period, and population.
02 · CHALLENGEName an alternative
Could a price change or seasonal demand explain the same observation?
03 · DECIDESet the next test
Record the evidence needed to continue, revise, or stop funding.
For campaign attribution, a well-designed randomized comparison can estimate incremental impact. It requires valid assignment, trustworthy measurement, and adequate statistical power; contamination between groups can undermine the comparison.[8]
From a sales increase to a defensible claim
How to read this: The fork tests the quality of your comparison, not your confidence. A credible comparison enables estimation; it does not guarantee a positive effect or justify scaling on its own.
If randomization is unavailable, another comparison needs explicit assumptions. A simple before/after chart does not become causal evidence because you need a budget decision. You may still act under uncertainty; record that uncertainty.
Campaign review · Decision record
Observation: Sales increased after launch.
Claim: The campaign caused additional sales.
Alternative: Demand or prices changed too.
Missing evidence: A credible comparison.
Next step: Design a test before scaling.
Try the decision · 1 question
Which response improves the decision?
Sales rose after launch, and the whole team expects the campaign to work again. What should you do before claiming it caused the increase?
Read the answer and why
Check alternative explanations and seek a credible comparison. Selecting a winning region risks confirmation bias; shared expectations need not be independent evidence. A suitable comparison helps test attribution. It does not predetermine the result.
Is cognitive bias the same as a logical fallacy?
No. A bias is a tendency in judgment; a logical fallacy is a flaw in an argument. Your desire for the campaign to succeed may shape which evidence you select, even if your arithmetic is correct.
Can data eliminate cognitive bias?
Data gives you evidence to check. You still choose the metric, comparison, and interpretation. For this campaign, adding more dashboard panels does not resolve a missing comparison.
How can you reduce bias in decision making?
Record predictions, collect independent views, examine contrary evidence, and revisit decisions against outcomes. Treat these as checks you can evaluate, not proof that you have become unbiased.
These studies support the mechanisms discussed here, within their experimental settings. The sales campaign, diagrams, and decision record are teaching examples, not business results or a validated diagnostic test.
Judgment under Uncertainty: Heuristics and Biases. Amos Tversky & Daniel Kahneman. Science, 1974. Anchoring and availability.Read the paper
On the Failure to Eliminate Hypotheses in a Conceptual Task. P. C. Wason. Quarterly Journal of Experimental Psychology, 1960. Testing hypotheses.Read the paper
Social Influence Bias: A Randomized Experiment. Lev Muchnik, Sinan Aral & Sean J. Taylor. Science, 2013. Influence of prior online ratings.Read the paper
The Trouble with Overconfidence. Don A. Moore & Paul J. Healy. Psychological Review, 2008. Three forms of overconfidence.Read the paper
Hindsight ≠ Foresight: The Effect of Outcome Knowledge on Judgment Under Uncertainty. Baruch Fischhoff. Journal of Experimental Psychology: Human Perception and Performance, 1975.Read the paper
The Psychology of Sunk Cost. Hal R. Arkes & Catherine Blumer. Organizational Behavior and Human Decision Processes, 1985.Read the paper
Considering the Opposite: A Corrective Strategy for Social Judgment. Charles G. Lord, Mark R. Lepper & Elizabeth Preston. Journal of Personality and Social Psychology, 1984.Read the paper
Controlled Experiments on the Web: Survey and Practical Guide. Ron Kohavi, Roger Longbotham, Dan Sommerfield & Randal M. Henne. Data Mining and Knowledge Discovery, 2009.Read the paper
Ali Reza Rashidi, a Senior Data Scientist-Gen Al | Al Architect | MLOps with over ten years of experience, He is the author of three books that delve into the world of data and management.
Related posts
A visual mosaic capturing the essence of America's socio-economic journey – from the heights of economic booms to the struggles of the working class, reflecting the key themes of 'Our Ways the Shining Future