7 Explanation
(Modified on 2026-08-11)
1,797 words
As we’ve discussed, the ultimate goal in research is usually to explain the behaviors and workings of universe around us. We want to understand why we experience or observe the phenomena that we do. There are many other human activities that provide explanations for phenomena. History explains past events. Aesthetics explains art. Religion explains the meaning of life. Poetry explains the human heart. In each of these cases, people may draw great satisfaction from the explanations offered. Who is to say which explanations are True in any ultimate sense?
In the last readings, I argued that the rhetoric we use to advance scientific explanations (theories) include rigorous self-skepticism, and falsifiable predictions. Scientific explanations are used to make predictions; when the predictions derived from these accounts are borne-out, people tend to ascribe greater credence to the originating explanation. In this process, deductive (a priori) knowledge is used to make predictions about inductive (a posteriori) observations—e.g., data or evidence—, which serves as a test of the theory. But what about the opposite direction? How do we proceed from observation to theory?; that is, come up with explanations for what we experience? But where do theories come from in the first place? How do we decide which theories/explanations and associated hypotheses should be tested?
The human mind is an explanation generating machine like no other. Human beings are very, very good at explaining and rationalizing observations. We do it unconsciously, without even realizing it. Responding rapidly to our senses is an essential survival tool. We need to take in sparse, imperfect, information from our senses and quickly “jump to conclusions” (jump to explanation) so we can take action. Our explanations may be wrong, so we’ll make a lot of Type 1 errors (our minds are prone to positives, right?). But as we’ve seen, Type 1 errors are often a lot more dangerous (literally) than Type 2 errors. “I believe there is a tiger sneaking up on me; I am wrong”—That’s a Type 1 error. “I believe there is not a tiger sneaking up on me; I am wrong”—That’s a Type 2 error. We’d obviously prefer to make the latter error.
Unfortunately, our cognitive tools for generating explanations are badly susceptible to various biases and errors (63 potholes). One of the worst is confirmation bias: interpret observations in a way which confirms our existing knowledge—we see what we want to see. However, the flip side of this is just as dangerous: Hindsight bias.
Hindsight
Anything seems commonplace, once explained.
— Dr. Watson to Sherlock Holmes (Sir Arthur Conan Doyle)
A large volume of research in psychology has shown that people are prone to think that events are much more obvious in retrospect (e.g., Slovic and Fischoff 1977; Baratz 1983). Asked to predict the outcome of an upcoming election, people will be unsure of their predictions. But after the election is over, when asked how well they would have predicted the outcome, people are very confident that they would have correctly anticipated the actual results. In hindsight, we think we “knew it all along” (Hawkins and Hastie 1990). In general, psychological research has established that we grossly overestimate our predictive abilities (Watts 2011).
At the University of Leicester (UK) Karl Teigen (1986) asked students to judge the truthfulness of both actual proverbs and their opposites. When given the genuine proverb “Fear is stronger than love,” most rated it as true. But the reverse form, “Love is stronger than fear,” was also rated by most students as true. Similarly, the actual proverb “He that is fallen cannot help him who is down” was rated by most students as true. But its opposite was also judged by most to be true—“He that is fallen can help him who is down.” Two other proverbs that were rated as likely to be true were: “Wise men make proverbs and fools repeat them” and its opposite: “Fools make proverbs and wise men repeat them.”
As the Danish philosopher-theologian Søren Kierkegaard put it, “Life is lived forwards, but understood backwards.” We over-estimate our ability to make sense of past events. It is common for people to think we “knew-it-all-along”—when, in fact, we didn’t.
As Paul Lazarsfeld (1949) noted, “Obviously, there is something wrong with the whole argument of”obviousness.” It is essential for researchers to be aware of the ease with which one can generate plausible theories to account for any result. There is a huge difference between explaining something once you know the results, compared with formulating the theory first, before the results are known. We should be much more suspicious of post hoc theories than a priori theories.
Intuition is important in life, but our intuitions about past events are clouded by what we know today. The ease with which researchers can explain data after they look at it is referred to as hindsight bias. Everything is obvious in retrospect.
Remember the grandmother effect?—what we call the tendency for people to think that some explanations are “obvious”? Notice that hindsight bias further aggravates the grandmother effect. It’s not simply the case that research often demonstrates the blindingly obvious, as we’ve seen, once people know the answer, they are convinced that they knew it all along. So the price paid for doing empirical research is three-fold: 1) the actual cost of doing the work, 2) the indignity of confirming that 95% of human intuitions are correct, and 3) the indignity of people—having learned the results—wrongly believing that they knew it all along. (Research invites failure.)
Post hoc explanation
Hindsight-bias is one demonstration of the dangers associated with post hoc reasoning. “Post hoc” reasoning means coming up with a theoretical explanation for something after you observe it. Inventing theories that fit the data. The (potential) problem is that theories, which are generated in our minds, are a dime-a-dozen—we can always make up more theories until we find something that “fits.” (We can also train computers to do this!)
When a hypothesis fails, our immediate inclination is to explore the data to find out why. With some exploration of the data, we can often come up with alternate (or modified) theory that makes it all work. We might tell ourselves, “Oh, I was right, I just didn’t take into account this other variable.” Or perhaps we think “I thought A would be bigger than B, but B is bigger than A…and that is even more interesting!”
Many post-hoc explanations may be correct—you really do need to just tweak your theory a little, etc. But if you can (almost) always come up with a post-hoc theory, the theory has less rhetorical impact. After all, what do we think of a prophet who makes up (or updates) their prophecy after the event? The fundamental problem is that we will always appear to be right—post-hoc reasoning can’t be falsified. Post-hoc reasoning, fueled by hindsight-bias, leads us to “move the goal post” so that we never lose. It’s thwarting our ability to be falsified.
Post-hoc (inductive) reasoning also has the disadvantage of being derived from a single sample of data. This can lead to “overfitting.” The theories we create fit our current data perfectly (think of the white swans), but don’t generalize to other data.
Exploration
At this point, you might be convinced that post-hoc reasoning, or inductive reasoning in general, is always pointless and unscientific. In fact, some philosophers argue that research is only “science” when engaging in falsifiable tests. This is a nice thought, but it is simply not feasible! We can’t operate without induction. We need to start somewhere, so start with what we see and feel, and come up with explanations. You should always, do post-hoc analyses on your data after a hypothesis test. This is how you get ideas for your next hypothesis! We’ll often come up with hypotheses based on our personal experience and intuition.
In fact, there is an entire class of valid research that we call exploratory research. In exploratory research, we essentially gather data freely, then try to invent theories based on it. Exploratory research often focuses on description, but can go further by suggesting new explanations for the data. This includes lots of “Big Data” research, where we design algorithms to search for patterns in the data.
Exploratory research is an essential part of science. Often, we don’t even know where to start when researching a new topic: we might have no idea what questions to ask, or what needs to be explained. We need some “starting” observations to motivate and generate ideas; We may need to “open the box” before we know what sort of phenomena need to be theorized about. However, exploratory (and descriptive) research doesn’t invite falsification. Since the theory is not presented prior to making the observations, there is no way to test our interpretations or descriptions. When we research this way, we always appear to be right since our observations are always consistent with our theories; We make no prior claims, and therefore can never be wrong. Without the possibility of falsification, exploratory/descriptive research is more susceptible to Type 1 errors, over-fitting, and over generalization. This means that these approaches can never achieve the same rhetorical power as hypothesis-driven research.
Post-hoc switcheroo
What is absolutely essential is that you are always clear and open, both with yourself and your peers, about what work is and isn’t exploratory. Don’t ever present exploratory research as if it were a priori. Successful a priori predictions have a hefty rhetorical weight—they could have been wrong, but weren’t. Trying to pass a post hoc explanation off as “confirming a prediction” is very misleading. Good reviewers will be very hostile to and suspicious of this. Be absolutely clear about what is and isn’t post hoc.
You might think I’m just telling you not to lie in your writing—which I am. But there is more to it. Due to hindsight bias, it is very easy to “slip” in post-hoc thinking without realizing it. This is especially true because research projects can go on for many months—you can lose track of what you actually thought in advance, and what ideas came later. To minimize this possibility, be sure to keep a research diary or lab notebook (we’ll soon learn about our “write the Paper First” strategy, which helps with this too). Particularly in data analysis, it can be difficult to thread the line between “pure” a priori analysis and post-hoc analysis.
There is only one sure way to avoid hindsight bias—formulate your theory before you collect your data. Human beings have a tendency to fall into post-hoc, “moving the goal post,” rationalization without even realizing it. This is why it is essential to set concrete research plans a priori. The most famous example is \(p = .05\), which is used as a totally arbitrary cutt-off in statistical analysis. \(p = .05\) is a goal post that you cannot move. No reviewer would trust you if you did.
Mantras
In order to prevent ourselves from consciously or unconsciously adapting our theories to match what we observe—“moving the goal post”—we must make the goal post fixed and immovable in advance. We must draw a line in the sand (like \(p < .05\)) in advance, and respect that line, whatever comes. Even if you have (or think you have) good reasons to adopt a new (post hoc) goal, you must always acknowledge the original goal!
We say “hindsight is 20/20”: once something has happened, we feel we understand exactly why—and imagine we could have predicted it (“I knew it all along”). This clarity is an illusion. Uncertain outcomes come to feel inevitable, and an explanation that merely fits what happened feels obvious in retrospect. But “it makes sense in hindsight” is no evidence the explanation is right—which is exactly why a prediction made in advance beats an explanation invented after the fact (Post-hoc is a four-letter word).
Humans are gifted at coming up with explanatory accounts of data we have already seen—post-hoc theorizing. These after-the-fact explanations feel as good as a priori ones (our hindsight bias), but aren’t. What’s worse, we can easily lose track of which of our ideas were post hoc and which a priori, which is why scholars are wary of post-hoc reasoning sneaking in where it shouldn’t be. An important act of self-skepticism and professional humility is clearly labeling post-hoc aspects of your work. We’ll often say things like “we found this pattern in the data, but only in a post-hoc analysis.”
(For non-native speakers: “four-letter word” is a euphemism for an offensive word.)