34  Blind procedures

(Modified on 2026-08-11)
1,040 words

Let me tell you the story of Clever Hans (1895–1916).

Hans was a horse. He was owned by Wilhelm von Osten, who was a mathematics teacher, an amateur horse trainer, phrenologist, and something of a mystic. Why was Hans “clever”? Well, von Osten taught Hans (the horse) to add, subtract, multiply, divide, work with fractions, tell time, keep track of the calendar, differentiate musical tones, and read, spell, and understand German! Von Osten would ask Hans, “If the eighth day of the month comes on a Tuesday, what is the date of the following Friday?” Hans would answer by tapping his hoof the correct number of times. Questions could be asked both orally and in written form. Von Osten exhibited Hans throughout Germany, and never charged admission.

Due to the large amount of public interest, the German board of education appointed a commission to investigate von Osten’s scientific claims. The psychologist Carl Stumpf formed a panel of 13 people, known as the Hans Commission. This commission consisted of a veterinarian, a circus manager, a Cavalry officer, a number of school teachers, and the director of the Berlin zoological gardens. This commission concluded in September 1904 that no tricks were involved in Hans’ performance.

The commission’s work was continued by Oskar Pfungst, who tested the basis for these claimed abilities by:

Using a substantial number of trials, Pfungst found that Hans the horse could give the correct answer even if von Osten himself did not ask the questions, ruling out the possibility of fraud. However, he determined that the horse got the right answer only when the questioner knew what the answer was and the horse could see the questioner. He observed that when von Osten knew the answers to the questions, Hans got 89 percent of the answers correct, but when von Osten did not know the answers to the questions, Hans only answered six percent of the questions correctly. Evidently, von Osten was giving subtle visual clues that indicated when Hans should stop tapping his hoof. However, by all accounts, von Osten was not just a fraud; he was genuinely unaware that he was giving Hans clues.

The Clever Hans Effect illustrates how easy it is for an experimenter to inadvertently provide non-verbal (or verbal) cues as to what they hope will happen. These clues can be obvious even to a horse, so you can imagine that humans are even more tuned-in to subtleties of human body language.

One of the cornerstones of human-participant research is blind procedures. In a blind procedure, experimental participants are not aware of the experimental manipulation applied to them. The classic example, comes from drug research, where participants are typically assigned (randomly) to treatment and control groups. However, if one group is given a pill (or injection) and the other not, participants would be aware of the experimental manipulation. To make the experiment blind, control-group participants are given a placebo pill, which looks identical to the real pill, and no participant is told whether they receive the real drug or not. The participants are thus “blind” to the experimental manipulation.

The key, as we’ve learned before, is that blind procedures allow us to disentangle covariates (and confounds) from the research variables that really interest us. The fundamental problem we face is that humans (and horses) are incredibly sensitive to their surroundings, which means it is very easy for human variables to be accidentally affected by other variables, even without the person’s (or the horse’s) conscious knowledge. In the case of Clever Hans, the variable “number of hoof taps” was influenced by an unexpected third variable (“cues from the questioner”). In the case of placebos, the variable “patient’s medical recovery” can be influenced by the covariate “patient believes they received medicine.” These two cases are quite different; the commonality is that the research solution is to hide what is going on from the humans involved.

Double Blind

The story of Clever Hans illustrates another possibility: that a researcher’s awareness of experimental manipulations (or other research variables) can influence results, even unconsciously. This is why most drug research actually uses a double blind procedure, where the researchers are also not aware which participants receive placebos and which receive the real drug. When both the experimenter and the participants are unaware of experimental manipulations, the experiment is said to be double blind.

Any of us is capable of subtly, unconsciously biasing our decisions to favor the results we want to find.

Double blind procedures are important wherever a researcher makes decisions that affect the research results, especially when those decisions require significant subjective interpretation. Any of us is capable of subtly, unconsciously biasing our decisions to favor the results we want to find. By taking these decisions out of our control we can remove these biases. Typically, we either rely on a mechanical decision maker (for example, a random number generator) or another independent human who isn’t aware of the research variables, goals, or hypothesis. This independent rater is kept “blind” to the research variables.

The need for double-blind procedures is most acute in between-participant research designs. It is important that participants be assigned randomly to different conditions, that the experimenter should not be aware of which group they are assigned to, and that each group is treated exactly the same, except for the manipulation. This can usually be accomplished by randomization in the computer-interface, but may require more elaborate efforts in other cases. In some cases, it can be impossible to hide which group a participant belongs to—for example, if perhaps one group is taking piano lessons and the other guitar lessons. We can still implement blind procedures to some extent: for example, by hiring an independent evaluator, unaware of the hypotheses, to evaluate student performances—or perhaps, two separate evaluators, one for each group. This way the measurement of the dependent variable (at least) is still blind.