27  Quantifying Variables

(Modified on 2026-08-11)
2,128 words

An operational definition is a concrete procedure which associates observations with various levels of a theorized variable. In most scientific/engineering research, this means associating levels with numbers: quantification. Using numbers imposes a lot of structure on our variables, which can be extremely reductive. However, these same numerical structures can be very useful, as they equip us with a vast arsenal of mathematical tools for modeling relationships: statistics, linear algebra, calculus, etc.

As always, we must consider construct validity: does a quantified operational measurement really reflect the structure of the theoretical construct we are studying? Maybe it does. Maybe it doesn’t. But if a variable can be fruitfully viewed as numbers, it makes a lot of things (for example, falsification) much easier!

Levels of measurement

One basic part of any theoretical or operational definition for a research variable is how many levels it has. We can also consider what mathematical structure (if any) these levels contain. The most broad distinction we can make is between categorical (qualitative) and numeric (quantiative) variables. However, there is actually a continuum between qualitative categories and quantitative numbers. Research methodologists have long identified four “classic” levels of measurement (Stevens 1946):

  1. Categorical
  2. Ordinal
  3. Interval
  4. Ratio

However, these are just the tip of the iceberg. Numbers are themselves theoretical constructs (plural), with various definitions and related properties and structures. There are many basic sets of numbers we commonly engage with:

  • Natural numbers (\(\mathbb{N}\))
  • Integers (\(\mathbb{Z}\))
  • Rational numbers (\(\mathbb{Q}\))
  • Real numbers (\(\mathbb{R}\))
  • Complex numbers (\(\mathbb{C}\))

\(\mathbb{N} \subset \mathbb{Z} \subset \mathbb{Q} \subset \mathbb{R} \subset \mathbb{C}\)

There are also other more specific or abstract concepts (e.g., vector spaces, linear operators, tensors, topologies, probabilities, sequences) and associated mathematical ideas (e.g., identities, inverses, conjugates, norms, etc.). One question we must consider is to what extent the research variables we measure can be effectively modeled using these various ideas.

Categorical and Ordinal variables

Many variables we interact with consist of a set of distinct categories, each (usually) associated with a name. These categorical variables—also known as nominal variables—have a finite (and usually very small) number of levels, with no particular relationship between the levels. If a categorical variable has only two levels, it is called a binary or binomial variable; More than two categories make a multinomial variable.

For categorical variables, the only meaningful mathematical operators are \(=\) and \(\neq\)—levels can be the same or not, that’s it.

In some cases, categories have a defined order, making an ordinal variable. For example, the sets \(\{Cold, Lukewarm, Warm, Hot \}\) and \(\{Strongly\ disagree, Disagree, Agree, Strongly\ agree \}\) are ordinal.

For ordinal variables, \(>\), \(<\), \(\geq\), and \(\leq\) are all meaningful (in addition to \(=\) and \(\neq\)). We can say that one level is greater than another. However, in an ordinal variable, no “distance metric” is defined. This means that mathematical tools like \(+\) and \(-\) are not meaningful. So, if we have an ordinal set like \(A < B < C < D\):

  • \(B - A \neq C - B \neq D - C\)
  • \(C - A \neq D - B\)

The “distance” (or difference) between levels is not a meaningful concept.

Case: Rank Data

Many behavioral studies ask participants to rank choices, and then code their ranked ordering as numbers. For example, we might ask a participant to…

Rank the Beatles from the most loved to least loved.

A participant might respond \(John = 1\), \(Paul =2\), \(Ringo = 3\), \(George = 4\). It seems reasonable to treat these like numbers right? But how about a slightly different question?:

Rank these artists from greatest to worst: The Beatles, Led Zeppelin, Michael Jackson, The Baha Men, Justin Beiber.

A participant might respond \(The Beatles = 1\), \(MJ = 2\), \(Led Zep = 3\), \(Beiber = 4\), \(Baha Men = 5\). But does our participant really think the distance between 3 and 4 is the same as the distance between 1 and 2? It seems unlikely! Thus, this rank data is probably best treated as an ordinal scale!

Numeric variables

Categorical and ordinal variables aren’t quite “numbers” in the sense we normally expect. In contrast, many variables do behave like numbers, but exactly what we mean by “number” needs to be more specific. We can break this down into four dichotomies:

  • Interval vs ratio scales
  • Discrete vs continuous numbers
  • Bounded vs unbounded ranges
  • Linear vs non-linear relationships

Interval vs Ratio

Unlike an ordinal scale, on an interval scale \(+\) and \(-\) are defined. This means that we can calculate consistent “distances” between any two values.

  • \(A < B < C < D\)
  • \(B - A = C - B = D - C\)
  • \(C - A = D - B\)

This is more like what we normally think of as “numbers.” However, on an interval scale, there is no meaningful zero value. This means that \(*\) and \(\div\) are not meaningful.

Consider the example of the Fahrenheit scale:

  • \(40^{\circ}F - 30^{\circ}F = 50^{\circ}F - 40^{\circ}\)

Subtraction makes sense. Check. However:

  • \(40^{\circ}F * 2 \neq 80^{\circ}F\)

Eighty degrees isn’t really “twice as hot as 40 degrees.” This is because \(0^{\circ}\) is not really a meaningful zero on the scale. Temperature (in Farenheit or Centigrade) is an interval scale.

In contrast, a ratio scale is a numeric scale where there is a zero, which makes multiplication/division meaningful. For example, consider the Kelvin scale, where \(0^{\circ}K\) is really absolute zero. \(100^{\circ}K * 2 = 200^{\circ}K\) is a valid statement, because Kelvin is a ratio scale.

Discrete vs Continuous

Another consideration is whether numeric values can take on any value—like “continuous” real numbers (\(\mathbb{R}\))—or only specific values—like “discrete” integers (\(\mathbb{Z}\)) and natural numbers (\(\mathbb{N}\)). In reality, we often work with constructs that are theoretically continuous, but use discretized operational definitions. For example, we might use a “number between 1 and 100” to represent a construct like “how much pain are you feeling”; In theory, somebody might be experiencing “17.923” units of pain, but we’d probably measure that as 18, so the measured scale is actually discrete.

If you want to be pendantic, all numbers represented by computers are discrete. However, the floating point representation used by computers is a good enough approximation of continuous real numbers for most purposes.

Bounded and Unbounded

We must also consider whether a numeric variable has bounds: is there a maximum or a minimum value? Many common measures are bounded:

  • General examples
    • Proportions (and probabilities) are bounded between 0 and 1 (inclusive);
    • Counts have a lower bound at 0 (inclusive);
    • Ranks have a lower bound at 1 (inclusive);
    • Logarithms of numbers have a lower bound of zero (excluding 0).
  • Specific examples
    • MIDI pitch values are bounded between 0 and 127.
    • Perceived acoustic frequencies—as percieved by humans—are bounded between 20 and 20,000.
    • Likert scales are always bounded.
Case: Count Data

A common type of discrete numeric value is counts: counting how many times something occured in a sample. For example, how many notes in a score, how many tracks in a playlist, or how many users clicked a link. Counts are necessarily discrete numbers, with a lower bound of zero, and are well modeled by the set of natural numbers \(\mathbb{N}\).

Linearity

A final consideration for dealing with numeric variables is whether they can be treated as linear or not. For example, consider the construct “frequency” (Hertz). Frequency seems to be a fairly straightforward rational scale:

  • \(100_{Hz} < 200_{Hz} < 300_{Hz} < 400_{Hz}\)
  • \(300_{Hz} - 100_{Hz} = 400_{Hz} - 200_{Hz}\)
  • \(200_{Hz} * 2 = 400_{Hz}\)

But wait! We are generally interested in how humans percieve frequency. If you ask a human to judge the distance between frequencies they will tell you that:

  • \(300_{Hz} - 100_{Hz} \neq 400_{Hz} - 200_{Hz}\)
  • \(200_{Hz} - 100_{Hz} = 400_{Hz} - 200_{Hz}\)

The “distance” between \(200_{Hz}\) and \(100_{Hz}\) is an octave. The “distance” between \(400_{Hz}\) and \(200_{Hz}\) is also an octave. Thus, we perceive the ratios between frequencies as their distance, not their linear difference. If I multiply a frequency by 2, the result isn’t “twice as high”—it always increased by a fixed amount (an octave). Thus, the perceptual construct of frequency is logarithmic.

If you take the logarithm of frequencies, they are translated onto a linear interval scale. Other “numbers” we measure might best be modeled on logarithmic, linear, quadratic, cubic, etc. scales, as well. In these cases, it can be appropriate to apply transformations (like logarithms) to the numbers to make them behave more like the constructs were are modeling.

In mathematics, there are (at least) two definitions of linearity. Here, I’m talking about “polynomial linearity”—“functional linearity” (the linearity of relationships between variables) is another, equally important topic.

Case: Distances

Above, I mentioned that “distance” is a consistent property of numbers on interval and ratio scales. Measuring “distances” between variable levels can take on many forms: we call these distance metrics. We are all familiar with basic distance metrics for real numbers (\(\mathbb{R}\)) in one or more dimensions (Euclidean distance). Other variables we work with may, or may not, have their own distance metrics.

For example, we might consider how we’d measure the distance between musical keys. One well known approach is to count the number of non-common tones between the keys:

  • Between A minor and G major, the distance is one;
  • Between A minor and C major, the distance is zero;
  • Between G major and B major, the distance is four;
  • Between E minor and Bb major, the distance is three;
  • etc.

But this isn’t the only possible distance metric for key. For example, you might want to consider parallel keys (like C major and C minor) to be closer together, and you could devise a different distance metric to represent this. How humans percieve “distance” between keys is an open question! Human perception of key-distances is a construct that is poorly understood, so we don’t know what the best operational definition for it is. Studying perceptual distance metrics is itself a large field of study.

Note that we can define a coherent distance metric for tonal keys even though it’s not clear how to order them! Is G major above or below C major? This illustrates that many mathematical concepts can be mixed and matched to appropriately model a variable.

Quantifying Error

Real-world measurements are usually affected by many processes, not just the intended construct. Even if our operational definitions are valid representations of our targeted theoretical constructs, the real world is chock full of other phenomena that also affect the measurement. These “nuisance” variables introduce random error into our measurements. Every time we measure, we get a (slightly) different result—random error. How we quantify that error must depend on the nature of our variables.

There are two main aspects of error to consider:

  • The magnitude of error (the inverse of accuracy);
  • The direction and consistency of error (the inverse of reliability).

Categorical error

Categorical variables can be characterized by accuracy: what proportion of measurements are correct. We can also count which specific categories are confused with each other, which corresponds to the “direction” of error.

For example, imagine we are working on a genre-classifying algorithm. We can feed a bunch of input tracks for which we know the correct genre and tally the algorithm’s output for each input:

Labeled Jazz Labeled Pop Labeled Punk Labeled Classical Labaled Film
True Jazz 56 15 0 4 10
True Pop 5 45 5 2 8
True Punk 0 9 78 0 1
True Classical 23 2 0 62 56
True Film 30 0 0 56 64

This is called a confusion matrix. Correct answers appear on the diagonal line; all other positions are errors. Overall, our classifier is correct 57.4% of the time. However, by looking at the confusion matrix, we can see specifically that the most common errors involve Classical being labeled as Film, and vice versa. We can also see that Punk is, overall, measured most accurately. So the magnitude and direction of error depends on which categories we are measuring.

If a variable is binary, the confusion matrix is very simple. A particularly important example, which we’ve seen before, is the “true or false” confusion matrix:

Measured true Measured false
Actually true True positive False negative
Actually false False positive True negative

There are a number of other metrics used to quantify the accuracy of binary measurement (binary classification), which we will return to in our classification chapter.

Numeric error

To quantify error in numeric variables (on linear interval or ratio scales), we can consider how far, on average, measured values are from the true value. Consider again our thermometer measurements from the measurement chapter.

  • Thermometer 1: \(99.6^{\circ}_F\), \(99.7^{\circ}_F\), \(99.6^{\circ}_F\), \(99.8^{\circ}_F\).
  • Thermometer 2: \(99.9^{\circ}_F\), \(98.5^{\circ}_F\), \(102.1^{\circ}_F\), \(100.4^{\circ}_F\)
  • Thermometer 3: \(100.8^{\circ}_F\), \(100.5^{\circ}_F\), \(101.6^{\circ}_F\), \(100.9^{\circ}_F\)

We can calculate the average deviation (difference) between each reading and the true value (\(100.6^{\circ}F\):

  • Thermometer 1: \(\frac{(99.6 - 100.6) + (99.7 - 100.6) + (99.6 - 100.6) + (99.8 - 100.6)}{4} = -0.9\)
  • Thermometer 2: \(\frac{(99.9 - 100.6) + (98.5 - 100.6) + (102.1 - 100.6) + (100.4 - 100.6)}{4} = -0.4\)
  • Thermometer 3: \(\frac{(100.8 - 100.6) + (100.5 - 100.6) + (101.6 - 100.6) + (100.9 - 100.6)}{4} = 0.4\)

These results match our observations from before: The first thermometer has the largest, negative, bias. The other two measures are more accurate.

The average deviation of a measure is useful to characterize the direction of error. However, it is not the best way to characterize the magnitude of error. For example, these numbers make it appear that thermometers 2 and 3 are equally accurate. Instead we might use the average absolute deviation:

  • Thermometer 1: \(\frac{|99.6 - 100.6| + |99.7 - 100.6| + |99.6 - 100.6| + |99.8 - 100.6|}{4} = 0.92\)
  • Thermometer 2: \(\frac{|99.9 - 100.6| + |98.5 - 100.6| + |102.1 - 100.6| + |100.4 - 100.6|}{4} = 1.12\)
  • Thermometer 3: \(\frac{|100.8 - 100.6| + |100.5 - 100.6| + |101.6 - 100.6| + |100.9 - 100.6|}{4} = 0.4\)

This shows us that, on average, thermometer 2 is actually further off than thermometers 3 and 1.

The average absolute deviation is intuitive, but it is not the best way of measuring numeric dispersion. The most common approach is to instead measure the squared deviations. Since squared numbers are always positive, the squared deviations are directionless, like the absolute deviation. However, squared deviations have useful mathematical properties, which we will learn more about in our goodness-of-fit chapter.

  • Thermometer 1: \(\frac{(99.6 - 100.6)^2 + (99.7 - 100.6)^2 + (99.6 - 100.6)^2 + (99.8 - 100.6)^2}{4} = 0.86\)
  • Thermometer 2: \(\frac{(99.9 - 100.6)^2 + (98.5 - 100.6)^2 + (102.1 - 100.6)^2 + (100.4 - 100.6)^2}{4} = 1.8\)
  • Thermometer 3: \(\frac{(100.8 - 100.6)^2 + (100.5 - 100.6)^2 + (101.6 - 100.6)^2 + (100.9 - 100.6)^2}{4} = 0.29\)

The average squared deviation is often called the mean square error (MSE), which is similar to the statistical variance. Here, we see that thermometer 3 has by far the smallest variance.

Since the MSE is squared, it is not on the same scale as the original data, which can be unintuitive. It is thus common to “undo” the squaring by taking the square root of the MSE, putting the result back on the same scale as the original deviations.

  • Thermometer 1: \(\sqrt{\frac{(99.6 - 100.6)^2 + (99.7 - 100.6)^2 + (99.6 - 100.6)^2 + (99.8 - 100.6)^2}{4}} = 0.93\)
  • Thermometer 2: \(\sqrt{\frac{(99.9 - 100.6)^2 + (98.5 - 100.6)^2 + (102.1 - 100.6)^2 + (100.4 - 100.6)^2}{4}} = 1.34\)
  • Thermometer 3: \(\sqrt{\frac{(100.8 - 100.6)^2 + (100.5 - 100.6)^2 + (101.6 - 100.6)^2 + (100.9 - 100.6)^2}{4}} = 0.53\)

This “square root of the mean squared deviation” is usually called the root mean square (RMS), or more commonly in statistics, the standard deviation.