22  Constructs

(Modified on 2026-08-11)
3,488 words

Knowledge is a model of reality. The elements of that model are constructs—abstract, a priori theoretical concepts. “Melody” is a mental construct; “Groove” is a mental construct. “Jazz” is a mental construct. To engage in meaningful scientific research and rhetoric, we need to define the properties of the constructs we are considering and articulate these ideas within the scope of our research. A theoretical definition is a description, for the purposes of our research, of the constructs involved in our theories.

A theoretical description is necessarily a model—a simplified, toy version—of the “real” things we want to understand. To make them manageable to study, we’ll often use extremely simplistic theoretical definitions. We might, for example, define emotion using a handful of broad categories, like “happy,” “sad,” “angry,” or “fearful.” This is overly simplistic (there are many other emotions) but it may be suitable for our immediate purposes in the context of a research study.

Other common, simplifying assumptions we make about theoretical constructs include:

In contrast, some constructs—especially the really interesting ones—are complex and multi-dimensional, often constructed from other “sub”-constructs. The construct “melody,” for example, might be modeled as a combination of other constructs like “contour,” “rhythm,” and “tonality.” Richer models afford more nuanced understandings of the universe, but are also more difficult to study.

Consider the construct “happiness.” What are some things we’d need to consider to study happiness? Is happiness either on or off (binary) or are there more levels? Can you have negative happiness? Is happiness really an idea in its own right, or is happiness simply one aspect/dimension of a more complex construct? In turn, we might ask if happiness is one unitary thing, or if itself might be multi-dimensional: maybe constructs like “comfort,” “satiation,” and “satisfaction,” are sub-dimensions of “happiness.” Is a degree of happiness something every human has at all times? How quickly can happiness change? Can you be happy and sad at the same time?

Phenomena

We are generally interested in constructs which are useful as models of external reality—constructs that model “things” that “exist.” We thus relate our abstract mental constructs to phenomena which we perceive (observe).

In many cases, the relationship between an abstract construct and a perceived phenomenon is so straightforward and obvious that we make the connection without thinking. It might be hard, or seem pointless, to separate the construct of “chair” from the phenomenon of “a chair,” or the construct of “rain” from the phenomenon. We can point to a chair, or feel the rain. However, even straightforward examples can get messy if you dig in to them: what exactly constitutes a “chair” vs a “stool”? Where is the bound between “rain” and “mist”? Do chains of crystallized carbon molecules (e.g., diamonds) tumbling through the lower atmosphere of Neptune count as “diamond rain”? What about dust blown into the air in the Sahara? Is that “sand rain”?

Many constructs, by their nature, are obviously abstract—like “happiness.” You can’t point to constructs like “emotional state” or “spectral centroid” in the same way you can rain or chairs—their existence can only be inferred through observation, as they influence other constructs which are in turn associated with observed phenomena. We often refer to these more abstract constructs as latent or “unobservable” variables.

Measurement

In order to engage in empirical induction, we need to relate phenomena that we observe to theoretical constructs in a consistent way. We call this measurement. Measurement is a procedure which associates observations with various levels of a theoretical construct (variable). A formal measurement procedure is called a protocol.

Any measurement protocol necessarily imposes a model on the construct we are measuring. We call this an operational definition—a working model of the construct that we can actually observe and/or manipulate.

An operational definition must be a specific, concrete procedure. For example, we might operationally define “musical enjoyment” by asking listeners to “rate their enjoyment” on a 7-point scale. In this example, our operational definition of “musical enjoyment” is a one-dimensional ordinal variable with seven levels, measured by asking human participants to rate their enjoyment. In our operational protocols, we aim to produce a mechanical procedure that anyone could follow, which results in a clear and repeatable measurement.

The “structure” of operational variables should generally match our theoretical models of constructs; e.g., dimensionality, number of levels, etc. However, an operational definition is always an approximation of the true construct. Compared to our true understanding of a construct, an operational definition is typically simpler, narrower in scope, more convenient, and temporary. An operational definition, allows us to define the meaning of what we are studying within the scope of our current project.

Operational definitions are often limited, as much as anything else, by feasibility. Measuring “musical enjoyment” on a 7-point scale is pretty simplistic, but it is easy to do quickly.

What sorts of measurements do we make in the Georgia Tech Center for Music Technology? Some major categories include:

  • Musical “features” (from audio or scores).
    • Spectral centroid; pitch histograms; auto-correlation matrices; etc.
  • Musical metainformation.
    • Recording release dates; Billboard chart position; number of likes on YouTube; etc.
    • Demographic information; years of musical training; etc.
  • Human behaviors and responses.
    • Felt emotions; EEG signals; memory abilities; musical judgements; etc.
  • The behavior/performance of non-human systems (software or hardware).
    • Accuracy of predictive models; system response latency; frequency response; etc.

In the next month we will learn about common operational definitions used to measure these sorts of constructs.

Operationalizing a hypothesis

Recall that a hypothesis must include a falsifiable prediction. To be falsifiable, our prediction can’t be about a construct. An abstract idea cannot be tested—only a concrete observation can be. Thus, a proper, falsifiable hypothesis must include a precise, specific, concrete, operational definition of what you predict.

I might hypothesize that “violinists use more vibrato than cellists.” But to test this hypothesis, I need an operational definition for “amount of vibrato.” I might define an operational protocol like this:

  1. Record audio of violin and cello performances using Ableton Live.
  2. Use Live’s existing algorithm to find the fundamental frequency contour and export it.
  3. Using Python, find any audio segments where the F0 contour changes direction more than five times within a \(500_{ms}\) window while staying within a half-semitone range.
  4. Excluding silence, measure what proportion of the F0 contour falls in one of these segments.

Notice that intelligent people can disagree about what makes a good operationalization. Especially rigorous research will identify two or more ways of operationalizing a hypothesis. One operationalization might be considered rather narrow (looking at a single composer, or single style, or looking at notated scores). Another operationalization might be considered rather broad (sampling with very loose criteria, having judges listen to recordings, etc.). The most impressive studies will seek converging evidence by presenting two or more tests of the hypothesis, where each test operationalizes the terms differently.

Some terms are easier to operationalize than others. A protocol to count the number of people attending a concert could be very straightforward: stand by the venue entrance and count people who pass in. However, refinement of the counting procedure is often necessary. Suppose for example that you are counting “the number of notes in a work” from a printed score. You might decide to exclude repeat signs but include a Da Capo return. You might reasonably exclude “tied-to noteheads”—but what about groupettos and grace notes? And what about trills and other ornaments? Some clarification will be necessary. How you settle on an operationalization will depend on the research goal and on the amount of labor required.

All details of your operational definitions should be written in your protocol before any measurements are made (Write the paper first). When operationalizing terms, try to anticipate the kinds of problems that might arise, and define your terms accordingly. Even so, once the measuring begins, you are apt to encounter situations which will require further refinement to the operational definitions. If changes are made to a protocol, data gathered using the new protocol should be separated from the old data (at least, potentially).

Human Judgement

In many cases, it will seem impossible to define a “mechanical” procedure that will always produce unambiguous results. In these cases, it may be necessary to rely on your, or preferably someone else’s, judgement. Most “ground truth” measures in machine learning datasets are based on human ratings. For example, many datasets indicate the tonal key (e.g., A major), which is typically “measured” by asking an expert. Even in 2026, many complex musical phenomena can still only be measured accurately by asking humans.

For some measurements, a researcher might provide the measurement themself, based on their own judgements. For example, in my dissertation work, I transcribed the vocal parts of 124 rap songs, following a rigorous protocol (transcription scheme) that I developed. However, reviewers may be wary of measurements made by a researcher, especially in hypothesis-driven work where the researcher might unconsciously bias the measurements to support the hypothesis. It is thus more common to recruit independent raters to provide measurements, with the protocol including detailed instructions for the raters. By keeping the hypotheses and research aims secret from raters, we can lessen the risk of them biasing the measurements.

In many cases, any competent adult might be capable of providing coherent measurements of the construct we are interested in. For example, if we are measuring “grooviness,” any music listener could make a valid judgement. However, in other cases, lay people may not be able make a valid judgement, at least explicitly. In these cases, we might instead recruit “experts”—an expert music theorist to judge key, or a professional jazz drummer to measure “groove”, etc. Of course, experts may have their own biases.

Unfortunately, in many musical judgements, even expert opinions can diverge considerably. For example, although it is relatively easy to determine whether pieces in the 18th century exhibit a “sonata-allegro” form, this becomes much more contentious in music from the 19th century. If a researcher needs to know whether a given movement is in sonata-allegro form, it may be necessary to poll the opinions of several expert music theorists. In this case, one could operationalize “in sonata-allegro form” as “those movements that a minimum of 4 of 5 PhD music theorists claim to conform to sonata-allegro form.”

All concepts ultimately prove to be difficult to “pin-down.” If that’s not possible, human “opinions” can be used as needed, but guided as much as possible by detailed instructions.

Construct Validity

When writing a scientific paper, we need to convince our readers that our constructs, as we present them, are meaningful and useful models of the universe—at least, within the scope of our research questions. We in turn need to convince them that our actual operational definitions are valid representations of those constructs. These are arguments of construct validity. Note that there is no one-size-fits-all argument for construct validity—it will depend on the exact research claims you want to make. A simplistic operational definition of emotion might be suitable for some studies, but not others.

In human-participant studies, we gather demographic data about our participants, including gender. “Gender” is a construct. We might measure gender by asking participants to tick a box to indicate either “M” or “F.” This imposes a binary operational definition of gender. Is this operational definition a good model of the construct of gender? Perhaps not. Is it a “good enough” operational definition of gender? If gender isn’t really that important to our theories, perhaps. But if gender is important to our research, we’d probably want to consider how other possible gender levels might fit into our theories and hypotheses. How will our simplistic gender definition behave when used in the real world? For example, if a participant identifies as neither male or female, how will they respond? Of course, even if we want to include more gender levels, we might not be able to recruit many participants who would use them. For example, you might recruit 34 participants, and get 19 women, 14 men, and 1 non-binary person. In this case, is the more sophisticated operational definition actually useful?

Assessing Validity

As skeptics, a question we should always ask ourselves is: are we really measuring what we think we are measuring?

  • We might aim to measure “musicality,” but really measure “performance skill.”
  • We might aim to measure “feeling sad,” but really measure “the degree to which someone is willing to admit they feel sad.”
  • We might aim to measure “intelligence,” but really measure “language facility” or “logical reasoning.”

How can we avoid such mistakes? How do we determine, or make a case for, construct validity?

The most basic approach to claiming construct validity is simply through intuition: Does our measure seem intuitively valid, “at face value”? We call this face validity. For example, I could suggest that we measure happiness by counting how often a participant smiles during an interview. This might not be the best measure, but it seems reasonable enough. Of course, its not enough for a measure to be intuitive to you—you’ll need to convince your readers.

Imagine if a researcher studying emotion proposed the following operational definition for “felt emotion”: ask a music listener to indicate the emotion they are feeling using a slider with the following labels: “angry”–“sad”–“happy.” This is a poor operational definition because placing anger and happiness at opposite ends of a spectrum with, sadness in between them, is obviously a poor model of human emotion. This is an example of bad content validity. The content of an operational measure should reflect the essential aspects of the theoretical construct.

To make stronger arguments about construct validity, we can look for empirical evidence that our operational definitions are valid. All empirical arguments for construct validity essential try to show that two or more measures are consistent with each other. Here are some examples:

  • Predictive validity: Does your operational definition help predict future observations?

    The classic example is something like the Graduate Record Examination (GRE). If students with higher GRE scores are ultimately more likely to get good grades, graduate, and get better jobs after graduation, this demonstrates the predictive validity of the GRE.

  • Concurrent validity: Does the measure correlate with measures of different, but related, constructs?

    Let’s say you are developing an operational measure of musical “heaviness” and I am developing a new signal-processing algorithm to detect overdriven electric guitar in audio mixes. These two constructs (heaviness and distorted guitar) should be correlated, so our measures should be as well.

  • Convergent validity: Does the measure correlate with other measures of the same construct?

    We could measure the construct “heaviness” in two ways: 1) by observing the frequency of headbanging or 2) by measuring how angular guitar bodies are. These two measures should correlate.

  • Discriminant validity: Does the measure not correlate with measures it shouldn’t?

    “Heaviness” should not correlate with use of electric guitars in general, since lots of non-heavy music also uses electric guitars.

Error

Even if you have a valid operational definition, measurement procedures always introduce error. Measurement error is a difference between the measured level of a variable and the “true” level. Since we can never directly observe or know The Truth, only our measure of it, we can never be certain what is error and what is real. We can, however, model the error in our measurements, which helps us make good guesses about what is error and what isn’t—this is the central concern of statistics, as we’ll learn in later chapters.

Error may be random or systematic: Random error is typically caused by the effects of unknown covariates, but doesn’t correlate with anything we care about. We can hope to account for it by “averaging out” samples of data. In contrast, systematic error—or bias—is a more serious problem, as it represents a distorted model of the truth. If an operational definition is not representing a construct the way we want it to, the result can be systematic error.

Measure Quality

There are two ways to characterize the quality of a measure. One is accuracy: how often the measure assigns the observation to the correct level, or how “close” it is to the correct level, on average. The other is reliability: how consistent is the measure? These two characterizations are totally independent:

  • A measure may be reliably inaccurate; For example, my dogs very reliably anticipate dinner time…half an hour early. The dogs make a systematic error when judging when their dinner will be.
  • A measure can also be accurate but unreliable: The measure gives a different result each time, but the results are close to the truth. My at-home medical thermometer will often return three different values if you use it three times in a row, but they will all be close to the true value—e.g., \(99.8^{\circ}_F\), \(99.7^{\circ}_F\), \(99.77^{\circ}_F\). There is persistent random error, but the errors are small.

Assessing or modeling accuracy is always difficult, because we never know for sure what the “truth” is. In contrast, reliability can be assessed more easily, since it only concerns consistency of measurement, not “the truth.” Reliability relates to the consistency of the score; A test is said to be reliable when a person scores the same value over time in similar circumstances. Sometimes the reliability has to do with the consistency of the raters. For example, the Royal Conservatory of Music (in Canada) has a graded system for musical performance, ranging from grade 1 to 10. Performance examinations are held across the country twice each year. The RCM trains examiners in an effort to make the grading criterion as consistent as possible, so that the final score received doesn’t depend on who the examiner is. If the adjudication method varies from rater to rater, then the instrument is said to have a low reliability.

Common approaches to assessing the reliability of measures include:

  • Test-retest reliability: Measure something twice and see if the measurements are consistent.

    It is common to include some questions/tasks twice in the same behavioral experiment to see if the participant responds reliably. When testing code, you might run it a few times to make sure it produces the same result each time.

  • Inter-rater reliability: Compare measurements from different sources.

    In musical performance competitions, we might ask three different judges to independently assess the quality of each performer, so we can see how reliable “expert judgement” is. Similarly, many machine learning “ground truth” datasets incorporate ratings from three or more experts.

  • Internal consistency reliability: In a multi-dimensional measure, compare different dimensions with each other.

    In an exam, we can measure how each question correlates with the overall score. For example, if everybody who answers question 6 correctly also gets question 7 correctly, this suggests some reliability.

As an example, let’s say that I have a mild fever and my true body temperature is \(100.6^{\circ}_F\). We compare the readings of three different thermometers, taking four readings from each so we can evaluate test-retest reliability.

  • Thermometer 1: \(99.6^{\circ}_F\), \(99.7^{\circ}_F\), \(99.6^{\circ}_F\), \(99.8^{\circ}_F\).
  • Thermometer 2: \(99.9^{\circ}_F\), \(98.5^{\circ}_F\), \(102.1^{\circ}_F\), \(100.4^{\circ}_F\)
  • Thermometer 3: \(100.8^{\circ}_F\), \(100.5^{\circ}_F\), \(101.6^{\circ}_F\), \(100.9^{\circ}_F\)

Of these three, Thermometer 1 gives the most consistent, reliable readings. However, they are not very close to the true temperature; This thermometer appears to be systematically biased to underestimate temperature—it’s reliable but inaccurate. In contrast, Thermometer 2 is unreliable, giving a wide range of responses—however, it is not clearly biased, as its readings do surround the true value. Thermometer 3 is both reliable and accurate.

Next, consider the construct “intelligence” and its most well-known operational definition, the intelligence quotient (IQ). The IQ measure is based on a number of assumptions about the construct “intelligence.” For one, IQ researchers assume that humans have a single unitary property called “general intelligence.” There may be other types of intelligence as well (spatial intelligence, mathematical intelligence, social intelligence, etc.) but IQ is focused only on “general intelligence.” The IQ measure also assumes people’s general intelligence is a fixed parameter: your IQ should stay the same throughout your life.

Modern IQ measures have seen decades of development, and have been extensively tested for reliability and accuracy, based on their assumed model of intelligence. The IQ measure is reliable. But is the construct useful? That is a separate question. Personally, I find any model of “intelligence” that purposely ignores education, skills, or knowledge gained in your lifetime to be dubious. IQ definitely measures something consistently, but is it really measuring “general intelligence”? I don’t know.

Reactivity

In some cases, the act of measurement affects the thing being measured. This is called measurement reactivity—the phenomenon “reacts” to being measured. The most obvious example is human participants, who will change their behavior in an experimental context. If you, for example, bring a video camera to measure musicians while they rehearse, the musicians will likely behave (and perform) differently than if they weren’t being filmed. More generally, it is well known that bringing a person into a lab—an unfamiliar and strange place—can affect their behavior. A favorite song that evokes powerful feelings when you listen at home might sound boring when you are sitting in a lab with a researcher looking over your shoulder. This is an area where observational studies (outside the lab) have an advantage, as you are much less likely to influence the measurement.

One approach to avoiding measurement reactivity is through clandestine observation. The invisible observer lurks in the background like the proverbial “fly-on-the-wall,” hoping not to influence or change what is going on. In most studies of animal behavior, the ethologist tries to observe clandestinely. For example, an ornithologist might build a blind in order to hide her/his presence. However, for some species, it may prove impossible to disguise your presence. Bonobos and chimpanzees, for example, are far too attentive and curious to allow a researcher to observe them in the wild without their awareness. As you might guess, human beings are also difficult to observe without changing their behavior. You might think you are being subtle and clandestine, but people will notice!

Of course, technology can help us engage in effective clandestine observation. For example, in a study of audience behavior, Olaf Post (2011) made use of video recordings made from security cameras in the Concertgebouw hall in Amsterdam. This allowed Post to observe members of the audience with minimal influence. Written documents can provide similar opportunities for “invisible” observation: when a theorist examines a published score, the observations made will have little or no impact on the behavior of the composer who created the score. Finally, observing public behaviors on social media or other internet venues can be done with little risk of reactivity.

Consider again our discussion of an operational definition of gender. One disadvantage of using a binary gender definition is that it could offend participants with different gender identities; they might react (consciously or unconsciously) differently to other questions as a result. On the flip side, if you were doing research with participants from a highly conservative society, they might react negatively to seeing “non-binary” listed as a possibility on a questionnaire.

Mantras

Splurge to converge

Every study is imperfect in some way. The best way to make a strong research claim is not to make your method/data/analysis absolutely perfect, but to investigate the same topic in two (or more) independent ways. Two different operational measures of a variable; two different experiments; two different evaluation metrics; etc. If multiple separate studies/approaches are consistent with an idea, this provides converging evidence, which makes for a stronger argument than any single piece of evidence.

You can’t falsify the fuzzy

To falsify a hypotheses, we need to make a specific prediction about what we will measure. This means we need precise operational definitions of exactly how (research protocol) we will measure the constructs we make predictions about.