42  Sampling Music

(Modified on 2026-08-12)
2,099 words

Whether we set out to study musical behaviors or concrete musical objects like scores or audio recordings, a fundamental question is: which music do we sample? What is the population we are interested in?

Recall that the “population” is the thing we want to make research claims about. Depending on the nature of our research questions, there are many possible musical populations that might be appropriate for us. There are two aspects to consider: 1) What are the “musical objects” we want to make claims about? 2) What is the population of those objects which we care about? Some examples might include:

Of course, much of the time we will necessarily work with convenience samples of score data, or audio recordings that we can access. Still, to make valid claims that generalize beyond our data (external validity), we need to consider what our samples represent.

Elements of the Population

One fundamental thing to consider, as we’ve discussed already, is whether we want to generalize about musical performances (sound) or musical pieces. If you sample a large number of commercial audio recordings, you might find a bunch of samples like this:

  • Smooth Criminal (1988), recorded by Michael Jackson
  • Smooth Criminal (2001), recorded by Alien Ant Farm
  • Smooth Criminal (remastered, 2009), Michael Jackson
  • Smooth Criminal (Live at Hollywood Bowl, 1995), Michael Jackson

These recordings are all samples of the same piece, but they represent three different performances, and four different recordings. So depending on what we are working on, we might consider these repeated measures, or not.

Another way to think about this is whether we want to make claims about musical production or about musical consumption.

If we are studying the composed elements of music—like chord progressions, melodies, or lyrics—then these four recordings of Smooth Criminal are all duplicates of the same piece. On the other hand, if we are studying performance details—like vocal inflections, drum fills, or timbres—then the Michael Jackson and Alien Ant Farm recordings are essentially unrelated data points; the live version by Michael Jackson might be similar but not identical to the studio release. If we are studying music production—like EQ and mastering—then all four recordings are distinct samples. All of these examples have to do with the production (creation) of music.

Studying music consumption—including buying and/or listening to music—is very different than studying music production. If we are studying record sales, then all four versions of Smooth Criminal are separate objects. What makes things really complicated is that, if we are studying music listening experiences, then these four versions have almost certainly been listened to millions of times each, in many different listening contexts. Instead of viewing these recordings (or the piece) as single data points to sample, we could look up weekly record sales of these tracks, or inspect these tracks’ listening histories on Apple Music or YouTube.

Sampling Criteria

In much data-driven music research, two main overarching sampling perspectives have tended to dominate. We either aim to sample great music or sample popular music. Generally, it is assumed that popularity and greatness might be at least somewhat correlated—but that notion is extremely debatable. Our lab’s CoCoPops dataset (Arthur and Condit-Schultz 2023) illustrates these two main approaches to sampling music (greatness vs popularity). CoCoPops is composed of two main subsets: The Rolling Stone subset (de Clercq and Temperley 2011), a sample of “great” music, and the Billboard subset (Burgoyne, Wild, and Fujinaga 2011), a sample of popular music.

Great Music

The study of “great music” tends to be associated with studies of musical compositions, pieces, and musical organization in general. After all, we often want to know what makes great music…great; or at least, what makes some music good, and some bad. For example, studying Beethoven’s piano sonatas or Charlie Parker solos, so we can learn what makes them so awesome. Targeting “great music” is very obviously subjective, and it is difficult to do in a way that avoids obvious sampling biases. Generally, you shouldn’t just pick music that you personally think is great—this will be viewed skeptically by any reviewer. A better way to operationalize “great music” is to rely on independent sources (disconnected from you, the researcher), and ideally collate multiple sources.

When David Temperley and Trevor de Clercq sought to create a dataset of popular music (2011), they decided to sample from Rolling Stone magazine’s widely publicized list of The 500 Greatest Songs of All Time. Rolling Stone created this list by polling a large number of music critics and industry insiders. Rolling Stone and its list contributors represent a credible body of “music experts.” By offloading the sampling decisions to Rolling Stone, Temperley and de Clercq avoided biasing their data with their own personal beliefs about musical greatness. An even better approach might be to cross-reference multiple critical sources, instead of relying on only one.

Control

A major weakness in many projects is that scholars often fail to sample appropriate controls for their samples. For example, if we only study a bunch of great music, how can we make any conclusions about what makes music great? We need to be able to compare great music to mediocre music, or bad music! Classical music research suffers badly from this problem, because only great music from the 18th and 19th centuries is well preserved or well known.

Making the effort to sample less-well-known contemporaries of Bach or Haydn, is rarely done. Temperley and de Clercq (de Clercq and Temperley 2011) made no effort to include any “bad” music to compare with Rolling Stone’s list of great songs. Burgoyne et al.’s (2011) careful sampling of different ranges of the Hot 100 chart is one of the better examples of control in pop music sampling. However, even charting at position \(\approx95\) on the Hot 100 chart is an exceptional achievement—unpopular music never charts at all!

Unfortunately, finding a consistent criterion to sample “unpopular” music, comparable to a sample like Billboard, is not easy. Sampling “not great” music is also challenging—should a control sample be full of “mediocre” music, or “bad” music? One good approach is matched random sampling: for example, first sample “great” or “popular” music, then search for comparable controls to match with them. A nice way to do it is to compare hit singles with obscure album-tracks from the same artist:

  • Enter Sandman (Metallica) could be compared with The Struggle Within.
  • Oops!…I Did It Again (Britney Spears) could be compared with One Kiss From You.
  • Shape of You (Ed Sheeran) could be matched with Dive.

It might also be possible to pair hits, or critical “greats,” with other similar songs released at the same time.

With or Without Replacement

On a weekly chart like the Billboard Hot 100, the same recording can appear on the chart multiple times, whether in consecutive weeks or years apart (it happens). Since Burgoyne et al. targeted weekly chart positions, songs that charted for multiple weeks could be (and were) sampled more than once. Indeed, of the total 890 successful samples 113 songs were sampled twice, and 19 were sampled thrice. If we allow “the same” element to be sampled more than once, this is called sampling with replacement—each sampled element is “put back in” (replaced in) the sample space, so it can (possibly) be sampled again.

Most analyses of the Billboard dataset remove the duplicates, leaving only the 739 unique tracks. But, including the replacements would actually reflect the intention of the sample: if a song keeps charting, it’s because people are still buying it (and presumably listening to it).

Consider another example: Spotify music “listens.” If we accessed data for what songs are being listened to on Spotify today, the top songs might have tens of thousands of listens, while other songs are listened to only a handful of times. If we want to represent the population of music listening experiences, we wouldn’t want to count a rare song and popular song as equivalent.

In contrast, if our goal is to represent composition today, it would make sense to treat each unique song that is composed as one sample, regardless of how many people heard it. This is called sampling without replacement—we do not allow duplicate samples.

Sticky Data

Musical datasets—symbolic and otherwise—often contain severely sticky data. This is especially the case when sampling is targeted towards “greatness.” Many projects have focused on sampling all the music by “great” artists, this is why many available datasets are samples of these artists’ entire output. For example, there are large bodies of works by Mozart, Bach, and Beethoven—The Beatles are similarly overrepresented!

Unfortunately, a sample of compositions by Bach is not a compelling representation of late-Baroque music in general. We can explore Bach’s music, and even test hypotheses using it, but claiming that these findings will generalize to other Baroque music is difficult. Popularity-based sampling tends to be less sticky; However, even in the McGill Billboard sample (Burgoyne, Wild, and Fujinaga 2011), 19 out of the 739 unique tracks are by the Beatles (either as a group or solo artists), 14 by Elvis, and 12 by the Rolling Stones. In fact, there are only 418 unique artists in the sample! This means that the effective sample size is really somewhere between 418 and 739 unique songs.

Data stickiness gets even worse in computational musicology (and music information retrieval). The Billboard subset of data CoCoPops (Arthur and Condit-Schultz 2023) includes 96,681 notes. However, as we learned in the Data Independence chapter, these notes are not statistically independent at all. They are clustered within each of the 214 CoCoPops pieces which include melodic transcriptions, and within each piece, note information is highly repetitive. This means that the effective sample size of notes is far less than 96,681.