Melodies and Metrics: Musical Attributes and the Stories They Tell

Yasiin Bey, formerly known as Mos Def, once advised that if “you want to know how to rhyme, you better learn how to add -- it's mathematics.” The music-as-math analogy extends to my university’s campus, where Penn allows both “Calculus I” and “Making Sense of Music” to satisfy the Formal Reasoning & Analysis requirement. Machine learning has even allowed a computer to imitate Bach. So, with the hope of better understanding my favorite artists and possibly even finding some new jams, I decided to analyze the data buried within my music.

A couple of existing datasets attempt to quantify a song’s sound. One such example, Pandora’s vaunted Music Genome Project, unfortunately keeps its data confidential. Luckily, in 2014, Spotify acquired a music data analysis company, The Echo Nest, and now makes available on its public API a set of automatically generated “audio features.” The list of audio features includes not only objective statistics such as duration and tempo, but also “attributes” that attempt to describe what the song actually sounds like. I used seven attributes for calculating similarities between songs: acousticness, danceability, energy, instrumentalness, liveness, speechiness, and valence. The metrics are scored between 0.0 and 1.0, and their descriptions are available in the Spotify Web API documentation.

Using Spotipy, a Python library for the Spotify Web API, I imported the discography of each artist, storing the audio features for every track. I analyzed a musician’s changes in style over time by comparing the musical attributes of their songs. I focused on a few of the most prodigious artists of all time (which also happen to be personal favorites) and investigated the stories of their careers.

Bob Dylan: Acoustic and Back Again

Who better to start with than the greatest songwriter of all time? Dylan led the American folk music revival of the early 1960s, before famously and controversially going electric in 1965. His first electric concert incited boos from the audience, and a fan notoriously shouted “Judas!” at him¹. Throughout his career, he has drifted across the boundaries of genres, showcasing his talents in country, rock, blues, gospel, jazz, and even Christmas music. While merely looking at the audio features could never do justice to his biting lyrics and their crucial political context, we can still observe some interesting trends.

The second visualization shows a heat map of the similarity scores of all Dylan’s albums in relation to one another. The approach I used was a sum of squares of differences between attributes on each song, so a lower score means the two albums are more similar. The method was inspired by this article from The Pudding. Blocks that use the same album on both axes describe how much all the songs on an album sound alike.

Compared to the rest of his discography, his three least-similar albums are Pat Garrett & Billy the Kid, Oh Mercy, and Shadows in the Night. Each of these albums also approximately signals a shift in the style of his broader catalog. Spotify scored his first decade — characterized mostly by folk and folk-rock ballads — as higher in acousticness and lower in energy. Lower acousticness and higher energy typifies the second block, stretching from 1974 to 1988, as he refined his heartland-rock sound. After 1990, stretches of high similarity across his albums are rarer. Good as I Been to You and World Gone Wrong are the most similar albums, while Christmas in the Heart and Tempest rate more in line with his earlier work than his modern approach.

The Beatles: Across the Musical Universe

With only a decade of work to analyze, The Beatles’ chart gives us a less hectic picture. Their first few albums were quintessential — though revolutionary — pop music, as illustrated by their high valence, energy, and danceability scores. These values decreased steadily between 1964 and 1969 as their music became heavier, bolder, and more introspective.

Meanwhile, the similarity heatmap shows us the increasing experimentation apparent in The Beatles’ discography: a trend of divergence from their earliest work culminates in Yellow Submarine, clearly the black sheep of the catalog.

Nas: Long Live the Idols

Straight out the dungeons of rap, Nas has dazzled audiences with his flow and storytelling since Illmatic dropped in 1994. As a hip-hop artist, his swings in audio features are less intense, tending towards high energy and danceability scores and low instrumentalness and acousticness scores throughout his catalog.

Nas lacks the blocks of consecutive similar albums present in the previous artists’ heatmaps. Interestingly, The Lost Tapes scores as the most self-similar album, despite being a compilation of leaked and unused cuts from earlier records. Additionally, Kanye West’s executive production of 2018’s NASIR clearly sets the album apart as unique in Nas’ discography.

Using Musical Metadata to Generate Personal Song Recommendations

Sometimes, a song seems to perfectly match my natural groove, and I just can’t keep it out of my head. I might go two weeks, two months, or two years without listening to it, yet I’ll still feel that familiar melody sneaking into my brain while I’m chopping peppers or stepping into the shower. “Doe, Jane” by Shakey Graves has been one of those songs for me. Ordinarily, I would try to scratch the itch by listening to the rest of the album in search of other songs that hit me the same way. Two years before, however, I had fallen in love with another Shakey Graves song, “Family and Genus,” but felt let down by the rest of the album. Despite loving these two songs, I found cuts from his newest project to be similarly mediocre. If I could identify phases within artists’ discographies, could I find more songs I liked by Shakey Graves without having to slog through all 154 minutes of his music on Spotify?

My playlists tend to be eclectic mixes of artists; I’m more attracted to the styles of certain musicians than to uniform genres. As a result, auto-generated recommendation playlists tend to fall flat for me. Spotify and Pandora will recommend artists who are similar to the ones I listen to, but they almost always fail to capture the subtleties I actually enjoyed about the initial song. Despite all the data Spotify has about my listening habits, my friends have a much higher success rate on recommendations.

Rather than using the audio features to find similar folk artists, as Spotify’s Discover Weekly playlists and song radios do, I wanted to drill down within one artist’s catalog to find more songs I would enjoy. After inputting an artist (or set of their albums) and specifying which two of their songs I liked, I used K-means clustering, an iterative machine learning algorithm, to group the songs into a specified number of clusters. I ran the clustering algorithm on all the tracks by an artist, increased the number of clusters until the function would no longer put my two songs in the same cluster, and output the other songs that were grouped with them. I found four of the attributes — danceability, energy, instrumentalness, and speechiness — gave the most useful groupings for most artists (in some cases, switching instrumentalness out for acousticness was more effective). By reducing the number of attributes (from the seven used for similarity), K-means was able to focus on the more relevant aspects of the songs I liked.

Using Shakey Graves as my test case, the algorithm made it to six clusters before breaking up my two songs. There were four songs in my final output cluster: my two input songs, plus “Climb on the Cross” and “Alexander, City Born.” I eagerly gave them a listen and… really liked both! Neither was in Shakey Graves’ 10 most popular songs, according to Spotify, so I felt satisfied that the recommendations were unique to me in some way.

I then recruited some friends to see if the recommendations produced by K-means and Spotipy worked for others. I looked for artists that had at least one song the user liked, a catalog that the user hadn’t fully heard, and ideally some music that didn’t appeal to the user as much (so positive reactions would be noteworthy). The results from the experiment are below, along with the number of clusters it took to get them:

Overall, I was pleased with the results. My friend Anna asked for Gallant and Sinead Harnett, and described the results as “nail on the head” and “pretty good,” respectively. My dad requested The White Stripes and said he liked “almost all” of the recommendations, especially “Hello Operator” and “Rag and Bone.” Rock on, Dad.

Conclusion:

The musical attributes available on Spotify may not tell us everything about a song’s sound, but they can give us valuable insights into songs that share traits and style. The data backed up the popular stories of the artists’ career arcs and indicated connections that might not be apparent to a casual listener. I also successfully generated recommendations for myself that outperformed Spotify’s tailored playlists. Even if my calculated recommendations won’t overtake Spotify’s or Pandora’s on a large scale anytime soon, it is cool to know that I can use the data that Spotify offers. Going forward, I would like to use natural language processing to analyze trends in lyrical content of the songs. If you’d like to try to generate your own recommendations, the GitHub repository for my code can be found here. Happy listening!

1: Dylan responding with “I don’t believe you, you’re a liar,” before turning to his band and telling them to “Play it [expletive] loud!” as they started “Like a Rolling Stone” has to be one of the greatest Rock & Roll moments ever.