Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Instructor: Abby Noyce
Lecture Topics:
Music cognition, More video, Music and emotions
Lecture 20
» Download English-US transcript (PDF)
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
ABBY NOYCE: So if we're going to talk about music and cognition, about how we understand music, how we kind of make sense out of music, we have to back up a step and say, so what is this music thing? What is music? What defines music as opposed to other kinds of sounds?
AUDIENCE: It's catchy.
ABBY NOYCE: It's catchy. OK. Some music is catchy.
AUDIENCE: It's a string of notes that actually sound good together.
ABBY NOYCE: It's a stream of notes that actually sound good together.
AUDIENCE: Generally, if you're [? in P ?] all of the notes have a common frequency involved.
ABBY NOYCE: OK. They're all like multiples of some base harmonic.
AUDIENCE: There's a pattern to the tones.
ABBY NOYCE: There's a pattern to the tones.
Good. OK. And if you listened to all of the things that we have just given as definitions, you can probably think of some example or another of something that claims to be music that is not true for these things. There's certainly music that is not catchy. There's certainly music that is not based on a harmonic scale. There's certainly all of these things.
All right. So we all know that Wikipedia is not an academic reference, but nonetheless, Wikipedia defines music as an art form. The medium of which is sound arranged over time. This is a very broad definition.
You'll notice that none of the things that we discussed are actually in there. It's merely this organization of sound over time.
AUDIENCE: So any sound basically would be music?
ABBY NOYCE: That is organized over time by somebody.
AUDIENCE: Oh, [INAUDIBLE] OK.
ABBY NOYCE: Is that a distinction that makes sense?
AUDIENCE: [INAUDIBLE]
ABBY NOYCE: So for example, painting, like the medium is the surface on which you're painting, and the kinds of pigment and colors, things that you're using, right? So Wikipedia says that-- and I suspect, because this sounds like a very Wikipedia thing that was argued over in the back channel extensively, that there was some discussion over how can we make this definition as inclusive as possible. So they're saying that the organization of sound over time. So somebody has got to arrange the sounds in a certain way. Thus it becomes music.
AUDIENCE: So like [INAUDIBLE]. Would that be called music?
ABBY NOYCE: So yeah. So one of the things that I have a problem with with this definition is it doesn't make a distinction between music and, for example, speaking. Right. I'm clearly organizing sounds over a period of time when I'm sitting up here talking. And I think the general consensus would be that this is not music, right?
So this is not a perfect definition in any way, shape, or form. But it gives us something to start with.
And Wikipedia tells me that the sorts of aspects of this arrangement of sound have four key qualities. So pitch of the sounds as they are arranged. Rhythm of the sounds. So things like tempo and meter, all of this.
So we've got pitch, which is like melody, and harmonies, and things like that. We've got rhythm and tempo.
The dynamics, the loudness and softness of different pieces of it.
Who here plays a musical instrument of one sort or another?
OK. Everyone except for me. Woo. So you guys know all of this stuff.
So changing the dynamics is one of the things that defines what makes a particular piece of music sound in its distinctive way.
And finally, and almost kind of at the bottom compared to all of these others, is the kind of timbre of the sounds that you can control. So the things that make a violin playing an A sound different from a flute playing an A, sound different from a saxophone playing an A. So it's all the same pitch, but the timbre, the sounds of each particular instrument are very different.
So the other thing I wanted to talk about really quickly is this idea. So we all have this kind of sense that music can convey emotional content. And we're pretty good with this. We know what a sad sound sounds like. We know what a happy song sounds like. We know what an angry song might sound like.
AUDIENCE: [INAUDIBLE]
ABBY NOYCE: Yes. Thank you. No wonder we're having difficulties.
[LAUGHTER]
AUDIENCE: Great timing.
ABBY NOYCE: And so there's been a lot of studies showing that with various kinds of melody instruments that people are really good. If you tell a musician to play this and play it so it's sad, versus play it so it's happy, people are really good at identifying what emotion the musician was trying to get across.
And in my hunt for random and interesting musical stuff today, I found two studies about just how much subtlety you can get into this idea of conveying information with music.
And the first one is these guys went down to just a straight up drum beat rhythm. This is a study by Laukka and Gabrielsson a few years ago. And they had professional drummers. And they had them play a particular rhythm. And they said, play it so you're happy. Play it so it sounds sad. Play it so it sounds angry.
And then they played these same rhythms for a group of subjects, probably undergrads. They're at a university. And they said, rate this. Rate on a scale of 1 to 10 how happy, how angry, how fearful, or how sad it is.
So here are some drumbeat clips for you guys to listen to. Tell me what you think this one is.
AUDIENCE: [INAUDIBLE] that'll screw up the feed.
ABBY NOYCE: It shouldn't, because this isn't streaming. I think it's just an MP3.
AUDIENCE: Are you sure?
ABBY NOYCE: Load.
[DRUMS]
Happy, sad, angry, fearful?
AUDIENCE: Happy.
ABBY NOYCE: Anyone feel otherwise?
AUDIENCE: Happy.
ABBY NOYCE: Happy? Bum-ba-da-dum.
OK. So there's one.
[CYMBALS]
ABBY NOYCE: So it's the same base rhythm being played differently.
All right. So which emotion is this?
AUDIENCE: That sounds sad or solemn.
AUDIENCE: I'd say solemn.
AUDIENCE: I'm going with sad.
ABBY NOYCE: Solemn or sad.
AUDIENCE: Sad, sad, sad.
ABBY NOYCE: How about this one?
[DRUM BEAT]
AUDIENCE: [INAUDIBLE]
ABBY NOYCE: Angry?
AUDIENCE: It sounds happy to me.
ABBY NOYCE: Happy?
AUDIENCE: [INAUDIBLE] to me.
ABBY NOYCE: It sounds like what?
AUDIENCE: Like expressive.
AUDIENCE: It sounds [INAUDIBLE] neutral.
ABBY NOYCE: It sounds neutral?
AUDIENCE: Like something to look forward to.
ABBY NOYCE: Like anticipatory sort of? OK.
AUDIENCE: So it could be fearful.
AUDIENCE: That could be fearful.
ABBY NOYCE: What's this one?
[DRUM BEAT]
[LAUGHTER]
AUDIENCE: I don't know.
AUDIENCE: I think extremely happy or angry.
ABBY NOYCE: Happy or angry?
AUDIENCE: I thought [INAUDIBLE].
AUDIENCE: Yeah.
ABBY NOYCE: All right. So let's see what these actually were.
So those were happy, sad, angry, and fearful.
AUDIENCE: That was fearful?
AUDIENCE: That was fearful?
ABBY NOYCE: Interestingly enough, we got the first two. Like everybody pretty much agreed on the first two. And then the last two we were a little bit more varied on.
And these guys actually-- yes. They threw in here a graph of the data these guys got.
So, again, they had people rate each sample based on how well they thought it fit a particular emotion.
So when the musician was trying to play happiness, people rated happy the highest emotion, and others lower. Likewise for sadness. Likewise for anger. Likewise for fear.
So the thing that's interesting here is that those dark blue bars are the places where one is actually significantly different from the others in the statistical sense, so that it's significantly different.
So fear was rated significantly higher than all of the other categories. The fear sample was rated as more fearful than all of the other categories. And also then the sample was rated for all of the other categories. It was also rated as more fearful than all of the other samples.
So this is a really solid result. And it's cool because it's this really minimalist input that they're using, these really basic samples that are just basically only varying by tempo and by dynamic versus like a melodic instrument, which has a lot more ways of variation.
AUDIENCE: Do you think it could have something to do with the culture?
ABBY NOYCE: It might.
AUDIENCE: If they expanded the sample to people from different cultures--
ABBY NOYCE: Well, so we have a cross-culture here, because we just did it on a bunch of American kids. And they did it. Oopsala. Where's Oopsala. I'm guessing somewhere in like Scandinavia, right?
AUDIENCE: Oopsala.
ABBY NOYCE: Anyone know? I don't know. My guess is it's like Sweden or somewhere.
AUDIENCE: Google it.
ABBY NOYCE: We can Google it.
AUDIENCE: Google. Google.
ABBY NOYCE: Where is Oopsala? Sweden.
AUDIENCE: Sweden!
ABBY NOYCE: All right. So there's two samples right there. And you can argue that Northeastern America and Northwestern Europe are pretty closely related cultures. It would be interesting to run it somewhere very different.
OK. And the other one I wanted to show you guys is this other really cool study. And this is a German group that were doing an ERP study. So they're using an EEG cap with the electrodes to measure the electrical activity of the brain.
And they're not just going for emotions. They're trying to see if music can convey some really kind of sophisticated concepts. The example they use in their paper is wideness. Can music convey this idea of wideness.
So what they did is they had people listen to a sentence, which was either related or unrelated. And these are translated, since the originals are in German.
So the related sentence would be "The gaze wandered into the distance." And the unrelated sentence would be, "The manacles allow only little movement."
And you can see those are just the sonograms of that, numbered. So showing how the sound-- I think those are just dynamic graphs actually.
Anyway, if you look at the graph on the upper right there, then you can see the purple line and the blue line are the EEG measurements of the unrelated prime and the related prime. And in particular, you can see that the blue line and the purple line follow each other exactly for the first like 300 milliseconds.
And then that N400. Remember, because negative is up on this graph. So that spike where it goes back towards the negative at about 400 milliseconds after the stimulus is different. It's bigger for the unrelated prime than for the related prime. The related prime has a smaller spike.
And so they looked at that, and they said, OK, what about for music? And they played two musical samples. One of which they claim conveys the concept of wildness.
[MUSIC PLAYING]
AUDIENCE: Very Disney.
ABBY NOYCE: And one of which they claim does not. And you can argue all you like about whether or not these are good primes.
[MUSIC PLAYING]
[GIGGLING]
AUDIENCE: Accordions.
AUDIENCE: Wait. People still actually play accordions.
AUDIENCE: Yeah.
AUDIENCE: Yeah. Obviously.
ABBY NOYCE: And so they found two music samples that were the same length that didn't show dramatic dynamic differences. Although there's definitely patterns in them that you can see. This one gets much louder at the end in a way that the unrelated prime--
AUDIENCE: She's still talking.
ABBY NOYCE: --doesn't.
And, again, the same thing. They measured the event revoked event response potential in response to these two primes.
So if they had people listen to the Strauss piece and then asked them whether it was related or unrelated to wideness, then they found, again, that this N400 is smaller in the related case than in the unrelated case.
And what's interesting is that that's the exact same pattern they found for the linguistic prime. And one of the big confounds in this study is that they are asking people to consider whether the prime, whether the sentence or the musical piece, is or is not related to the concept.
So the difference here could just be falling out to people's difference in response, and people's difference in perception as they judge it after the fact, and not to an actual difference in seeing this as more wide or less wide, but having decided that it was more wide or less wide.
So it's not the world's best control study. But it's nonetheless a really interesting result. And I'm moderately impressed. I wouldn't have thought you could pull out something with that level of semantic content from a musical sample. So I thought that was really cool.
Yes.
AUDIENCE: So they asked them to consider wideness before they listened?
ABBY NOYCE: They played them the sentence, or the sample. And then they put the word on the screen in front of them and said, was the thing you just heard related to this word? Yes or no?
So people didn't have wideness in mind when they were listening to these things.
I think the timescale on the sample is-- yeah. This is measured from when they see the word that they're asked to judge. So it's when they're making that judgment decision about related or unrelated.
And you see this difference whether the thing that they're considering is a musical prime or a linguistic prime. But the difference could be an artifact of the decision making process, rather than an artifact of the original perception, which they're kind of fuzzy around.
What do you guys think? Was the Strauss sample wide? Do you think it sounded wide?
AUDIENCE: I think if you hadn't mentioned the wideness before, [INAUDIBLE]
ABBY NOYCE: It would be a better sample? Yes. This is probably true.
AUDIENCE: I was practically listening for the wideness.
ABBY NOYCE: Do you think the Strauss sample sounded wider than the-- who else was it-- [INAUDIBLE] sample? Did it sound wider?
AUDIENCE: Yes.
ABBY NOYCE: Is that because you were expecting it to be wide?
AUDIENCE: It kind of is hard to define wideness.
ABBY NOYCE: Yeah. Definitely.
How would you define wideness in music, [INAUDIBLE]? How would you express wideness in music? So Strauss does it with this big crescendoing thing.
AUDIENCE: Yeah. So more dramatic I guess.
ABBY NOYCE: Yeah.
Cool.
AUDIENCE: I think they expressed it by starting with something that wasn't wide. So the less voluminous, the less dynamics, and the less instruments were participating in it. It's sort of the contrast so that when you reach the peak of the crescendo you say, wow.
ABBY NOYCE: OK.
Yeah. So it definitely like expands, like the dynamic range goes up.
I don't know. Does it make you feel like you're looking out on an open vista? If you had to animate, if you were like doing Fantasia 3 or something and trying to animate a scene to go with that particular piece of music, what would you do?
AUDIENCE: [INAUDIBLE]
ABBY NOYCE: No? Yes?
AUDIENCE: Walking on a dark street with a random guy playing the accordion.
[LAUGHTER]
ABBY NOYCE: For the second one? Or for the Strauss.
AUDIENCE: For the second one.
ABBY NOYCE: OK.
AUDIENCE: I was thinking of walking [INAUDIBLE]
[INTERPOSING VOICES]
AUDIENCE: Yeah.
AUDIENCE: Yeah. I mean, like, if no one else was there except him and you--
AUDIENCE: Yeah. And just [INAUDIBLE].
AUDIENCE: Just walking [INAUDIBLE].
AUDIENCE: There you go.
AUDIENCE: The first one to be walking [INAUDIBLE].
ABBY NOYCE: All right.