Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Instructor: Abby Noyce
Lecture Topics:
Introduction to Production of Language, Brain and Language, Speech Errors, Grammatical Encoding, Phonological encoding and articulation
Lecture 19
» Download English-US transcript (PDF)
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
ABBY NOYCE: So we've been talking this week about perception of language, how you take all of this information that's coming in, sort it out, figure out the structure to it. We're going to talk today about a slightly smaller subject, or something that has been studied in less depth than comprehension has, which is how you produce language.
You can do the same thing in reverse. You can go from this kind of non-semantic idea of something that you want to convey to your listeners. So that's this kind of message level, where you've got an idea and from there, you select the words you're going to use to convey it and you put together a sentence structure for it. And from there, once you have words in a sentence structure, then you need to pull up the phonological information about the sentence, about all of the words that are going into it.
And finally, you pass all of this information onto the motor control centers so that you can actually articulate your sentence. So this is a fairly standard--
This model shows that there are four different levels of processing, and these don't necessarily happen sequentially, like you do all of your grammatical encoding before any of your phonological encoding. There's some reasonably good evidence that they interact a bit. And likewise, with phonological encoding and articulation.
But first, just to get our brains working, we're going to have a moment of design an experiment. So you're a cognitive neuroscientist, or psycholinguist, perhaps, and you believe that there are distinct levels involved in the production of language. So when somebody produces a sentence, they have to go through different kinds of processing to do that.
How might you prove or disprove-- how might you test this hypothesis that there are distinct phases in language production? Think about it. You can talk to your classmates.
Things in your brain that are involved in language. So we talked earlier this week about, remember Wernicke had had this idea that there were auditory images in Wernicke's area and motor images and Broca's area. And so Wernicke's aphasia versus Broca's aphasia, the difference was which of these centers was harmed. And there's a lot of flaws in that model.
And this guy named Marsel Mesulam modified it a little bit and said that all language, again, depends on circuits, on activities between different language areas. And Mesulam was one of the first ideas to point out that different kinds of language production require different kinds of processing. And then the example he used is that if I ask you to do something like name all the months in the year. Ready? Go. Come on.
January, February, March, April, May, June, July, August, September, October, November, December. So this is something that requires a lot less planning and a lot less structuring than actually producing a well-formed sentence does.
And Mesulam hypothesized that just rote motor connection like this, or recite the alphabet, or recite your times tables, or any of these things that you learned by heart as a little kid, really only requires premotor and motor areas up here in the frontal lobe. So we have our primary motor cortex is this strip right along here, and premotor is the blue stuff next to it.
He pointed out, though, when you're hearing words, this activates primarily auditory cortex down here in the frontal lobe in auditory association areas. Unimodal, so auditory alone versus these kind of crossmodal representations, when you're actually thinking about what you're hearing. If I say the word, I don't know, baseball, you probably come up with not just an auditory response to that, but all of these other associations. You might have visual memories that you associate with that or auditory memories that aren't just the sound of the word that you associate with that. So that would be like a crossmodal association.
But Mesulam says that when you're just listening to words but not acting on them, then most of what's activated is just auditory cortex. And Mesulam says when you're producing words, then you end up seeing activity all over the brain. You get motor stuff. You get word knowledge stuff. All of these different pieces are involved.
So here's an example of-- this is some old-ish, at this point, data-- a PET scan. So these are participants doing four different tasks. So they're looking at a computer screen. And in the first case, they would just show a word on the screen, and the participants didn't have to do anything but look at the word.
And as you can see, most of what's most active in that condition is occipital. It's all that visual cortex. You can see there's a little bit in the frontal lobes, a little bit in the temporal lobes, but the areas that are most active are the visual cortex in the back of the brain there.
The second case was pretty similar, except instead of looking at words, this was auditory. So you'd have headphones on and look at the screen and you would hear words being played into the headphones. And in this case, it's mostly, as you might expect, temporal lobe stuff that's most active. It's auditory cortex on the sides of your head there in the temporal lobe.
And one of the things that's moderately interesting about that-- and I don't have a good story for why this is-- is that a lot of the frontal lobe activation you see in the looking-at-words case goes away in the auditory case.
AUDIENCE: [INAUDIBLE]?
ABBY NOYCE: They were just looking at a blank screen. I think it had like a fixation point on it. "Keep looking at the little dot on the screen."
In the third case, they showed them a word on the screen and participants had to read the word aloud. They had to pronounce it. And as you can see, at this point, a lot of the activity is suddenly in the motor cortex. I'm sure you all are shocked. So this is one piece of evidence that articulation requires a different kind of activity than simply understanding or perceiving words, that you do, in fact, need motor activation. So this is that strip between the frontal lobe and the parietal lobe, where motor cortex is.
And finally, they asked them to do a word generation task. So they would show them one word on a screen and ask the participant to say a word that was related. So if the word that was shown to them was "bike," they might say "ride" or they might say "helmet" or they might say "Cambridge." If the word on the screen was "boat," they might say "sail" and so on. So trying to find a word that's associated with the word on the screen.
So this suddenly requires participants to actually use the semantics of the word in a way that they didn't have to on any of the previous levels. If you're just reading it or just hearing it or just reading it aloud, you don't really have to think about what it means. Whereas if you're trying to come up with a related word, you've got to tap into all of your semantic knowledge about what that word represents.
And what's interesting about this is that you get, again, some motor activation, and you get some auditory activation, but you get a whole bunch of stuff going on kind of in that premotor area, that area that kind of coordinates and plans-- utterances and other motor things, but in this case, utterances. So this kind of imaging scan of participants on tasks that require them to think about language on different levels is one piece of evidence showing that there probably are different phases of processing.
Another piece of evidence that shows that there are different stages in processing are the kinds of mistakes that people make when they talk. We all got used to making fun of President Bush because his command of English language can be a little weak at times, but the fact of the matter is that, if you followed anybody, no matter how articulate they are, around with microphones and wrote down everything they said, you'd find tons of errors. We all make speech errors anytime you're talking and not just reading aloud, and sometimes even then.
So one of the most common types of speech errors are what are called exchange errors, and you'll see these when people are talking fast all the time. And you'll get both word exchange errors like, "I wrote a mother to my letter," and you'll also see sound exchange errors, phoneme exchange errors, and these are called spoonerisms in honor of Archibald Spooner, who was a professor at one of the colleges in New York in like the early 20th century and was notorious for doing this.
And there are many statements, such as the one there, that have been attributed to Monsieur Spooner, and he has given his name to this kind of sound exchange error. "You've hissed all my mystery lectures." "You have tasted the whole worm."
AUDIENCE: What's it supposed to really be?
AUDIENCE: Yeah. "Wasted the whole term."
AUDIENCE: Oh.
ABBY NOYCE: There you go. So sound exchange errors are pretty common. I mean, I remember my eighth grade earth science teacher sent the entire class of us into about five minutes of uncontrollable giggling. We were talking about volcanoes, and he talked about how volcanoes would release clouds of gash and ass. And of course, this is eighth grade, and [GASPS] he said "ass." And there was much giggling. And it's just a straight up sound exchange error like people make all the time.
So let's look at some of these different levels of processing that have to happen in order to produce an utterance of some sort. So the message level is probably pretty straightforward. You have some piece of information, some idea, that you want to convey to the people around you. How you move on from there is a little bit harder.
So one of the things that happens is that there's this word selection stage. So you're going from this raw meaning idea and trying to find the words that are necessary to represent it. Most likely, if we look at the sort of models that we've been looking at throughout this course, where you've got competing activations, you can think of this as like the meaning that you want to get across then is going to activate different word representations depending on how strongly related to the meaning they are. So if we think of the cortical interconnections that we've considered all along.
And sometimes, every once in a while, you'll get errors that are a blend of two different words. So sometimes when you have two words that are both strongly activated for a particular position in your sentence, you'll try to say both of them at once and get all tangled up.
The example in the reading is, if you dropped a pen under somebody in front of you's chair and you're trying to ask them to pick it up for you and you've got both the word "chair" and the word "seat"-- you know, "My pen is under your chair. My pen is under your seat. Can you get it for me?" But the "chair" and the "seat" are both competing for that noun slot in the middle there.
And if they're both equally strongly activated, they can both get passed down to the phonological stage and you can end up trying to pronounce both. And you'll say things like "My pen is under your cheat-- under your sair-- under-- under your chair. Can I have it, please?" And I don't know if anyone here has ever experienced that kind of tangled up language production, but it's fairly common if you start listening for this stuff.
The other step in grammatical encoding that's happening kind of at the same time as you're trying to find the words you need is you're trying to understand how the syntax is put together-- what guides sentence syntax. And this is, at least immediately, hard to think about because all that you get to see of it is the output. It's what sentences finally come up. You don't get to see any of the intermediate processes.
And so somebody did a kind of clever study. They told participants they were doing a memory experiment. And so they showed them a string of simple images and said, hey, in order for you to remember these better, we want you to say one sentence about each image as it's presented to you. These were simple scenes.
So you might have-- I don't know if I showed you. Here's a house and a tree. And so for this sentence, you could say either, the house is next to the tree, or you could say, the tree is next to the house. And what these guys did is, before each picture was presented, they just put up a word and told subjects just to read it out loud.
And so for a scene like this, they might put up a word like, oh, I don't know, "building" versus a word like "pine." And what they were hypothesizing was that, if your priming word was a word that was related to the house part, then when you tried to develop a sentence about this picture, your ideas about houses are going to be faster to activate because they've been primed by the word than your ideas about trees.
And so you would expect to see people say sentences that were like, the house is next to the tree. They'd pull up the house part first and then fill the rest of it in later. Whereas if the priming sentence was something like pine or maple, then your tree representations would be more active and easier and faster to be pulled up. And so you would start developing a sentence structure that let the tree part go first and fill in later with the other half of the image with the house, so that you would see people say, the tree is next to the house, if they were primed with one of those tree-related words.
And pretty much what they found is that the part of the scene that was primed generally came first in the sentences people spoke. And the part of the scene that was not related to the priming word came second. So you'd see people putting things into active versus passive voice depending on which part of the scene was first-- not so much for this image, but an image that shows one object acting on another object.
So this idea that word selection and sentence structure happen kind of at the same time and how fast you can pull up the words you want is going to affect how you structure your sentences. If you can get one word right away, then you're going to build the rest of the sentence to accommodate putting that word at the beginning and filling in later with the other words that are a little bit slower to come up.
So back to these ideas about stages that are involved in language production. We talked about grammatical encoding earlier. Let's talk about this next stage down, this phonological encoding.
When people are speaking fluently, they usually produce about three words every second. And so for each of these words, the speaker's got to be able to pull up the phonological information about it-- the information about what pattern of phonemes, what pattern of sounds, makes up that word, how you stress them, how it maybe changes when it's next to other words in a particular order, all of that information.
And one thing that you tend to see happening when people fail at pulling up this phonological information is what's call a "tip of the tongue" state. When somebody maybe reads you a definition and asks you, what's that word? And you know it. You know it's a word you know, and the word just won't come to you, or the name. Who's that person? Oh, I know them. They are-- and you just can't get it. Anyone ever have this happen to them?
AUDIENCE: All the time.
ABBY NOYCE: Right. So it's a "tip of the tongue" state. And in this model, this "tip of the tongue" scenario is explained by when you know all of the semantic stuff about the word. If it's a name that you can't get, you know which person you're talking about. You can tell somebody all sorts of things about them, like if they were in your history class last semester or whatever, but you can't come up with the name.
And this is generally believed that when the connection between the semantic representation and the phonological information is blocked for whatever reason, that you can't go from the semantic step to the phonological step. "Tip of the tongue" states are more common for moderately uncommon words. So for really common, everyday, short words, "tip of the tongue" states are pretty rare. For longer, more precise words, words that you don't see so often, they're more common.
Usually when you have a "what's that word?" it's for a word that's not one of the most common in the language. And there have been a few cases of patients, usually following some kind of brain injury, brain trauma, where they have this "tip of the tongue" phenomenon all the time. They have a great deal of difficulty. For every word they try and come up with, they just can't-- they know what they want to say. They've got the semantic representation right there, and they just can't find the phonological representation for it. Yeah, tip of the tongue.
So this is another kind of example of a stage where the sentence construction stage, the grammatical encoding stage, of producing language is working OK, and then this later phonological encoding is not working. So another piece of evidence that these might, in fact, be separate phases in language processing, language creation, language production, something.
So do these things ever interact with each other? The easiest way of looking at this model is saying that sentences start out as abstract meanings, which then are passed to this grammatical encoding level, which finds words and builds a structure, which then passes them to the phonological level, which finds sounds, which then passes them to this articulation machine. But it turns out there's actually at least some feedback going on between the levels, or it seems to be.
And the best evidence for this at the moment is that we talked about word exchange errors. "I wrote a mother to my letter." And these occur-- if there was no interaction between the grammatical levels and the phonological levels, then you'd expect to see that word exchange errors happen equally often between pairs of words, disregarding whether they have any phonological similarity.
If you actually document what kinds of word exchange errors people tend to make, this isn't true. The word exchange errors people make occur more often in words that have some kind of similar phoneme pattern. So "mother" and "letter" both have a stressed first syllable, and then that last syllable with just the E-R on it that's unstressed. And patterns like that are what you tend to see. So there's some amount of feedback going on between these levels.
Articulation, of course, this last stage. Articulation is mostly just a matter of motor control. It's saying, OK, I've got a phonological representation. How do I turn that into actual, specific movements of my mouth, my tongue, my lips, all of this stuff that's actually articulating it?
So here's our good old brain diagram. Again, primary motor cortex here. And it's been really well documented at this point that is kind of laid out over a strip along motor cortex are regions that respond to different things. So toes, ankles, hip, knee, trunk, arm. Notice there's a lot of cortex devoted to your hands. That's probably not too shocking. Fine control over hands is really important for humans. It's important for a lot of things that we do.
There's also a lot of it connected to, like, lips and jaw and tongue. Again, this ability to have really fine and precise control over these things is important. I'm always kind of startled at just how much motor cortex goes to the swallowing and throat control, which makes sense, but it's not something that we think of as requiring a lot of control.
So motor cortex happening right up there. Remember, if we go back to-- what was his name? The guy we were talking about at the beginning of class who had a model of how language depends on different parts of the motor cortex. And I'm totally blanking on his name. "Tip of the tongue" moment. Starts with an M. What was it?
AUDIENCE: Mesulam.
ABBY NOYCE: Mesulam, OK. Yeah, him. And he was saying that, depending on what you're doing, some kinds of articulation basically just require motor control. Others have to be kind of coordinated from further up. So we've got our motor cortex. So the primary motor cortex, the strip that's right along the sulcus there, right up against the parietal lobe, controls mostly fine motor coordination, very fine control of movement. There's actually nerve fibers that go from primary cortex all the way out to the muscles. This is like direct muscular control for fine motion.
And then next to it is this kind of premotor area. And the premotor area seems to be most involved in setting up sequences of actions, especially in response to perceptual information. So for example, if I was to toss something across the room to a student. Sarah, catch. And Sarah is-- very nice. So Sarah is taking in all of this perceptual information, where she sees the pen coming towards her, and coordinating this plan that involves moving her hands up at the right time and in the right place in order to be able to catch the pen when it gets to her.
And that's coordinated in part-- setting up that sequence is coordinated through the premotor area and the cerebellum. The cerebellum has a lot to do with getting this kind of smooth coordination so that our muscles are all working at the right time relative to one another so that we aren't moving all jerkily.
And then there's this kind of supplementary motor area that is involved in more self-directed action planning. So premotor is really working with responding to your immediate environment, taking that perception and building a sequence from there. Supplementary motor area is involved in longer-term planning. If I was like, I have a motor goal of getting to that window, that would be coordinated by the supplementary motor area. And then that premotor area would be more involved in helping me walk around obstacles and make sure I don't walk into anything on my way of getting there. So all of these are going to be involved in articulation.
Someone did a study in monkeys, training them to-- no, what am I thinking of? Yeah, this was a study in monkeys, but it wasn't a directly articulation-related one. Someone did a study in monkeys-- because monkeys don't have language-- where they had either trained them to push a set of three buttons in a certain pattern or train them to look at how the buttons lit up and then repeat the pattern by pushing the buttons.
So one of these was a task where the monkey had to know the plan in advance. One of these was a task where the monkey had to respond to environmental input to perceptual information. And this is one of the really solid pieces of evidence for the difference between supplementary motor and premotor in terms of what they do.
When the monkeys were working entirely from memory, then it was mostly their supplementary motor area that was active. When they were working in response to what was shown to them right then, it was mostly premotor.
One other thing about articulation before we move on is just that, because our face is right up here close to the brain, a lot of the nerves that control the muscles that are involved in articulation don't run through the spinal cord the way a lot of them do for, say, controlling your hands. A lot of it is these direct cranial nerves that come right from the brain. They don't go out and through the spinal cord and back. So the way that the circuit works for controlling facial stuff is different than for controlling a lot of the rest of your body.
Moving on very quickly. The other context in which we tend to produce language is writing things. How we produce spoken language is not well understood. How we produce written language is even less well understood. But there are some key differences between producing written language and between speaking language that I just wanted to highlight for you guys.
Unlike when you talk-- you're usually talking to people, with people, with people immediately around you-- when you're writing, it's usually just you, usually just your stream of language that's being put down. You aren't building it into like a conversation or anything like that. If you analyze the syntax that's used in people's writings versus a syntax that's used when they are speaking, it's more complex. You'll see more recursion, more subordinate clauses, more nested phrases, all of that.
And this seems to be easier to follow when reading, too. So this make sense on both sides. And when you're writing, you get to futz with it and change it after you can write it down and be like, no, that doesn't make sense. Let me go back. You don't get to do that when you're talking. If you say something that doesn't make sense, you're kind of stuck with it being out there.