1 00:00:01,040 --> 00:00:03,460 The following content is provided under a Creative 2 00:00:03,460 --> 00:00:04,870 Commons license. 3 00:00:04,870 --> 00:00:07,910 Your support will help MIT OpenCourseWare continue to 4 00:00:07,910 --> 00:00:11,560 offer high quality educational resources for free. 5 00:00:11,560 --> 00:00:14,460 To make a donation or view additional materials from 6 00:00:14,460 --> 00:00:18,390 hundreds of MIT courses, visit MIT OpenCourseWare at 7 00:00:18,390 --> 00:00:19,640 ocw.mit.edu. 8 00:00:31,970 --> 00:00:33,532 PROFESSOR: OK, good afternoon. 9 00:00:36,970 --> 00:00:39,540 So today we're going to talk about learning. 10 00:00:39,540 --> 00:00:50,590 So what can you do now that you couldn't do as an infant, 11 00:00:50,590 --> 00:00:52,490 very simply? 12 00:00:52,490 --> 00:00:53,300 AUDIENCE: [INAUDIBLE] 13 00:00:53,300 --> 00:00:54,530 PROFESSOR: Talk, that's a good one. 14 00:00:54,530 --> 00:00:56,860 Language, that's pretty good, right? 15 00:00:56,860 --> 00:00:59,050 Anything else that you couldn't do, yeah? 16 00:00:59,050 --> 00:01:01,070 AUDIENCE: [INAUDIBLE] 17 00:01:01,070 --> 00:01:02,700 PROFESSOR: Yeah, we don't actually know what the 18 00:01:02,700 --> 00:01:04,330 internal mental life of an infant's like. 19 00:01:04,330 --> 00:01:06,830 But it doesn't seem like they remember many very specific 20 00:01:06,830 --> 00:01:07,990 things about their lives. 21 00:01:07,990 --> 00:01:09,450 You do. 22 00:01:09,450 --> 00:01:12,160 But you know it's a lot more than, you know a ton of stuff, 23 00:01:12,160 --> 00:01:15,860 a lot of facts, right, things about the world, the values 24 00:01:15,860 --> 00:01:18,820 you hold, about your family, your culture, the country or 25 00:01:18,820 --> 00:01:20,650 countries you've grown up in. 26 00:01:20,650 --> 00:01:24,650 You just know a ton compared to an infant. 27 00:01:24,650 --> 00:01:27,010 And there's only two ways that you know this. 28 00:01:27,010 --> 00:01:30,470 It's either in your genes when you're born or you learned it 29 00:01:30,470 --> 00:01:30,960 through life. 30 00:01:30,960 --> 00:01:34,420 And so we're going to talk today about how scientific 31 00:01:34,420 --> 00:01:36,720 psychology tries to approach learning. 32 00:01:36,720 --> 00:01:38,820 And in a remarkable thing about learning is we often 33 00:01:38,820 --> 00:01:40,980 think, well, we're learning lessons or whatever. 34 00:01:40,980 --> 00:01:42,150 We're learning material. 35 00:01:42,150 --> 00:01:43,790 But the important part of learning the way we think 36 00:01:43,790 --> 00:01:46,350 about it is that it allows us to predict the future on the 37 00:01:46,350 --> 00:01:47,700 basis of the past, right? 38 00:01:47,700 --> 00:01:50,750 We want to learn lessons in life to know what's desirable, 39 00:01:50,750 --> 00:01:53,790 what's dangerous, what's a smart way to do things so 40 00:01:53,790 --> 00:01:54,880 we're more effective. 41 00:01:54,880 --> 00:01:57,080 We choose what we want to do better in the future to learn 42 00:01:57,080 --> 00:01:59,390 from the past and predict the future. 43 00:01:59,390 --> 00:02:01,630 It imbues our word with meaning. 44 00:02:01,630 --> 00:02:04,830 For an infant, not much means much, right? 45 00:02:04,830 --> 00:02:05,930 They just look around. 46 00:02:05,930 --> 00:02:07,160 Who knows what anything means. 47 00:02:07,160 --> 00:02:11,700 Words, gestures, facial expressions, there's not much 48 00:02:11,700 --> 00:02:12,720 meaning of the world out there. 49 00:02:12,720 --> 00:02:15,860 For you, the world is full of meaning, history, social 50 00:02:15,860 --> 00:02:18,610 interactions, everything that you're thinking about. 51 00:02:18,610 --> 00:02:21,950 So how do we acquire the meanings of things in the 52 00:02:21,950 --> 00:02:23,590 world that matter to us? 53 00:02:23,590 --> 00:02:25,930 And it's also had an interesting kind of a give and 54 00:02:25,930 --> 00:02:28,790 take with how we learn about learning in scientific 55 00:02:28,790 --> 00:02:29,930 psychology and sort of historically. 56 00:02:29,930 --> 00:02:32,290 I'll just touch a tiny bit on that. 57 00:02:32,290 --> 00:02:35,200 And we'll talk about three aspects of learning today: 58 00:02:35,200 --> 00:02:38,150 classical conditioning, operant conditioning, and then 59 00:02:38,150 --> 00:02:41,000 various ways in which there are limits to conditioning. 60 00:02:41,000 --> 00:02:43,420 Thank you, Todd. 61 00:02:43,420 --> 00:02:46,800 So let's start with sort of one of the most famous figures 62 00:02:46,800 --> 00:02:49,780 in all of conditioning or learning, Ivan Pavlov, who 63 00:02:49,780 --> 00:02:52,160 actually won a Nobel Prize, not for his studies of 64 00:02:52,160 --> 00:02:56,470 learning, but for his sort of groundbreaking studies for the 65 00:02:56,470 --> 00:02:58,000 reflexes of digestion. 66 00:02:58,000 --> 00:03:00,260 He worked out a lot of the fundamentals of how, in 67 00:03:00,260 --> 00:03:04,000 mammals, your stomach breaks down food and that how food in 68 00:03:04,000 --> 00:03:06,500 the mouth provokes specific salivation that begins the 69 00:03:06,500 --> 00:03:10,640 process by which food is decomposed and then digested 70 00:03:10,640 --> 00:03:12,870 in your stomach, the fundamentals of really how we 71 00:03:12,870 --> 00:03:15,310 live, right, eat to live. 72 00:03:15,310 --> 00:03:17,985 And he was very interested in salivation reflexes, what is 73 00:03:17,985 --> 00:03:21,090 it that drives that first bit of breaking down food in your 74 00:03:21,090 --> 00:03:22,590 mouth with saliva? 75 00:03:22,590 --> 00:03:27,690 And Pavlov was a pretty intense researcher, as you 76 00:03:27,690 --> 00:03:29,360 might be used to at MIT. 77 00:03:29,360 --> 00:03:31,930 There's a famous story where a graduate student came somewhat 78 00:03:31,930 --> 00:03:33,850 late to the lab and said, but Professor, there's a 79 00:03:33,850 --> 00:03:36,350 revolution going on with shooting in the streets. 80 00:03:36,350 --> 00:03:37,330 This is in Russia. 81 00:03:37,330 --> 00:03:39,080 And he said, what difference does it make when you've got 82 00:03:39,080 --> 00:03:40,160 work to do in the laboratory? 83 00:03:40,160 --> 00:03:44,560 Next time there's a revolution, get up earlier; so 84 00:03:44,560 --> 00:03:49,840 anyway, not an easygoing supervisor. 85 00:03:49,840 --> 00:03:52,480 He would do experiments a little rough in dogs, like cut 86 00:03:52,480 --> 00:03:55,790 the esophagus, the path by which food goes from the mouth 87 00:03:55,790 --> 00:03:59,300 to the stomach, to understand the role of the esophagus. 88 00:03:59,300 --> 00:04:01,570 And he found that, even in those animals, when he placed 89 00:04:01,570 --> 00:04:04,370 food in the dog's mouth, he expected, as a fundamental 90 00:04:04,370 --> 00:04:06,940 biologist studying how digestion works, that nothing 91 00:04:06,940 --> 00:04:07,670 would happen. 92 00:04:07,670 --> 00:04:10,580 But he noticed not only would the dog salivate, even though 93 00:04:10,580 --> 00:04:13,090 the food will never make it to the stomach, but furthermore 94 00:04:13,090 --> 00:04:15,680 the stomach juices, the gastric juices that break down 95 00:04:15,680 --> 00:04:18,810 food, also were released within the stomach even though 96 00:04:18,810 --> 00:04:20,610 the food would never get there. 97 00:04:20,610 --> 00:04:22,720 And then he got interested in how it is that these things 98 00:04:22,720 --> 00:04:25,340 are driven if they're not driven simply as a direct 99 00:04:25,340 --> 00:04:28,060 response to the arrival of the food. 100 00:04:28,060 --> 00:04:31,890 How is it that the mere sight of the food or the sight of 101 00:04:31,890 --> 00:04:35,950 the person bringing the food would drive the salivation and 102 00:04:35,950 --> 00:04:38,060 what he called psychic secretions? 103 00:04:38,060 --> 00:04:38,900 Because he was a biologist. 104 00:04:38,900 --> 00:04:41,560 So he expected, basically, you put the food in the mouth. 105 00:04:41,560 --> 00:04:44,390 There's some chemistry between the food and salivation. 106 00:04:44,390 --> 00:04:47,010 And here he's finding, just seeing food, just seeing the 107 00:04:47,010 --> 00:04:49,080 person who would bring the food, was driving these 108 00:04:49,080 --> 00:04:52,460 fundamental biological functions. 109 00:04:52,460 --> 00:04:54,300 So there's a kind of experiment that he would set 110 00:04:54,300 --> 00:04:55,360 up where the dog would be there. 111 00:04:55,360 --> 00:04:58,030 And he would be measuring the amount of salivation in some 112 00:04:58,030 --> 00:04:59,790 way or another. 113 00:04:59,790 --> 00:05:02,550 And in these kinds of experiments-- and I'll go 114 00:05:02,550 --> 00:05:05,480 through this language a couple times-- 115 00:05:05,480 --> 00:05:07,165 in conditioning they've come up with a sort of vocabulary 116 00:05:07,165 --> 00:05:09,350 to describe the pieces of things that go into this kind 117 00:05:09,350 --> 00:05:10,210 of learning. 118 00:05:10,210 --> 00:05:12,690 So there's an unconditioned stimulus, which is food. 119 00:05:12,690 --> 00:05:13,790 I mean you start with that. 120 00:05:13,790 --> 00:05:15,080 You want to eat food. 121 00:05:15,080 --> 00:05:16,820 And there's an unconditioned response, which is you 122 00:05:16,820 --> 00:05:19,510 salivate when you see the food or if it's in your mouth. 123 00:05:19,510 --> 00:05:21,330 That's the beginning. 124 00:05:21,330 --> 00:05:22,930 Then you're going to add what's initially 125 00:05:22,930 --> 00:05:24,270 a meaningless stimulus. 126 00:05:24,270 --> 00:05:25,830 In this case, it could be anything. 127 00:05:25,830 --> 00:05:27,100 But in this case, we'll say it's a bell. 128 00:05:27,100 --> 00:05:29,920 You ring a bell before the food comes. 129 00:05:29,920 --> 00:05:31,800 And you're going to get to a conditioned response. 130 00:05:31,800 --> 00:05:34,020 The animal will say, well, the bell predicts the food. 131 00:05:34,020 --> 00:05:35,410 And I'll salivate to the bell itself. 132 00:05:35,410 --> 00:05:37,460 Even though I'll never eat the bell, I learned that it 133 00:05:37,460 --> 00:05:38,930 predicts the food is coming. 134 00:05:38,930 --> 00:05:40,940 And so you've created a new association. 135 00:05:40,940 --> 00:05:43,980 You learned something new about the world, that a bell 136 00:05:43,980 --> 00:05:47,120 signals something that drives salivation. 137 00:05:47,120 --> 00:05:49,640 So the two things are sort of occurring next to each other 138 00:05:49,640 --> 00:05:51,630 in time or what Aristotle talked about as the law of 139 00:05:51,630 --> 00:05:52,340 contiguity. 140 00:05:52,340 --> 00:05:56,320 The animal learns that the bell predicts the food coming. 141 00:05:56,320 --> 00:05:57,860 And so here's the basic idea. 142 00:05:57,860 --> 00:06:00,750 At first, the tone does not drive salivation, right? 143 00:06:00,750 --> 00:06:03,780 There's no reason to salivate when you hear tone by itself. 144 00:06:03,780 --> 00:06:05,660 But as the animal discovers that the tone predicts the 145 00:06:05,660 --> 00:06:07,630 food, the tone predicts the food, the tone predicts the 146 00:06:07,630 --> 00:06:11,290 food, then all of a sudden the animal starts to learn that 147 00:06:11,290 --> 00:06:15,020 simply the tone alone will drive the salivation. 148 00:06:15,020 --> 00:06:18,040 Now this stimulus is conditioned. 149 00:06:18,040 --> 00:06:20,160 And you have a conditioned response that wasn't there to 150 00:06:20,160 --> 00:06:21,060 start with. 151 00:06:21,060 --> 00:06:21,870 It's very simple, right? 152 00:06:21,870 --> 00:06:23,590 You learn the two are associated. 153 00:06:23,590 --> 00:06:27,440 And now you begin to have a biological response to the 154 00:06:27,440 --> 00:06:29,890 tone alone. 155 00:06:29,890 --> 00:06:32,760 And typically what happens is here's a learning curve. 156 00:06:32,760 --> 00:06:35,090 Here's how often perhaps you salivate. 157 00:06:35,090 --> 00:06:37,810 And across trials you learn more and more of that. 158 00:06:37,810 --> 00:06:42,310 And you finally plateau at some level of performance. 159 00:06:42,310 --> 00:06:45,540 So a neat thing about Pavlov is, although there weren't a 160 00:06:45,540 --> 00:06:47,700 lot of movies at his time, there is a little bit of film 161 00:06:47,700 --> 00:06:49,500 of him and his actual dogs. 162 00:06:49,500 --> 00:06:52,195 So here's Pavlov's actual dogs. 163 00:07:04,240 --> 00:07:06,170 So we said conditioning is going to do two magical 164 00:07:06,170 --> 00:07:06,850 things, right? 165 00:07:06,850 --> 00:07:08,380 It's going to let us predict the future on the 166 00:07:08,380 --> 00:07:09,260 basis of the past. 167 00:07:09,260 --> 00:07:11,670 And it is a very simple experiment. 168 00:07:11,670 --> 00:07:15,910 Now the bell predicts the food coming up and the salivation. 169 00:07:15,910 --> 00:07:17,700 And it imbues the world with meaning. 170 00:07:17,700 --> 00:07:20,780 In a sense, the bell now means, here's dinner, right? 171 00:07:20,780 --> 00:07:22,030 The bell is not just a bell. 172 00:07:22,030 --> 00:07:23,425 It's something with meaning and significance. 173 00:07:26,856 --> 00:07:30,650 These are another example of classical conditioning. 174 00:07:30,650 --> 00:07:33,350 And it's got some sophisticated properties. 175 00:07:33,350 --> 00:07:35,990 You could go, well, balloons popping water in your face, 176 00:07:35,990 --> 00:07:38,200 food for dinner, that's pretty basic stuff. 177 00:07:38,200 --> 00:07:40,610 But here are some sophisticated properties of 178 00:07:40,610 --> 00:07:42,490 these as learning mechanisms. 179 00:07:42,490 --> 00:07:46,600 So one was already talked about, extinction, that, if 180 00:07:46,600 --> 00:07:50,940 the bell continues to ring and no food arrives, the learning 181 00:07:50,940 --> 00:07:53,000 gets weakened, weakened, and disappears. 182 00:07:53,000 --> 00:07:54,590 So you can learn something. 183 00:07:54,590 --> 00:07:57,380 And you can unlearn something because it's no longer an 184 00:07:57,380 --> 00:07:59,900 effective predictor of the future. 185 00:07:59,900 --> 00:08:01,990 Generalization is kind of cued also. 186 00:08:01,990 --> 00:08:05,680 So imagine the frequency of the stimulus is this, just the 187 00:08:05,680 --> 00:08:07,790 specific frequency of the tone. 188 00:08:07,790 --> 00:08:13,250 Well, if the tone is a little bit different, you might still 189 00:08:13,250 --> 00:08:14,810 think food is coming, right? 190 00:08:14,810 --> 00:08:18,810 So it's not that you learn the precise thing. 191 00:08:18,810 --> 00:08:20,900 You'll still respond if it's a pretty similar tone. 192 00:08:20,900 --> 00:08:23,620 And as the tone gets more and more different, the response 193 00:08:23,620 --> 00:08:25,090 gets weaker and weaker, right? 194 00:08:25,090 --> 00:08:27,020 So that's smart generalization. 195 00:08:27,020 --> 00:08:28,490 I stay pretty close to the tone. 196 00:08:28,490 --> 00:08:29,470 I go for it. 197 00:08:29,470 --> 00:08:31,970 I get further away, eh, not so sure. 198 00:08:31,970 --> 00:08:33,370 It's not all or none. 199 00:08:33,370 --> 00:08:36,610 It's reasonably graded and sensitive to the specifics of 200 00:08:36,610 --> 00:08:38,200 the situation. 201 00:08:38,200 --> 00:08:42,559 There's also one fascinating thing, which is your acquired 202 00:08:42,559 --> 00:08:45,150 conditioned response is extinguished because you're 203 00:08:45,150 --> 00:08:47,950 only hearing the bell and there's no more food coming. 204 00:08:47,950 --> 00:08:53,290 But when you come back in, you often have a kind of preserved 205 00:08:53,290 --> 00:08:54,900 recovery function of that CS. 206 00:08:54,900 --> 00:08:57,050 It's like it's lurking in you somewhere. 207 00:08:57,050 --> 00:09:00,480 It's like I have a mental note that this bell might mean 208 00:09:00,480 --> 00:09:02,250 food, even though it was extinguished. 209 00:09:02,250 --> 00:09:04,590 It's still pretty interesting stimulus. 210 00:09:04,590 --> 00:09:07,300 So you'd have some easily spontaneous recovery of these 211 00:09:07,300 --> 00:09:11,290 kinds of learned conditioning. 212 00:09:11,290 --> 00:09:12,990 It can also be sophisticated in other ways. 213 00:09:12,990 --> 00:09:16,270 And let me give you a couple of examples. 214 00:09:16,270 --> 00:09:17,970 You can learn to discriminate between two things. 215 00:09:17,970 --> 00:09:20,670 We've only talked about one thing so far, bell, food. 216 00:09:20,670 --> 00:09:23,920 But what if you have a black patch, for example, that 217 00:09:23,920 --> 00:09:26,570 predicts food coming, and a gray one that predicts it's 218 00:09:26,570 --> 00:09:28,990 not coming, you can distinguish between those two. 219 00:09:28,990 --> 00:09:30,720 You can get second order conditioning. 220 00:09:30,720 --> 00:09:33,080 And let me show you that. 221 00:09:33,080 --> 00:09:35,940 The dog learns that the bell predicts food. 222 00:09:35,940 --> 00:09:38,180 Now, there's no food presented for a while. 223 00:09:38,180 --> 00:09:41,330 But there's a black patch presented that you see 224 00:09:41,330 --> 00:09:42,720 presented with the tone. 225 00:09:42,720 --> 00:09:44,740 Because you know the tone predicts food, the animal 226 00:09:44,740 --> 00:09:46,740 figures, well, maybe this is a pretty good hint 227 00:09:46,740 --> 00:09:47,840 that food is coming. 228 00:09:47,840 --> 00:09:50,740 And after a while, it begins to salivate to this one, even 229 00:09:50,740 --> 00:09:54,620 though it was never directly paired with the food itself. 230 00:09:54,620 --> 00:09:56,550 So it's learning. 231 00:09:56,550 --> 00:09:59,340 It's a transitive inference that it's making. 232 00:09:59,340 --> 00:10:02,490 If the bell predicts it and the square goes with the bell, 233 00:10:02,490 --> 00:10:04,570 at some point I'll start salivating for the square 234 00:10:04,570 --> 00:10:06,120 because that sounds like a pretty good candidate for 235 00:10:06,120 --> 00:10:06,970 predicting the future. 236 00:10:06,970 --> 00:10:08,590 So that's pretty sophisticated, too. 237 00:10:08,590 --> 00:10:10,990 You don't even have food in this part. 238 00:10:10,990 --> 00:10:13,730 But it's even more sophisticated than that. 239 00:10:13,730 --> 00:10:16,290 So look at these two things. 240 00:10:16,290 --> 00:10:17,850 And you can think about, I'm going to let you think about 241 00:10:17,850 --> 00:10:18,820 this for a moment. 242 00:10:18,820 --> 00:10:20,110 Here are two groups of animals. 243 00:10:20,110 --> 00:10:20,810 Or it could be people. 244 00:10:20,810 --> 00:10:22,950 It would work just the same way. 245 00:10:22,950 --> 00:10:25,610 So here's the conditioned stimulus and the unconditioned 246 00:10:25,610 --> 00:10:28,360 stimulus, for example, a bell and the food, the bell and the 247 00:10:28,360 --> 00:10:29,920 food, the bell and the food. 248 00:10:29,920 --> 00:10:32,600 One group only gets bell, food, bell, 249 00:10:32,600 --> 00:10:34,940 food kinds of pairings. 250 00:10:34,940 --> 00:10:36,520 The other group gets the same pairings. 251 00:10:36,520 --> 00:10:41,750 But in between, they'll have a random food reward, two random 252 00:10:41,750 --> 00:10:45,130 bells, a random food, random bell. 253 00:10:45,130 --> 00:10:46,830 Now, the pairings are identical. 254 00:10:46,830 --> 00:10:48,560 That's what's shown in orange. 255 00:10:48,560 --> 00:10:52,410 This group will learn better and faster, more strongly, 256 00:10:52,410 --> 00:10:53,290 than this group. 257 00:10:53,290 --> 00:10:55,380 Even though they've had the identical number of pairings. 258 00:10:55,380 --> 00:10:59,210 Why will this group learn more slowly? 259 00:10:59,210 --> 00:11:01,560 Why will, even though they have an identical number of 260 00:11:01,560 --> 00:11:03,470 times that they got the tone and the food, the 261 00:11:03,470 --> 00:11:04,090 tone and the food-- 262 00:11:04,090 --> 00:11:04,385 Yeah? 263 00:11:04,385 --> 00:11:07,190 AUDIENCE: Is there a little bit of extinction between-- 264 00:11:07,190 --> 00:11:09,090 PROFESSOR: It's an idea that is an extinction between-- 265 00:11:09,090 --> 00:11:09,970 I think that's a good idea. 266 00:11:09,970 --> 00:11:11,940 So that's a very good answer, the extinction between-- 267 00:11:11,940 --> 00:11:15,650 because, when you get this CS alone, you're 268 00:11:15,650 --> 00:11:17,770 fighting the learning. 269 00:11:17,770 --> 00:11:20,170 But it's telling you the system is smart, right? 270 00:11:20,170 --> 00:11:23,420 Because what it's kind of doing is saying, look, eh, CS 271 00:11:23,420 --> 00:11:26,550 alone, not always a reliable predictor. 272 00:11:26,550 --> 00:11:27,560 So I'll learn it. 273 00:11:27,560 --> 00:11:30,010 But I'm not going to believe it so strongly, right? 274 00:11:30,010 --> 00:11:32,780 It's basically because you're sticking extinction in between 275 00:11:32,780 --> 00:11:33,410 those, right? 276 00:11:33,410 --> 00:11:35,460 You're also having some trials with the food that aren't 277 00:11:35,460 --> 00:11:37,000 predicted either, right? 278 00:11:37,000 --> 00:11:40,300 So it's like the animal or you are learning what's a really 279 00:11:40,300 --> 00:11:41,660 excellent signal. 280 00:11:41,660 --> 00:11:43,750 And if there's other stuff in there that gives the signal 281 00:11:43,750 --> 00:11:47,180 not the same predictive power for the future you learn more 282 00:11:47,180 --> 00:11:49,360 slowly unless certainly that relationship. 283 00:11:49,360 --> 00:11:49,650 OK? 284 00:11:49,650 --> 00:11:51,385 So that's pretty smart, too. 285 00:11:51,385 --> 00:11:55,440 Here's another way in which it's smart. 286 00:11:55,440 --> 00:11:56,745 And I'll tell you the phenomenon. 287 00:11:56,745 --> 00:12:00,360 And you think about why you think learning might work this 288 00:12:00,360 --> 00:12:01,940 way in animals and people. 289 00:12:01,940 --> 00:12:05,110 So pretend there's a tone that predicts food just like we've 290 00:12:05,110 --> 00:12:07,280 been talking about all the way through. 291 00:12:07,280 --> 00:12:08,600 Food stops. 292 00:12:08,600 --> 00:12:11,850 And now you get a tone and a light, a tone and a light, a 293 00:12:11,850 --> 00:12:12,250 tone and a light. 294 00:12:12,250 --> 00:12:13,840 The tone predicts it's food. 295 00:12:13,840 --> 00:12:14,800 You're getting this part, again, 296 00:12:14,800 --> 00:12:16,850 without food being presented. 297 00:12:16,850 --> 00:12:18,870 And now you get the light alone. 298 00:12:18,870 --> 00:12:20,340 OK? 299 00:12:20,340 --> 00:12:24,440 Or everything's the same. 300 00:12:24,440 --> 00:12:26,620 But there isn't the initial learning with the tone. 301 00:12:29,230 --> 00:12:31,250 And then in the end you're tested for the lights. 302 00:12:31,250 --> 00:12:33,890 And what happens is there's less conditioning to the light 303 00:12:33,890 --> 00:12:37,280 here than here, even though you've seen a light an exact 304 00:12:37,280 --> 00:12:38,570 even number of times. 305 00:12:38,570 --> 00:12:44,170 Why is the conditioning to the light weaker here? 306 00:12:44,170 --> 00:12:45,920 Why is it weaker here than here? 307 00:12:49,960 --> 00:12:54,260 This is an information problem for the brain to solve about 308 00:12:54,260 --> 00:12:57,480 the truth of the world. 309 00:12:57,480 --> 00:12:59,740 Well, our interpretation is this. 310 00:12:59,740 --> 00:13:02,250 Tone, food, tone, food, you go, good, I got it. 311 00:13:02,250 --> 00:13:03,530 Easy, right? 312 00:13:03,530 --> 00:13:06,290 Now you get tone and light together for a while. 313 00:13:06,290 --> 00:13:09,930 And you go, OK, I know the tone is really important. 314 00:13:09,930 --> 00:13:11,100 The light, I don't know. 315 00:13:11,100 --> 00:13:12,580 The tone was perfectly good. 316 00:13:12,580 --> 00:13:16,760 The light seems redundant, informationally redundant. 317 00:13:16,760 --> 00:13:19,500 So now when it's a light alone, you don't have so much 318 00:13:19,500 --> 00:13:19,920 conditioning. 319 00:13:19,920 --> 00:13:22,150 Because you didn't care about the light so much. 320 00:13:22,150 --> 00:13:23,580 The food had all the information you needed. 321 00:13:23,580 --> 00:13:25,250 The light was informationally redundant, right? 322 00:13:25,250 --> 00:13:26,880 It was superfluous. 323 00:13:26,880 --> 00:13:30,590 Here you only got trained with the tone and the lights. 324 00:13:30,590 --> 00:13:33,490 So you never made a bet on what's the critical thing. 325 00:13:33,490 --> 00:13:35,210 They both predicted evenly. 326 00:13:35,210 --> 00:13:37,650 So when the light comes by itself, that was always 327 00:13:37,650 --> 00:13:40,430 something that seemed like a pretty good signal equally 328 00:13:40,430 --> 00:13:41,760 good to the tone. 329 00:13:41,760 --> 00:13:44,935 So that's very sophisticated that the animal learning it or 330 00:13:44,935 --> 00:13:46,930 the human learning it is deciding what's the 331 00:13:46,930 --> 00:13:49,880 information that's diagnostic and what's the information 332 00:13:49,880 --> 00:13:50,520 that's redundant. 333 00:13:50,520 --> 00:13:52,560 And it's conditioning across that, right? 334 00:13:52,560 --> 00:13:54,700 So it's not just very simple reflexes. 335 00:13:54,700 --> 00:13:55,950 It's sophisticated learning. 336 00:13:59,260 --> 00:14:00,720 So now I'm going to switch gears. 337 00:14:00,720 --> 00:14:02,640 So we're going to have a demonstration that involves 338 00:14:02,640 --> 00:14:07,660 the opposite of fearful things like spraying water and 339 00:14:07,660 --> 00:14:09,610 popping balloons. 340 00:14:09,610 --> 00:14:11,390 But I can't tell you too much about this. 341 00:14:11,390 --> 00:14:14,620 But I can tell you it'll be pretty pleasant. 342 00:14:14,620 --> 00:14:16,010 But I need somebody who's willing to step out of the 343 00:14:16,010 --> 00:14:17,740 room for a moment to do this. 344 00:14:17,740 --> 00:14:20,545 OK, thanks. 345 00:14:20,545 --> 00:14:22,630 Yeah, yeah, you can safely take off your jacket now. 346 00:14:28,090 --> 00:14:30,050 And what's your name? 347 00:14:30,050 --> 00:14:30,730 AUDIENCE: I'm Sam. 348 00:14:30,730 --> 00:14:32,270 PROFESSOR: Sam, OK. 349 00:14:32,270 --> 00:14:34,300 Try not to listen if possible, OK? 350 00:14:34,300 --> 00:14:35,550 AUDIENCE: Yeah. 351 00:14:37,596 --> 00:14:40,780 PROFESSOR: So you guys are going to help me perform 352 00:14:40,780 --> 00:14:42,580 instrumental conditioning. 353 00:14:42,580 --> 00:14:44,430 And then we'll discuss what that is. 354 00:14:44,430 --> 00:14:48,470 Our job is to have Sam come over here and pick this up. 355 00:14:48,470 --> 00:14:50,230 OK? 356 00:14:50,230 --> 00:14:54,490 Your job, as Sam comes in for every action he takes, a step 357 00:14:54,490 --> 00:14:57,100 this way, you played this game as a kid, or an arm going in 358 00:14:57,100 --> 00:15:00,230 the right direction, applaud if he's doing something that 359 00:15:00,230 --> 00:15:03,460 gets him closer to the goal of picking up the brain and 360 00:15:03,460 --> 00:15:06,225 withhold your applause if he's doing something that's taking 361 00:15:06,225 --> 00:15:07,170 him away from that. 362 00:15:07,170 --> 00:15:07,790 Is that OK? 363 00:15:07,790 --> 00:15:09,040 All right, OK. 364 00:15:11,530 --> 00:15:15,610 So we're going to discuss what instrumental conditioning is. 365 00:15:15,610 --> 00:15:17,740 And then we're going to slightly think philosophically 366 00:15:17,740 --> 00:15:20,610 about whether this is how you learn to do everything. 367 00:15:20,610 --> 00:15:24,710 You get to choose who the applause comes from, your 368 00:15:24,710 --> 00:15:29,530 parents, your family, your friends, medical schools and 369 00:15:29,530 --> 00:15:31,790 law schools and graduate schools. 370 00:15:31,790 --> 00:15:34,570 You get to choose who the applause comes from, about how 371 00:15:34,570 --> 00:15:37,600 much of our lives is responding to the applause 372 00:15:37,600 --> 00:15:38,630 that we care about. 373 00:15:38,630 --> 00:15:41,170 So we'll start more simply, less philosophically. 374 00:15:41,170 --> 00:15:41,880 Let's think about this. 375 00:15:41,880 --> 00:15:44,120 The examples we talked so far in classical conditioning all 376 00:15:44,120 --> 00:15:47,900 build on what feels like a basic reflex, salivating for 377 00:15:47,900 --> 00:15:50,550 food, flinching for water, right? 378 00:15:50,550 --> 00:15:55,310 So if we ask somebody, why do you work hard, there's not to 379 00:15:55,310 --> 00:15:57,780 be very good explanation in terms of those initial 380 00:15:57,780 --> 00:15:59,240 reflexes, right? 381 00:15:59,240 --> 00:16:00,010 You're not flinching. 382 00:16:00,010 --> 00:16:02,820 You're not looking for food or water. 383 00:16:02,820 --> 00:16:05,680 And yet much of our life is involved in endeavors that we 384 00:16:05,680 --> 00:16:08,850 learn where there's no primary reflex that we 385 00:16:08,850 --> 00:16:10,400 can see in the story. 386 00:16:10,400 --> 00:16:13,320 And so operant or instrumental conditioning, those two words 387 00:16:13,320 --> 00:16:16,320 are interchangeable, is the way that psychologists have 388 00:16:16,320 --> 00:16:19,250 tried to understand those forms of learning. 389 00:16:19,250 --> 00:16:20,990 And it began something like this. 390 00:16:20,990 --> 00:16:24,060 And this is from Thorndike. 391 00:16:24,060 --> 00:16:25,810 He put a cat into a puzzle box. 392 00:16:25,810 --> 00:16:28,250 I'll show you what the box looks like or a sketch of it. 393 00:16:28,250 --> 00:16:30,890 The cat had to unlatch a door by pulling a latch. 394 00:16:30,890 --> 00:16:33,370 So here's, the cat's in a box like this. 395 00:16:33,370 --> 00:16:37,250 To get out, which cat wants to get out, he or she has to push 396 00:16:37,250 --> 00:16:38,590 this, which will push the latch. 397 00:16:38,590 --> 00:16:39,950 Now, that's not obvious to a cat. 398 00:16:39,950 --> 00:16:41,570 It wouldn't be necessarily obvious to a person but 399 00:16:41,570 --> 00:16:44,670 definitely not obvious to a cat. 400 00:16:44,670 --> 00:16:48,650 And the animal learned it by trial and error sort of like 401 00:16:48,650 --> 00:16:49,830 Sam did, right? 402 00:16:49,830 --> 00:16:53,510 And where is the unconditioned stimulus? 403 00:16:53,510 --> 00:16:56,230 There's no salivation or any reward like that. 404 00:16:56,230 --> 00:17:00,090 So Thorndike proposed that people and animals learn 405 00:17:00,090 --> 00:17:02,550 things by the consequence of their response. 406 00:17:02,550 --> 00:17:04,970 And the Law of Effect says the consequence of a response 407 00:17:04,970 --> 00:17:07,540 determines whether it is strengthened or weakened. 408 00:17:07,540 --> 00:17:08,339 You can be rewarded. 409 00:17:08,339 --> 00:17:09,520 That will strengthen a response. 410 00:17:09,520 --> 00:17:10,349 You can have no reward. 411 00:17:10,349 --> 00:17:11,010 That will weaken it. 412 00:17:11,010 --> 00:17:11,930 You can be punished. 413 00:17:11,930 --> 00:17:14,790 That will greatly weaken the response. 414 00:17:14,790 --> 00:17:15,859 So here's the cat. 415 00:17:15,859 --> 00:17:17,500 Here's an example of a cat. 416 00:17:17,500 --> 00:17:20,260 At first, it takes a long time for the cat to get out of 417 00:17:20,260 --> 00:17:22,520 there as it tries different things. 418 00:17:22,520 --> 00:17:25,240 But over repeated trials, the cat can get out 419 00:17:25,240 --> 00:17:26,440 really, really fast. 420 00:17:26,440 --> 00:17:30,080 And how Thorndike, by observation and thinking, 421 00:17:30,080 --> 00:17:32,330 thought about it was at the beginning the animal 422 00:17:32,330 --> 00:17:33,100 wants to get out. 423 00:17:33,100 --> 00:17:34,670 It has no idea how it gets out. 424 00:17:34,670 --> 00:17:36,470 So it does everything it can. 425 00:17:36,470 --> 00:17:37,700 It scratches at the bars. 426 00:17:37,700 --> 00:17:38,750 It pushes at the ceiling. 427 00:17:38,750 --> 00:17:39,520 It digs through the floor. 428 00:17:39,520 --> 00:17:40,360 It howls. 429 00:17:40,360 --> 00:17:42,300 And at some moment, it presses a lever. 430 00:17:42,300 --> 00:17:44,300 And that's good, right? 431 00:17:44,300 --> 00:17:46,950 So here's different things the animal starts with thinking 432 00:17:46,950 --> 00:17:48,110 might be good ideas. 433 00:17:48,110 --> 00:17:51,030 But because it discovers with repeated efforts, repeated 434 00:17:51,030 --> 00:17:52,250 trials, that pressing the lever 435 00:17:52,250 --> 00:17:54,390 works, these get weakened. 436 00:17:54,390 --> 00:17:55,480 This gets strengthened. 437 00:17:55,480 --> 00:17:58,010 So the consequence of a response determines whether 438 00:17:58,010 --> 00:17:59,900 it's strengthened or weakened. 439 00:17:59,900 --> 00:18:03,450 And now there's no salivating or reflex or 440 00:18:03,450 --> 00:18:04,280 anything like that. 441 00:18:04,280 --> 00:18:05,780 You can learn then almost anything. 442 00:18:08,450 --> 00:18:11,420 So this was picked up a lot in the US. 443 00:18:11,420 --> 00:18:12,670 John Watson is a famous name. 444 00:18:12,670 --> 00:18:18,410 You'll see kind of a haunting film of him performing this 445 00:18:18,410 --> 00:18:20,730 kinds of stuff with infants in a kind of a way that we would 446 00:18:20,730 --> 00:18:21,980 never allow ethically now. 447 00:18:21,980 --> 00:18:22,530 It's not horrible. 448 00:18:22,530 --> 00:18:24,590 But it's pretty bad. 449 00:18:24,590 --> 00:18:25,790 But it's a famous thing. 450 00:18:25,790 --> 00:18:27,790 And you'll just get a feeling of the power of this. 451 00:18:27,790 --> 00:18:31,870 Behaviorism, that if you study behavior, instead of going for 452 00:18:31,870 --> 00:18:35,200 things like unobservable thoughts, that you would just 453 00:18:35,200 --> 00:18:36,350 do observable actions. 454 00:18:36,350 --> 00:18:38,310 And you wouldn't make inferences about the mind that 455 00:18:38,310 --> 00:18:39,420 you can't measure. 456 00:18:39,420 --> 00:18:40,600 You make a stimulus. 457 00:18:40,600 --> 00:18:41,760 You measure the response. 458 00:18:41,760 --> 00:18:43,400 And that's it. 459 00:18:43,400 --> 00:18:45,480 No fundamental differences between animals and humans, 460 00:18:45,480 --> 00:18:47,570 and you make laws that describe the relationships 461 00:18:47,570 --> 00:18:50,260 between stimulating the environment and physical 462 00:18:50,260 --> 00:18:53,980 behaviors that you can observe or responses out in the world 463 00:18:53,980 --> 00:18:55,830 that an organism has. 464 00:18:55,830 --> 00:18:57,010 So I'll come back to this a little bit. 465 00:18:57,010 --> 00:18:59,620 But one of the ways he wanted to show that anybody could 466 00:18:59,620 --> 00:19:02,920 learn anything, because right now, don't forget, the way we 467 00:19:02,920 --> 00:19:04,920 set this up, anybody can learn anything. 468 00:19:04,920 --> 00:19:08,060 You can applaud anything, right? 469 00:19:08,060 --> 00:19:12,230 So you can teach anybody anything by this perspective. 470 00:19:12,230 --> 00:19:14,010 And he wanted to show that he could teach an infant 471 00:19:14,010 --> 00:19:16,500 initially, whose name was Little Albert. 472 00:19:16,500 --> 00:19:18,040 That was his cover name. 473 00:19:18,040 --> 00:19:22,255 Nobody actually knows his real name or what happened to him. 474 00:19:22,255 --> 00:19:24,501 Well, nothing horrible happened to this person we 475 00:19:24,501 --> 00:19:25,130 know about. 476 00:19:25,130 --> 00:19:28,200 And now you would never do an experiment like this. 477 00:19:28,200 --> 00:19:29,310 You have this in your notes. 478 00:19:29,310 --> 00:19:33,530 This is a long version of the movies taken of this actual 479 00:19:33,530 --> 00:19:35,900 sort of experiment that we wouldn't do now. 480 00:19:35,900 --> 00:19:37,590 And I'll show you a short version in a couple minutes 481 00:19:37,590 --> 00:19:40,475 where he got an infant to become afraid of a rabbit that 482 00:19:40,475 --> 00:19:44,130 the infant previously was not afraid of. 483 00:19:44,130 --> 00:19:46,180 And then the other huge name in this is B.F. Skinner who 484 00:19:46,180 --> 00:19:50,810 worked at Harvard, consequence of a response, again, creates 485 00:19:50,810 --> 00:19:52,380 the responses that we make in the world. 486 00:19:52,380 --> 00:19:55,310 And his sense response is not just the laboratory one but 487 00:19:55,310 --> 00:19:58,762 things we do in the world with each other and to each other. 488 00:19:58,762 --> 00:20:01,240 And so the classical conditioning, the 489 00:20:01,240 --> 00:20:03,130 CS elicits the CR. 490 00:20:03,130 --> 00:20:05,520 The conditioned stimulus, the bell, elicits the conditioned 491 00:20:05,520 --> 00:20:06,820 response, salivation. 492 00:20:06,820 --> 00:20:09,820 But in instrumental conditioning CRs are omitted. 493 00:20:09,820 --> 00:20:12,410 And you can do really anything that's in the range of what an 494 00:20:12,410 --> 00:20:14,160 organism can do. 495 00:20:14,160 --> 00:20:17,410 And then he could show kind of impressively how you might 496 00:20:17,410 --> 00:20:19,550 imagine a chain of these things would 497 00:20:19,550 --> 00:20:21,440 lead to complex behaviors. 498 00:20:21,440 --> 00:20:25,010 So he would teach pigeons and other animals to first to 499 00:20:25,010 --> 00:20:27,500 click to get a pellet, then where to click it, then to 500 00:20:27,500 --> 00:20:29,370 where to face before you click it. 501 00:20:29,370 --> 00:20:31,670 And he would just keep adding little pieces, shape the 502 00:20:31,670 --> 00:20:34,560 pieces up from the first step to the second step, until he 503 00:20:34,560 --> 00:20:38,120 got pretty complex behaviors. 504 00:20:38,120 --> 00:20:40,050 OK, so now I'm going to show a slightly longer clip that has 505 00:20:40,050 --> 00:20:42,580 in it these kinds of experiments and Little Albert. 506 00:20:45,700 --> 00:20:54,180 It's kind of cool because in the sense that we can actually 507 00:20:54,180 --> 00:20:56,260 see films of these sort of historical things, which often 508 00:20:56,260 --> 00:20:57,995 we don't get to do in various science fields. 509 00:21:07,800 --> 00:21:13,330 So let's talk about that for just one moment. 510 00:21:13,330 --> 00:21:16,315 So we can talk about primary reinforcers like food, thrist, 511 00:21:16,315 --> 00:21:18,830 or pain, things that go with life, right, pain, not being 512 00:21:18,830 --> 00:21:20,300 injured, food, and thirst. 513 00:21:20,300 --> 00:21:23,820 We can talk about secondary reinforcers that we know a lot 514 00:21:23,820 --> 00:21:28,520 about, money, attention of other people, praise from 515 00:21:28,520 --> 00:21:30,820 other people, admissions to places you want to be admitted 516 00:21:30,820 --> 00:21:33,000 to, promotions when you want to get them. 517 00:21:33,000 --> 00:21:36,020 There can be positive or negative rewards. 518 00:21:36,020 --> 00:21:36,890 And I'm going talk about one thing. 519 00:21:36,890 --> 00:21:42,280 But just for a moment the behaviorists said anybody can 520 00:21:42,280 --> 00:21:45,390 learn anything, right? 521 00:21:45,390 --> 00:21:46,260 We'll come back to that. 522 00:21:46,260 --> 00:21:50,050 So that's been a big debate whether anything can be 523 00:21:50,050 --> 00:21:51,890 equally learned with any other thing. 524 00:21:51,890 --> 00:21:56,180 And the other thing that's striking is B.F. Skinner 525 00:21:56,180 --> 00:21:59,070 particularly said that we have an illusion of free will, that 526 00:21:59,070 --> 00:22:00,760 all we are, are learning these things. 527 00:22:00,760 --> 00:22:04,330 And we might as well get over our illusion of free will. 528 00:22:04,330 --> 00:22:07,590 Scientifically, it's very hard for even the best scientific 529 00:22:07,590 --> 00:22:09,120 psychologists to tell you whether free 530 00:22:09,120 --> 00:22:10,270 will exists or not. 531 00:22:10,270 --> 00:22:12,290 That's not something that's easy for us to begin to 532 00:22:12,290 --> 00:22:14,440 imagine how to measure. 533 00:22:14,440 --> 00:22:16,230 These kinds of learning things are very 534 00:22:16,230 --> 00:22:17,160 impressive in many ways. 535 00:22:17,160 --> 00:22:18,410 And we'll talk about some of that. 536 00:22:20,940 --> 00:22:27,610 So let me talk to you about one way in which, for example, 537 00:22:27,610 --> 00:22:31,120 casinos get people to come back often and other things in 538 00:22:31,120 --> 00:22:33,930 life, without giving you money every time. 539 00:22:33,930 --> 00:22:36,220 Because then they wouldn't make money, right? 540 00:22:36,220 --> 00:22:38,320 So your book talks about a number of examples of this. 541 00:22:38,320 --> 00:22:41,070 But I'll just pick one, partial reinforcement. 542 00:22:41,070 --> 00:22:44,290 So this means when do you get, in these animals, when do you 543 00:22:44,290 --> 00:22:45,160 get a reward? 544 00:22:45,160 --> 00:22:48,070 Every time or only 1/3 of the time? 545 00:22:48,070 --> 00:22:52,290 So of course, you learn faster if you get a reward every time 546 00:22:52,290 --> 00:22:53,350 than if it's 1/3 of the time. 547 00:22:53,350 --> 00:22:55,020 You're not quite as impressed and excited. 548 00:22:55,020 --> 00:22:56,510 But here's the interesting thing. 549 00:22:56,510 --> 00:23:00,080 If you stop giving the reward, these animals will go on for 550 00:23:00,080 --> 00:23:02,390 quite a while before they extinguish. 551 00:23:02,390 --> 00:23:04,590 These animals will extinguish right away and stop 552 00:23:04,590 --> 00:23:05,330 responding. 553 00:23:05,330 --> 00:23:10,360 Why will these animals keep behaving for a time, even 554 00:23:10,360 --> 00:23:14,210 though you've entirely stopped the rewards? 555 00:23:14,210 --> 00:23:14,560 Yeah? 556 00:23:14,560 --> 00:23:18,760 AUDIENCE: Since they only got it 30% of the time anyway, it 557 00:23:18,760 --> 00:23:21,040 seems plausible to them that it would go a couple more 558 00:23:21,040 --> 00:23:22,380 times without them getting it. 559 00:23:22,380 --> 00:23:23,630 PROFESSOR: Yes, since they only got, the answer was, 560 00:23:23,630 --> 00:23:26,760 which was right, they only got it 30% of the time, so the 561 00:23:26,760 --> 00:23:30,010 first one or two nonpayments, right, they're going to go, 562 00:23:30,010 --> 00:23:33,090 well, it's coming up next, OK, in a couple more, OK, I'm 563 00:23:33,090 --> 00:23:36,600 coming up soon until they finally decide that the whole 564 00:23:36,600 --> 00:23:38,270 thing is over, right? 565 00:23:38,270 --> 00:23:41,000 So that's very interesting, the different reinforcement 566 00:23:41,000 --> 00:23:42,720 schedules, how they relate. 567 00:23:42,720 --> 00:23:45,390 You might think, well, you always learn better if you 568 00:23:45,390 --> 00:23:46,150 reward more. 569 00:23:46,150 --> 00:23:48,950 But in some cases, if you want to get people to keep doing 570 00:23:48,950 --> 00:23:51,810 stuff without direct reward, partial reinforcement is 571 00:23:51,810 --> 00:23:54,770 actually more powerful. 572 00:23:54,770 --> 00:23:57,720 Now we're going to move into dog experiments and then 573 00:23:57,720 --> 00:23:59,340 directly to sort of thinking about 574 00:23:59,340 --> 00:24:01,260 people and our own lives. 575 00:24:01,260 --> 00:24:04,910 So this is a famous, famous experiment at the University 576 00:24:04,910 --> 00:24:06,200 of Pennsylvania from Seligman. 577 00:24:06,200 --> 00:24:07,460 In this case, he used dogs. 578 00:24:07,460 --> 00:24:09,840 It's been done with many species in many circumstances. 579 00:24:09,840 --> 00:24:11,070 And he put them in a hammock. 580 00:24:11,070 --> 00:24:13,430 So there's two dogs on two sides of a thing that they 581 00:24:13,430 --> 00:24:14,130 walked around. 582 00:24:14,130 --> 00:24:16,170 And they don't see each other. 583 00:24:16,170 --> 00:24:21,850 The dog, A, would periodically get a shock. 584 00:24:21,850 --> 00:24:25,670 And it could stop the shock by pushing a panel near its nose. 585 00:24:25,670 --> 00:24:27,560 As soon as the shock came, it could turn its nose 586 00:24:27,560 --> 00:24:29,390 and stop the shock. 587 00:24:29,390 --> 00:24:33,930 Dog B on the other side of the divider would get exactly the 588 00:24:33,930 --> 00:24:34,910 same shock. 589 00:24:34,910 --> 00:24:37,370 It would stop at exactly the moment that the first dog 590 00:24:37,370 --> 00:24:38,740 stopped it. 591 00:24:38,740 --> 00:24:42,370 But dog B had no direct control, equal number of 592 00:24:42,370 --> 00:24:45,610 shocks, equal duration of objective pain, if you want to 593 00:24:45,610 --> 00:24:46,610 call it that way. 594 00:24:46,610 --> 00:24:49,180 One animal has the capacity to stop it. 595 00:24:49,180 --> 00:24:51,000 The other does not. 596 00:24:51,000 --> 00:24:53,060 So that's the first part of the experiment. 597 00:24:53,060 --> 00:24:54,840 And the second part of the experiment they're put into a 598 00:24:54,840 --> 00:24:56,090 shuttle box. 599 00:24:58,420 --> 00:25:01,890 It's just a box with a big divider in the middle. 600 00:25:01,890 --> 00:25:03,510 And they're just hanging out there. 601 00:25:03,510 --> 00:25:04,510 And then they hear a tone. 602 00:25:04,510 --> 00:25:07,450 At first, it means nothing to them. 603 00:25:07,450 --> 00:25:11,370 And then comes a shock on the grid on the floor. 604 00:25:11,370 --> 00:25:13,850 And at first, they don't know what to do. 605 00:25:13,850 --> 00:25:15,900 But then after a while, they get smart like the cat. 606 00:25:15,900 --> 00:25:18,120 And they jump over the divider to the other side. 607 00:25:18,120 --> 00:25:20,020 And it's all OK. 608 00:25:20,020 --> 00:25:22,680 So you can imagine that when they hear the tone pretty 609 00:25:22,680 --> 00:25:24,860 quickly that what do they learn to do? 610 00:25:24,860 --> 00:25:27,860 Jump over the divider and stop the pain. 611 00:25:27,860 --> 00:25:29,040 And here's the remarkable thing. 612 00:25:29,040 --> 00:25:31,910 For the animals who had the experience that pushing the 613 00:25:31,910 --> 00:25:35,290 panel stopped the pain, they learned that pretty quickly. 614 00:25:35,290 --> 00:25:39,800 For the animals who could not stop the pain initially, they 615 00:25:39,800 --> 00:25:42,680 don't learn how to escape when they could. 616 00:25:42,680 --> 00:25:46,030 It's as if the animals in group B, what they learned 617 00:25:46,030 --> 00:25:48,580 about life, it's painful. 618 00:25:48,580 --> 00:25:51,820 And you can't do anything about it. 619 00:25:51,820 --> 00:25:55,030 The animals in group A learned pain happens. 620 00:25:55,030 --> 00:25:58,560 But I can do something about it. 621 00:25:58,560 --> 00:26:00,190 So here's the animal. 622 00:26:00,190 --> 00:26:02,030 At first, it doesn't know what to do. 623 00:26:02,030 --> 00:26:02,870 It's getting the shock. 624 00:26:02,870 --> 00:26:06,400 But with trials, the animal learns to jump and jump so 625 00:26:06,400 --> 00:26:07,990 quickly that it doesn't have almost any problem. 626 00:26:07,990 --> 00:26:08,720 Here's a tone, boom. 627 00:26:08,720 --> 00:26:09,710 It's out of there. 628 00:26:09,710 --> 00:26:11,730 It learns to avoid the pain. 629 00:26:11,730 --> 00:26:14,620 But what happens to the animals who either had the 630 00:26:14,620 --> 00:26:16,240 escapable shock or the ones? 631 00:26:16,240 --> 00:26:20,400 The ones who could poke their nose, they also learned this. 632 00:26:20,400 --> 00:26:23,350 The ones who could jump over, there's nothing preventing 633 00:26:23,350 --> 00:26:24,780 them from learning that. 634 00:26:24,780 --> 00:26:27,490 They just don't escape. 635 00:26:27,490 --> 00:26:30,950 It's what Seligman called learned helplessness, that the 636 00:26:30,950 --> 00:26:34,550 animals learn, from the first experience, their conclusion 637 00:26:34,550 --> 00:26:36,350 was pain happens. 638 00:26:36,350 --> 00:26:37,840 And I can't do anything about it. 639 00:26:37,840 --> 00:26:39,600 And then when they get in a second situation where they 640 00:26:39,600 --> 00:26:43,060 can do a lot about it, they continue to feel helpless and 641 00:26:43,060 --> 00:26:44,220 not do something about it. 642 00:26:44,220 --> 00:26:46,620 It's like a deep lesson about life, like when do you think 643 00:26:46,620 --> 00:26:49,000 you can make your situation better and when do you think 644 00:26:49,000 --> 00:26:50,600 there's nothing I can do about it? 645 00:26:50,600 --> 00:26:52,140 It's just bad, bad, bad. 646 00:26:54,700 --> 00:26:57,010 So there's ideas about that some of this might be related 647 00:26:57,010 --> 00:26:59,720 to people who struggle with depression but also other 648 00:26:59,720 --> 00:27:00,290 things in life. 649 00:27:00,290 --> 00:27:03,820 So here's the idea from Seligman as an explanation of 650 00:27:03,820 --> 00:27:06,460 how we have resilience. 651 00:27:06,460 --> 00:27:10,310 The idea is that we all get in sometimes in small doses, 652 00:27:10,310 --> 00:27:12,590 sometimes in tragically large doses, difficult 653 00:27:12,590 --> 00:27:13,940 things into our lives. 654 00:27:13,940 --> 00:27:15,630 How do we explain those difficulties that have 655 00:27:15,630 --> 00:27:17,930 happened to us, the setbacks we have? 656 00:27:17,930 --> 00:27:19,780 Do we think they're internal to us? 657 00:27:19,780 --> 00:27:21,980 I'm the kind of person who has bad luck. 658 00:27:21,980 --> 00:27:24,050 I'm the kind of person that these things happen to. 659 00:27:24,050 --> 00:27:25,420 Or is it external? 660 00:27:25,420 --> 00:27:29,440 Somebody gave me a test. 661 00:27:29,440 --> 00:27:30,380 What can I do about that? 662 00:27:30,380 --> 00:27:31,630 But next week the test is over. 663 00:27:31,630 --> 00:27:32,990 And life will be better. 664 00:27:32,990 --> 00:27:34,730 Is it global or specific? 665 00:27:34,730 --> 00:27:36,610 Does it just seem miserable all over? 666 00:27:36,610 --> 00:27:39,960 Or is it just one specific squirt in the face that this 667 00:27:39,960 --> 00:27:41,680 faculty member gave me? 668 00:27:41,680 --> 00:27:43,210 Is it stable or unstable? 669 00:27:43,210 --> 00:27:45,720 Are bad things just seeming to happen all the time? 670 00:27:45,720 --> 00:27:47,340 Or is it one occasion? 671 00:27:47,340 --> 00:27:49,590 And you can imagine that people who believe that bad 672 00:27:49,590 --> 00:27:51,220 things are internal, global, and stable 673 00:27:51,220 --> 00:27:53,520 will learn to be helpless. 674 00:27:53,520 --> 00:27:56,770 Everything seems dark and hopeless. 675 00:27:56,770 --> 00:27:59,330 People who believe that things are external, specific, and 676 00:27:59,330 --> 00:28:01,830 unstable think, the next time I can do better. 677 00:28:01,830 --> 00:28:03,840 I can bounce back from this and do better. 678 00:28:03,840 --> 00:28:06,470 And it's not only in terms of the sort of spirit of people 679 00:28:06,470 --> 00:28:09,740 and trying to get people to bounce back from setbacks. 680 00:28:09,740 --> 00:28:11,770 It's even things like job applications. 681 00:28:11,770 --> 00:28:14,050 Seligman did the following bet. 682 00:28:14,050 --> 00:28:15,840 I don't even know what corresponds to it before. 683 00:28:15,840 --> 00:28:18,290 But there used to be an era when people would go knock on 684 00:28:18,290 --> 00:28:19,670 doors and try to sell you insurance. 685 00:28:19,670 --> 00:28:20,590 We don't do that anymore, right? 686 00:28:20,590 --> 00:28:22,025 Because if somebody comes to our door, knocks on our door, 687 00:28:22,025 --> 00:28:23,860 we're calling the police. 688 00:28:23,860 --> 00:28:26,650 But when things were more calm, you'd open 689 00:28:26,650 --> 00:28:27,700 the door and talk. 690 00:28:27,700 --> 00:28:30,240 Or they would give you cold calls, which you don't get 691 00:28:30,240 --> 00:28:31,340 much more anymore these days. 692 00:28:31,340 --> 00:28:34,760 You guys, have you ever even heard that phrase? 693 00:28:34,760 --> 00:28:35,940 You'd work at some insurance company. 694 00:28:35,940 --> 00:28:37,590 And you'd go, here's some random telephone numbers. 695 00:28:37,590 --> 00:28:38,370 You call them up. 696 00:28:38,370 --> 00:28:40,430 And you try to sell them insurance. 697 00:28:40,430 --> 00:28:42,150 Usually, you get abused on the phone. 698 00:28:42,150 --> 00:28:43,440 Or somebody slams the doors. 699 00:28:43,440 --> 00:28:44,980 But these salespeople, they make commissions. 700 00:28:44,980 --> 00:28:45,850 And they keep going. 701 00:28:45,850 --> 00:28:47,690 And he said, let me get my test. 702 00:28:47,690 --> 00:28:49,780 He gave a questionnaire whether people interpret the 703 00:28:49,780 --> 00:28:52,750 world one way or the other way, the resilient way or the 704 00:28:52,750 --> 00:28:54,190 learned helplessness way. 705 00:28:54,190 --> 00:28:57,490 And you give your usual interview way for picking 706 00:28:57,490 --> 00:28:58,250 sales people. 707 00:28:58,250 --> 00:29:00,230 And let me see who picks a better salesperson. 708 00:29:00,230 --> 00:29:02,810 And his questionnaire picked a better salesperson. 709 00:29:02,810 --> 00:29:06,340 Because if you have a sales job where people often kick 710 00:29:06,340 --> 00:29:09,970 you out and hang up on you, you better be pretty external, 711 00:29:09,970 --> 00:29:12,430 specific, and unstable in your perception on things, right? 712 00:29:12,430 --> 00:29:15,300 Because when you have 100 people who hang up on you or 713 00:29:15,300 --> 00:29:18,970 close the door on you for one sale, you need to have a thick 714 00:29:18,970 --> 00:29:20,500 resilient skin. 715 00:29:20,500 --> 00:29:23,350 So there's lots of ways in which I think this speaks to 716 00:29:23,350 --> 00:29:27,680 how do we respond to the setbacks in our lives. 717 00:29:27,680 --> 00:29:30,120 So let me talk about some limits to conditioning. 718 00:29:30,120 --> 00:29:32,490 And some of these that I think touch also on what are the 719 00:29:32,490 --> 00:29:34,910 values by which we lead our lives. 720 00:29:34,910 --> 00:29:36,690 It's not going to be telling you that. 721 00:29:36,690 --> 00:29:39,550 But it'll make you think about some of the mechanics of what 722 00:29:39,550 --> 00:29:40,850 matters to you. 723 00:29:40,850 --> 00:29:42,800 So let me slide back for a moment. 724 00:29:42,800 --> 00:29:45,050 So the behaviorists said everybody can learn 725 00:29:45,050 --> 00:29:45,520 everything. 726 00:29:45,520 --> 00:29:46,790 Everybody can learn everything. 727 00:29:46,790 --> 00:29:48,720 In that sense, it was seen as a very democratic and 728 00:29:48,720 --> 00:29:50,510 egalitarian perspective. 729 00:29:50,510 --> 00:29:52,020 Everybody can learn everything. 730 00:29:52,020 --> 00:29:54,270 It's just how you set up the contingencies in the 731 00:29:54,270 --> 00:29:55,380 environment. 732 00:29:55,380 --> 00:29:59,920 And then Garcia did a famous experiment where he had rats 733 00:29:59,920 --> 00:30:02,940 get shocks or receive lithium chloride, 734 00:30:02,940 --> 00:30:05,830 which made them nauseous. 735 00:30:05,830 --> 00:30:10,590 And they either had a bright noisy stimulus. 736 00:30:10,590 --> 00:30:13,770 Or they tasted some sweet water. 737 00:30:13,770 --> 00:30:17,030 And even though these things were arbitrarily combined, the 738 00:30:17,030 --> 00:30:21,990 rats learned, who received shocked, learn to fear the 739 00:30:21,990 --> 00:30:25,060 bright noise, the bright light and the noise. 740 00:30:25,060 --> 00:30:26,860 And the rats who got the lithium chloride that made 741 00:30:26,860 --> 00:30:29,360 them nauseous learned to fear the sweet water. 742 00:30:29,360 --> 00:30:31,520 Do you see why? 743 00:30:31,520 --> 00:30:34,720 When we get nauseous, do we think like, boy, it was noisy 744 00:30:34,720 --> 00:30:35,150 last night? 745 00:30:35,150 --> 00:30:37,310 Or do you think, when you wake up in the morning and you feel 746 00:30:37,310 --> 00:30:39,890 nauseous, what's the first thing you think? 747 00:30:39,890 --> 00:30:42,650 PROFESSOR: Yeah, something I ate, right, we're prepared for 748 00:30:42,650 --> 00:30:47,740 internal nausea to think about food as a risk. 749 00:30:47,740 --> 00:30:49,830 And if it's a shock, we're prepared to think of 750 00:30:49,830 --> 00:30:51,430 something out there. 751 00:30:51,430 --> 00:30:54,560 So these animals instinctively, not on the 752 00:30:54,560 --> 00:30:59,730 basis of the training, instinctively said, if I feel 753 00:30:59,730 --> 00:31:02,660 a shock, it's probably related to this bright noisy stimulus 754 00:31:02,660 --> 00:31:03,880 that warns me. 755 00:31:03,880 --> 00:31:08,460 If I feel nauseous, it's related to the sugar water. 756 00:31:08,460 --> 00:31:10,190 And they had these opposite patterns of learning for the 757 00:31:10,190 --> 00:31:14,780 stimuli as if these animals were prepared to fear some 758 00:31:14,780 --> 00:31:15,860 things and not others. 759 00:31:15,860 --> 00:31:17,670 So it wasn't that everything could be matched with 760 00:31:17,670 --> 00:31:18,150 everything. 761 00:31:18,150 --> 00:31:20,620 And in fact, with humans, they've done things like show 762 00:31:20,620 --> 00:31:22,680 you pictures either of snakes and spiders 763 00:31:22,680 --> 00:31:24,260 or flowers and mushrooms. 764 00:31:24,260 --> 00:31:26,420 And then those predicted a shock, a 765 00:31:26,420 --> 00:31:27,900 small shock to humans. 766 00:31:27,900 --> 00:31:29,820 And they measured your galvanic skin response and 767 00:31:29,820 --> 00:31:31,020 autonomic response. 768 00:31:31,020 --> 00:31:32,950 When you think you're going to get a shock, you start to get 769 00:31:32,950 --> 00:31:34,790 a little sweaty. 770 00:31:34,790 --> 00:31:36,170 And what they found was-- 771 00:31:36,170 --> 00:31:38,840 and this will not shock you-- they had better conditioning, 772 00:31:38,840 --> 00:31:41,330 you more quickly and completely learned that a 773 00:31:41,330 --> 00:31:43,810 snake or a spider predicts a shock than 774 00:31:43,810 --> 00:31:45,740 a flower or a mushroom. 775 00:31:45,740 --> 00:31:49,620 Even though, objectively, they were just as predictive. 776 00:31:49,620 --> 00:31:52,280 There's something about snakes and spiders that gets us ready 777 00:31:52,280 --> 00:31:53,660 for scary stuff. 778 00:31:53,660 --> 00:31:55,830 And flowers and mushrooms, they really have to punish us 779 00:31:55,830 --> 00:31:59,820 before we think they're evil signals so, again, a human 780 00:31:59,820 --> 00:32:01,110 example of preparedness. 781 00:32:01,110 --> 00:32:02,530 We don't learn everything equally. 782 00:32:02,530 --> 00:32:05,480 We are prepared to learn some things and not others. 783 00:32:05,480 --> 00:32:09,050 Same thing with Little Albert, the child that you saw, that 784 00:32:09,050 --> 00:32:10,320 kind of a film has a funny status. 785 00:32:10,320 --> 00:32:12,500 And we'll talk about a few more things this semester 786 00:32:12,500 --> 00:32:14,920 where that experiment, why should that 787 00:32:14,920 --> 00:32:16,520 experiment not be done? 788 00:32:19,890 --> 00:32:20,420 Yeah? 789 00:32:20,420 --> 00:32:22,430 AUDIENCE: Nobody knows what happened to Little Albert. 790 00:32:22,430 --> 00:32:25,640 PROFESSOR: Yeah, well, you would never terrorize a child, 791 00:32:25,640 --> 00:32:28,290 right, to scare the child. 792 00:32:28,290 --> 00:32:31,860 So weirdly enough, though, these studies remain in the 793 00:32:31,860 --> 00:32:34,000 field because since we can't do them again. 794 00:32:34,000 --> 00:32:36,470 Because we know they're not right to do. 795 00:32:36,470 --> 00:32:38,330 We try to learn lessons from them so 796 00:32:38,330 --> 00:32:39,730 that we learn something. 797 00:32:39,730 --> 00:32:42,190 The rat worked. 798 00:32:42,190 --> 00:32:43,560 He got scared of a rat. 799 00:32:43,560 --> 00:32:44,640 And then he got scared of a bunny. 800 00:32:44,640 --> 00:32:46,930 He got scared of a cat, anything 801 00:32:46,930 --> 00:32:48,390 animal, anything furry. 802 00:32:48,390 --> 00:32:51,690 But if they gave him a wooden block, that didn't work. 803 00:32:51,690 --> 00:32:52,925 It's not everything. 804 00:32:52,925 --> 00:32:56,550 It has to be prepared from that. 805 00:32:56,550 --> 00:33:02,440 Here's another one that also challenges simple ideas of do 806 00:33:02,440 --> 00:33:05,700 we always learn simply to get a reward. 807 00:33:05,700 --> 00:33:09,170 What kind of learning might exist that's rewardless? 808 00:33:09,170 --> 00:33:12,160 So there's three groups of rats in a goal maze. 809 00:33:12,160 --> 00:33:14,470 There's food rewards every day for one group. 810 00:33:14,470 --> 00:33:15,420 They're the happy group. 811 00:33:15,420 --> 00:33:18,160 There's no rewards for a grumbling group. 812 00:33:18,160 --> 00:33:20,220 There's a third group that gets no reward for 10 days. 813 00:33:20,220 --> 00:33:21,630 And then it gets reward. 814 00:33:21,630 --> 00:33:25,160 So reward all the time, reward never, no reward for 10 days 815 00:33:25,160 --> 00:33:27,260 and then you start to get rewards all the time, and 816 00:33:27,260 --> 00:33:28,930 here's what happens to the learning. 817 00:33:28,930 --> 00:33:31,950 If you look at this green line, that's the food that 818 00:33:31,950 --> 00:33:32,970 gets rewarded all the time. 819 00:33:32,970 --> 00:33:37,010 So they make fewer errors so they can 820 00:33:37,010 --> 00:33:39,100 get to the food faster. 821 00:33:39,100 --> 00:33:41,480 If you look at the group that's never rewarded, this 822 00:33:41,480 --> 00:33:42,515 yellow line, it's pretty steady. 823 00:33:42,515 --> 00:33:45,510 It's like I'm hanging out, no reason to go anywhere. 824 00:33:45,510 --> 00:33:47,080 There's not much going on. 825 00:33:47,080 --> 00:33:48,720 Look at the orange group. 826 00:33:48,720 --> 00:33:52,120 For 10 days, up to here, they look like the yellow line, 827 00:33:52,120 --> 00:33:53,860 right, not getting rewarded. 828 00:33:53,860 --> 00:33:55,090 I'm just going to saunter around. 829 00:33:55,090 --> 00:33:56,800 There's nothing interesting to learn here. 830 00:33:56,800 --> 00:34:01,030 But boom that first day that it's rewarded, but just one 831 00:34:01,030 --> 00:34:02,765 day of learning, they're all caught up. 832 00:34:02,765 --> 00:34:04,470 They're all caught up. 833 00:34:04,470 --> 00:34:07,000 So people call this latent learning, that they were 834 00:34:07,000 --> 00:34:10,340 really learning a lot about their environment anyway, even 835 00:34:10,340 --> 00:34:12,350 though it was of no use at the moment. 836 00:34:12,350 --> 00:34:15,540 And then the moment it became valuable information, boom, 837 00:34:15,540 --> 00:34:17,100 they're all caught up. 838 00:34:17,100 --> 00:34:18,414 It's as if they learned all this information. 839 00:34:18,414 --> 00:34:19,420 And it was ready to go. 840 00:34:19,420 --> 00:34:22,510 Latent learning, the information is stored up just 841 00:34:22,510 --> 00:34:23,810 because they were in the environment, even though it 842 00:34:23,810 --> 00:34:24,800 was unrewarded. 843 00:34:24,800 --> 00:34:26,980 And as we go around, we often pick up stuff, even though 844 00:34:26,980 --> 00:34:28,300 there's no direct reward. 845 00:34:28,300 --> 00:34:31,460 And sometimes it turns out to be useful. 846 00:34:31,460 --> 00:34:33,670 People also love, humans, including infants, 847 00:34:33,670 --> 00:34:37,150 contingency, the idea that we feel like we have 848 00:34:37,150 --> 00:34:38,909 control over our world. 849 00:34:38,909 --> 00:34:42,389 So here's a two-month-old infant, very young infant, in 850 00:34:42,389 --> 00:34:44,400 a little experiment where they put the infant in a crib. 851 00:34:44,400 --> 00:34:47,500 And above is a very colorful colored mobile. 852 00:34:47,500 --> 00:34:51,620 And in one version, when the infant moves his or her head, 853 00:34:51,620 --> 00:34:53,650 there's a switch in the pillow. 854 00:34:53,650 --> 00:34:54,810 The mobile moves. 855 00:34:54,810 --> 00:34:56,429 And the infants are smiling and cooing. 856 00:34:56,429 --> 00:34:56,889 They love it. 857 00:34:56,889 --> 00:34:57,570 This is awesome. 858 00:34:57,570 --> 00:34:58,190 They move their head. 859 00:34:58,190 --> 00:35:00,020 The thing moves, pretty impressive for a two-month. 860 00:35:00,020 --> 00:35:01,630 All they can do is wiggle their head, right? 861 00:35:01,630 --> 00:35:04,130 They're not walking or anything. 862 00:35:04,130 --> 00:35:08,740 They put another group of infants in who have no switch. 863 00:35:08,740 --> 00:35:12,940 And just periodically, they turn on the mobile. 864 00:35:12,940 --> 00:35:14,570 There's no smiling and no cooing. 865 00:35:14,570 --> 00:35:17,190 And they equate the number of mobile turnings. 866 00:35:17,190 --> 00:35:18,250 Why is that? 867 00:35:18,250 --> 00:35:20,730 Well, if it's just because you like the mobile turning, then 868 00:35:20,730 --> 00:35:23,050 you should have both groups be equaling cooing and smiling or 869 00:35:23,050 --> 00:35:23,760 disinterested. 870 00:35:23,760 --> 00:35:26,950 But there's something fundamentally rewarding for 871 00:35:26,950 --> 00:35:29,280 humans, even at the age of two, to feel like they have 872 00:35:29,280 --> 00:35:32,270 control over their environment and their pleasures. 873 00:35:32,270 --> 00:35:34,160 And that's not explained by conditioning. 874 00:35:34,160 --> 00:35:35,130 Nobody's getting anything more. 875 00:35:35,130 --> 00:35:37,690 They're both seeing the equal number of interesting turns. 876 00:35:37,690 --> 00:35:39,590 But one group feels like they're controlling their 877 00:35:39,590 --> 00:35:41,050 environment. 878 00:35:41,050 --> 00:35:43,580 Here's another one that I'm going to show you in an animal 879 00:35:43,580 --> 00:35:45,890 experiment where you can control things. 880 00:35:45,890 --> 00:35:48,020 But you can imagine in your own life. 881 00:35:48,020 --> 00:35:51,260 In my life, I see this all the time. 882 00:35:51,260 --> 00:35:54,500 It's nice when something pleasant happens in your life 883 00:35:54,500 --> 00:35:57,590 and your situation gets better in some way, right? 884 00:35:57,590 --> 00:35:59,220 You get a better iPad, right? 885 00:36:01,840 --> 00:36:04,970 But how hard is it emotionally when something gets worse? 886 00:36:04,970 --> 00:36:11,460 They take all your iPads away or more serious things. 887 00:36:11,460 --> 00:36:15,060 When things get worse, often we feel pretty miserable. 888 00:36:15,060 --> 00:36:16,560 And here's a neat example of that. 889 00:36:16,560 --> 00:36:19,400 So these are rats in an experiment so everything's 890 00:36:19,400 --> 00:36:20,250 controlled. 891 00:36:20,250 --> 00:36:21,840 So here's one group of rats. 892 00:36:21,840 --> 00:36:23,220 This is how quickly they're running. 893 00:36:23,220 --> 00:36:25,150 So it's good to be high. 894 00:36:25,150 --> 00:36:27,060 One group of rats is constantly getting 895 00:36:27,060 --> 00:36:28,290 one pellet a day. 896 00:36:28,290 --> 00:36:29,280 So they learn what to do. 897 00:36:29,280 --> 00:36:31,650 And they're sort of hanging out here doing well. 898 00:36:31,650 --> 00:36:34,500 A second group of rats is the luckiest group of all. 899 00:36:34,500 --> 00:36:35,430 They get eight pellets. 900 00:36:35,430 --> 00:36:36,940 And they learn more. 901 00:36:36,940 --> 00:36:38,370 But they flatten out. 902 00:36:38,370 --> 00:36:39,800 So you get more reward. 903 00:36:39,800 --> 00:36:41,480 You drive the behavior more. 904 00:36:41,480 --> 00:36:42,770 That's not unexpected. 905 00:36:42,770 --> 00:36:45,930 The interesting thing are the two contrast groups. 906 00:36:45,930 --> 00:36:48,810 The contrast groups start off with one or eight. 907 00:36:48,810 --> 00:36:51,030 And then, after a couple days, they're flipped around. 908 00:36:51,030 --> 00:36:52,910 So some started with one. 909 00:36:52,910 --> 00:36:54,370 Then they got eight pellets. 910 00:36:54,370 --> 00:36:55,930 Others started with eight. 911 00:36:55,930 --> 00:36:56,680 Then they got one. 912 00:36:56,680 --> 00:36:57,920 Does that make sense? 913 00:36:57,920 --> 00:37:01,110 So here's the contrast group of eight. 914 00:37:01,110 --> 00:37:02,026 They're learning one. 915 00:37:02,026 --> 00:37:05,230 They go eight, woo. 916 00:37:05,230 --> 00:37:06,360 They're happy. 917 00:37:06,360 --> 00:37:07,230 They're getting eight. 918 00:37:07,230 --> 00:37:08,670 Life got good. 919 00:37:08,670 --> 00:37:11,000 Look at the group for whom life got worse. 920 00:37:11,000 --> 00:37:11,990 I'm getting eight. 921 00:37:11,990 --> 00:37:12,890 I'm getting eight. 922 00:37:12,890 --> 00:37:13,720 I'm getting eight. 923 00:37:13,720 --> 00:37:14,360 I'm getting one. 924 00:37:14,360 --> 00:37:15,060 I'm getting one. 925 00:37:15,060 --> 00:37:15,700 I'm getting one. 926 00:37:15,700 --> 00:37:17,360 It really stinks. 927 00:37:17,360 --> 00:37:20,350 They're actually worse than the groups that got one all 928 00:37:20,350 --> 00:37:21,920 along, right? 929 00:37:21,920 --> 00:37:23,560 This is the lowest line of all. 930 00:37:23,560 --> 00:37:24,770 They're protesting. 931 00:37:24,770 --> 00:37:26,420 This stinks. 932 00:37:26,420 --> 00:37:27,140 I was getting eight. 933 00:37:27,140 --> 00:37:27,890 Now I'm getting one. 934 00:37:27,890 --> 00:37:29,120 What a rip off. 935 00:37:29,120 --> 00:37:31,130 What kind of world do I live in? 936 00:37:31,130 --> 00:37:33,080 So the learning is protesting that. 937 00:37:33,080 --> 00:37:35,850 Now, by purely conditioning one, why would 938 00:37:35,850 --> 00:37:36,920 this group be worse? 939 00:37:36,920 --> 00:37:39,340 They're still getting a pellet. 940 00:37:39,340 --> 00:37:41,610 But because they're interpreting it as a world 941 00:37:41,610 --> 00:37:45,570 that's going downhill, as a worsening situation, they're 942 00:37:45,570 --> 00:37:46,980 less prone to learn it. 943 00:37:46,980 --> 00:37:49,520 They're sort of protesting in their learning, things having 944 00:37:49,520 --> 00:37:50,950 gotten worse in the world. 945 00:37:54,820 --> 00:37:57,860 A really interesting theme along the line of this reward, 946 00:37:57,860 --> 00:37:59,810 an unbelievable one that has gotten a lot of attention, is 947 00:37:59,810 --> 00:38:01,500 delayed gratification. 948 00:38:01,500 --> 00:38:03,380 If you think about it, and the analysis is not purely 949 00:38:03,380 --> 00:38:06,980 scientific but is a theme, delayed gratification is a 950 00:38:06,980 --> 00:38:11,550 spectacular part of your life, right, especially if you go 951 00:38:11,550 --> 00:38:12,630 through higher education. 952 00:38:12,630 --> 00:38:14,700 Or maybe everything is like this, right? 953 00:38:14,700 --> 00:38:16,760 Do a good job in primary thing and you'll get to a good 954 00:38:16,760 --> 00:38:17,180 middle school. 955 00:38:17,180 --> 00:38:18,740 Do a good job in middle school, we'll get to a good 956 00:38:18,740 --> 00:38:19,170 high school. 957 00:38:19,170 --> 00:38:20,710 Do a good job in high school, you get to MIT. 958 00:38:20,710 --> 00:38:23,990 Do a good job at MIT, you go to med school or law school or 959 00:38:23,990 --> 00:38:25,120 become a teacher, whatever you want to do. 960 00:38:25,120 --> 00:38:27,240 A good job at that and you'll get to this thing. 961 00:38:27,240 --> 00:38:29,100 Do a good job of that, finally, thank you very 962 00:38:29,100 --> 00:38:29,890 much, you're 95. 963 00:38:29,890 --> 00:38:31,140 And you're done. 964 00:38:32,960 --> 00:38:37,640 I mean it's kind of like you guys are waiting a long time 965 00:38:37,640 --> 00:38:39,640 between the efforts you're doing and what you might 966 00:38:39,640 --> 00:38:43,410 consider a palpable reward. 967 00:38:43,410 --> 00:38:45,950 So here's a fantastically interesting study from Walter 968 00:38:45,950 --> 00:38:48,310 Mischel done at Stanford some years ago. 969 00:38:48,310 --> 00:38:51,160 These children were four to five years old. 970 00:38:51,160 --> 00:38:52,490 There were about 600 that were studied. 971 00:38:52,490 --> 00:38:55,110 They were mostly the children of faculty and graduate 972 00:38:55,110 --> 00:38:57,680 students at Stanford. 973 00:38:57,680 --> 00:38:59,430 And they would come into a room. 974 00:38:59,430 --> 00:39:00,740 And the experiment, slightly varied, was 975 00:39:00,740 --> 00:39:01,490 basically like this. 976 00:39:01,490 --> 00:39:03,530 They would sit in front of either a cookie or 977 00:39:03,530 --> 00:39:04,080 marshmallow. 978 00:39:04,080 --> 00:39:05,400 They used different rewards. 979 00:39:05,400 --> 00:39:07,910 But imagine it's a marshmallow. 980 00:39:07,910 --> 00:39:09,930 In the movie I'll show you, it's Oreo cookies. 981 00:39:09,930 --> 00:39:12,890 And they were told this. 982 00:39:12,890 --> 00:39:16,170 And they tried to make this, Walter Mischel who did the 983 00:39:16,170 --> 00:39:18,390 experiment describes how he played with kids to try to get 984 00:39:18,390 --> 00:39:22,010 the marshmallow look really awesome. 985 00:39:22,010 --> 00:39:24,620 And the kid would sit there, the four or five-year-old. 986 00:39:24,620 --> 00:39:26,670 And they were told, I'm going to leave the room now. 987 00:39:26,670 --> 00:39:27,860 And there's a bell here. 988 00:39:27,860 --> 00:39:30,970 If you really want to eat that marshmallow, ring the bell and 989 00:39:30,970 --> 00:39:32,410 just eat it. 990 00:39:32,410 --> 00:39:34,200 But if you wait a while-- 991 00:39:34,200 --> 00:39:35,800 they didn't tell them it was 15 minutes. 992 00:39:35,800 --> 00:39:36,470 It was 15 minutes. 993 00:39:36,470 --> 00:39:37,050 I'll come back. 994 00:39:37,050 --> 00:39:38,410 And I'll give you two. 995 00:39:38,410 --> 00:39:41,680 Delayed gratification, I can have one marshmallow now. 996 00:39:41,680 --> 00:39:44,750 Or if I hang in there for 15 minutes and don't touch it, I 997 00:39:44,750 --> 00:39:47,070 can get two. 998 00:39:47,070 --> 00:39:49,380 Let me tell you the results. 999 00:39:49,380 --> 00:39:51,230 And then let me show a film of this. 1000 00:39:51,230 --> 00:39:52,960 When I was looking on YouTube and the internet and stuff, 1001 00:39:52,960 --> 00:39:55,540 there's a lot of posed movies out there. 1002 00:39:55,540 --> 00:39:58,630 This is the only one I found from Walter Mischel himself 1003 00:39:58,630 --> 00:40:00,970 that's a real movie of real kids really doing this. 1004 00:40:00,970 --> 00:40:03,700 I don't know why all the ones are posed out there. 1005 00:40:03,700 --> 00:40:06,340 But here's something astounding. 1006 00:40:06,340 --> 00:40:07,640 So these are all pretty similar 1007 00:40:07,640 --> 00:40:10,290 children, children of faculty. 1008 00:40:10,290 --> 00:40:12,550 Children varied how long they could wait. 1009 00:40:12,550 --> 00:40:14,180 And if they took, for example, the children who 1010 00:40:14,180 --> 00:40:15,290 didn't wait very long. 1011 00:40:15,290 --> 00:40:16,690 They could only wait 30 seconds until 1012 00:40:16,690 --> 00:40:17,910 they ate the cookie. 1013 00:40:17,910 --> 00:40:19,680 And they compared them to the children who would wait for 1014 00:40:19,680 --> 00:40:22,470 the full 15 minutes to eat the cookie. 1015 00:40:22,470 --> 00:40:23,470 These are four and five-year-olds. 1016 00:40:23,470 --> 00:40:27,230 When they looked at them at 18 and their SATs, the children 1017 00:40:27,230 --> 00:40:30,800 who would wait 15 minutes scored over 200 points higher, 1018 00:40:30,800 --> 00:40:32,210 which you know is a lot. 1019 00:40:32,210 --> 00:40:34,000 That was in the days we just had two SAT tests. 1020 00:40:34,000 --> 00:40:37,550 So it was 1,600. 1021 00:40:37,550 --> 00:40:41,400 And by many, many measures in terms of educational 1022 00:40:41,400 --> 00:40:45,320 attainment, in terms of health, and many measures, the 1023 00:40:45,320 --> 00:40:48,890 longer you waited at age four or five, the healthier you 1024 00:40:48,890 --> 00:40:51,220 were, the better SAT scores you were, the further you went 1025 00:40:51,220 --> 00:40:53,790 in school, astounding correlations 1026 00:40:53,790 --> 00:40:56,380 with real life stuff. 1027 00:40:56,380 --> 00:40:58,630 And people are stunned by this. 1028 00:40:58,630 --> 00:41:00,700 Partly, it's again, who knows what goes into 1029 00:41:00,700 --> 00:41:01,540 that at four or five. 1030 00:41:01,540 --> 00:41:02,960 How much is the influence of parents? 1031 00:41:02,960 --> 00:41:05,330 How much is the influence of cultures and genes? 1032 00:41:05,330 --> 00:41:06,170 Who knows? 1033 00:41:06,170 --> 00:41:09,010 But that fateful moment when the child goes for the 1034 00:41:09,010 --> 00:41:12,410 marshmallow or not correlates with incredibly important 1035 00:41:12,410 --> 00:41:14,370 outcomes in the future. 1036 00:41:14,370 --> 00:41:16,050 So let's see. 1037 00:41:16,050 --> 00:41:17,300 Here we go. 1038 00:41:24,070 --> 00:41:28,220 And you know, right, every college I remember personally, 1039 00:41:28,220 --> 00:41:30,325 it was always a battle, like am I going 1040 00:41:30,325 --> 00:41:31,200 to have fun or study? 1041 00:41:31,200 --> 00:41:32,220 Am I going to have fun or study? 1042 00:41:32,220 --> 00:41:33,600 There's so many chances to have fun. 1043 00:41:33,600 --> 00:41:36,100 And there's so many pressures to study, right? 1044 00:41:36,100 --> 00:41:39,480 Well, studying can be fun. 1045 00:41:39,480 --> 00:41:41,030 Learning is fun. 1046 00:41:41,030 --> 00:41:42,220 How about when rewards harm? 1047 00:41:42,220 --> 00:41:44,290 Again, these are things that are not strictly predicted by 1048 00:41:44,290 --> 00:41:45,160 conditioning. 1049 00:41:45,160 --> 00:41:46,660 I have three or four more minutes. 1050 00:41:46,660 --> 00:41:50,100 So rats love to run, if you had a hamster or anything like 1051 00:41:50,100 --> 00:41:51,540 that, they love running wheels. 1052 00:41:51,540 --> 00:41:56,030 What if you give them, every time they run, a food reward? 1053 00:41:56,030 --> 00:41:58,540 Then it turns out rats no longer run for fun. 1054 00:41:58,540 --> 00:42:01,870 They only run when the food is given. 1055 00:42:01,870 --> 00:42:04,350 If you start to extinguish, they stop running. 1056 00:42:04,350 --> 00:42:07,270 So what had been a natural thing that seemed like a whole 1057 00:42:07,270 --> 00:42:11,180 lot of fun, if it became the price you pay for a reward, 1058 00:42:11,180 --> 00:42:12,490 all a sudden becomes a chore. 1059 00:42:15,780 --> 00:42:20,870 A widely cited experiment for Mark Lepper, he took 1060 00:42:20,870 --> 00:42:24,220 preschoolers, so four-year-old children, who love to draw. 1061 00:42:24,220 --> 00:42:27,220 Lots of preschoolers love to draw. 1062 00:42:27,220 --> 00:42:28,760 Then they started to give them, every time they 1063 00:42:28,760 --> 00:42:31,300 drew, a gold star. 1064 00:42:31,300 --> 00:42:32,950 Every time they got a reward. 1065 00:42:32,950 --> 00:42:35,570 What happened when they stopped given the gold stars? 1066 00:42:35,570 --> 00:42:37,470 The kids stopped drawing. 1067 00:42:37,470 --> 00:42:40,350 This is incredibly often cited in incredibly interesting 1068 00:42:40,350 --> 00:42:44,910 debates about whether children in impoverished schools should 1069 00:42:44,910 --> 00:42:48,710 be given cash rewards for studying hard. 1070 00:42:48,710 --> 00:42:50,630 There's lots of debates about whether that's a good thing or 1071 00:42:50,630 --> 00:42:52,420 not a good thing, an effective thing or not 1072 00:42:52,420 --> 00:42:53,560 an effective thing. 1073 00:42:53,560 --> 00:42:56,640 This study is often cited for arguing, if you make something 1074 00:42:56,640 --> 00:42:59,330 that you should love, like learning, purely rewarded, the 1075 00:42:59,330 --> 00:43:04,250 minute the reward stops, it becomes a complete chore. 1076 00:43:04,250 --> 00:43:06,750 But we don't know how that applies. 1077 00:43:06,750 --> 00:43:09,710 Because we might read about things like 1078 00:43:09,710 --> 00:43:10,970 baseball player salaries. 1079 00:43:10,970 --> 00:43:12,810 And it seems like there's a pretty good relationship 1080 00:43:12,810 --> 00:43:14,860 between performance and salary. 1081 00:43:14,860 --> 00:43:18,490 So it's not simply that way in the world. 1082 00:43:18,490 --> 00:43:21,240 We know that often rewards are associated with high 1083 00:43:21,240 --> 00:43:23,000 performance and promote high performance. 1084 00:43:23,000 --> 00:43:24,810 It's pretty complicated. 1085 00:43:24,810 --> 00:43:29,050 The last slide is about a fantastic debate that occurred 1086 00:43:29,050 --> 00:43:32,140 here in many ways between B.F. Skinner at Harvard and Noam 1087 00:43:32,140 --> 00:43:36,970 Chomsky at MIT, which is, in what sense is language learned 1088 00:43:36,970 --> 00:43:39,060 by conditioning? 1089 00:43:39,060 --> 00:43:41,290 So here's an example against it. 1090 00:43:41,290 --> 00:43:45,330 At one month, there's a switch inside a rubber nipple hooked 1091 00:43:45,330 --> 00:43:46,690 to a tape recorder. 1092 00:43:46,690 --> 00:43:48,410 And when the baby sucked, the tape plays. 1093 00:43:48,410 --> 00:43:51,980 This allows the baby to perform, to behave. 1094 00:43:51,980 --> 00:43:55,480 And they would play syllables from different languages. 1095 00:43:55,480 --> 00:43:59,950 And they may or may not be in their own languages. 1096 00:43:59,950 --> 00:44:04,680 And by four days, babies preferred the languages that 1097 00:44:04,680 --> 00:44:07,670 they are exposed to. 1098 00:44:07,670 --> 00:44:11,970 And one of the questions is, people said, well, how do we 1099 00:44:11,970 --> 00:44:13,980 get kids to learn language from a behaviorist 1100 00:44:13,980 --> 00:44:14,580 perspective? 1101 00:44:14,580 --> 00:44:18,700 So one of the things that they studied children, and they'll 1102 00:44:18,700 --> 00:44:21,480 often, young children will say wrong things like mama isn't a 1103 00:44:21,480 --> 00:44:22,290 boy, he a girl. 1104 00:44:22,290 --> 00:44:24,140 That's not good grammar, right? 1105 00:44:24,140 --> 00:44:26,500 And so there was an intuition that parents would then go, 1106 00:44:26,500 --> 00:44:27,340 that's not good grammar. 1107 00:44:27,340 --> 00:44:28,700 And sometimes parents do that. 1108 00:44:28,700 --> 00:44:31,020 But when they did studies where they recorded long 1109 00:44:31,020 --> 00:44:33,260 sessions with parents doing this, parents 1110 00:44:33,260 --> 00:44:35,690 almost never corrected. 1111 00:44:35,690 --> 00:44:37,540 It was very rare. 1112 00:44:37,540 --> 00:44:39,220 You may feel different from your own record. 1113 00:44:39,220 --> 00:44:43,100 But statistically, it's very rare to collect. 1114 00:44:43,100 --> 00:44:45,680 They often say that's right or something like that. 1115 00:44:45,680 --> 00:44:48,920 So how's the child picking up language if nobody's giving 1116 00:44:48,920 --> 00:44:51,580 them rewards in terms of whether 1117 00:44:51,580 --> 00:44:52,970 they're right or wrong? 1118 00:44:52,970 --> 00:44:55,540 The idea is that we can say an infinite number of sentences. 1119 00:44:55,540 --> 00:44:59,580 Well, how can we possibly be rewarded or conditioned for an 1120 00:44:59,580 --> 00:45:00,820 infinite variety of sentences? 1121 00:45:00,820 --> 00:45:02,070 How could that work? 1122 00:45:04,360 --> 00:45:07,240 And finally, the phenomenon of overgeneralizing, you may know 1123 00:45:07,240 --> 00:45:10,680 that children will overgeneralize things. 1124 00:45:10,680 --> 00:45:14,700 Because we usually end E-D to make a verb past tense, right? 1125 00:45:18,100 --> 00:45:20,360 But some verbs don't work that way. 1126 00:45:20,360 --> 00:45:22,110 So you would say, my teacher held the rabbit. 1127 00:45:22,110 --> 00:45:24,300 Children often make these overgeneralized rules. 1128 00:45:24,300 --> 00:45:26,560 They say, my teacher holded the rabbit. 1129 00:45:26,560 --> 00:45:29,740 So obviously, they're learning a rule and then applying it to 1130 00:45:29,740 --> 00:45:32,040 an instance that they never hear. 1131 00:45:32,040 --> 00:45:34,570 Because almost no adult says that. 1132 00:45:34,570 --> 00:45:36,520 So they must be producing that, not on the basis of 1133 00:45:36,520 --> 00:45:40,100 conditioning, but Chomsky argued on the basis of sort of 1134 00:45:40,100 --> 00:45:45,670 genetically prepotent ways in which we learn language, rules 1135 00:45:45,670 --> 00:45:47,660 that we develop and figure out language in the world. 1136 00:45:47,660 --> 00:45:50,500 I will say that, for those of you happen to be interested in 1137 00:45:50,500 --> 00:45:54,690 this topic, there's been a striking pushback on that in 1138 00:45:54,690 --> 00:45:57,460 what people call statistical learning. 1139 00:45:57,460 --> 00:46:02,770 So as much as this seemed like a done deal 10 years ago, it 1140 00:46:02,770 --> 00:46:05,290 turns out the truth might be somewhere in the middle with 1141 00:46:05,290 --> 00:46:08,480 some genetic prepotentiziation but also a lot of learning 1142 00:46:08,480 --> 00:46:12,430 from your environment both. 1143 00:46:12,430 --> 00:46:15,940 So you can think, as you leave here, what are the applause 1144 00:46:15,940 --> 00:46:17,150 that you listen to? 1145 00:46:17,150 --> 00:46:19,880 Where's your free will and your choice of what goals you 1146 00:46:19,880 --> 00:46:21,690 pursue and what matters to you? 1147 00:46:21,690 --> 00:46:25,370 And how does the world attempt to teach you things that you 1148 00:46:25,370 --> 00:46:27,050 learn about what you want to do and why 1149 00:46:27,050 --> 00:46:28,300 you want to do them?