Lec 13: Review: The visual and oculomotor systems

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: All right so today, we're going to have a review of the visual and oculomotor systems that I've covered so far. And what I'm going to do is I'm going to go over many, many basic facts in a rather quick fashion, which will sort of refresh your memory of what we have covered so far and also will make you more aware of what you want to look at carefully when you look at the material on the website, on Stellar, and also when you read the assigned readings.

I want to remind all of you again that you will have to put together that paper on the accessory optic system that I will mention very briefly at the end of the review. And your prime task there will be-- that's an old paper published in the 1960s, which was a major discovery at the time-- and your task will be predominantly to say what-- well, first of all what has been discovered there, that you can cover in a paragraph and then to add to it what people have contributed to the study of that area since that original paper.

All right, so anyway then I will mention at the end or so a bit more about the exam, which is going to take place on Wednesday right in here, which is going to consist of multiple choice questions. All right, so to begin with then, let's talk about the basic wiring of the visual system that we have covered.

And that is outlined here for the primate and for the human. And I should mention as I had gone in the initial lecture that this is different from many of the lower level species in which the two eyes look sideways, and each eye sends all of its retinal ganglion cell axons across to the other hemisphere in the brain.

Now this big change occurred when the eyes move to the front. And we have discussed already why that may have happened. And as a result of this, if you imagine cutting your vertically in half, you divide it into the nasal and temporal hemi retinae. And it so happens that the nasal hemi retina of one eye and the temporal retina of the other eye goes to one side, and the obvious happens to the other side.

The connections are made to several areas, most notably for our purposes, was the lateral geniculate nucleus but also the superior colliculus and several other structures that we have talked about that include the accessory optic system. Once the connections come up to the cortex, several cortical areas-- we'll talk about that in a minute-- have evolved that are involved in progressively higher levels of visual analysis.

Now this circle here-- hopefully you guys remember-- is either called the Vieth-Muller circle, or it's called the horopter. And it was shown by a clever experimentalist that if you put any spot along that circle when the person fixes at this point in the circle, all of those points impinge on corresponding points in the two retinae.

However, if there's an object that seemed to be beyond or closer to the eye than the circle, then they hit non equivalent points in the retina. And that non equivalency is actually used for depth perception, as we have discussed, and I will mention again when it comes to a stereopsis.

So that is the very, very basic wiring arrangement. And then if you proceed from here and look at the retina and the lateral geniculate nucleus in a bit more detail, first starting with the retina. I wanted to point out to you first of all that there are two different kinds of photoreceptors. You all know this very well by now. You knew this before you came to class.

You have the rods and the cones. There are three basic types of cones, red, green, and blue, which more appropriately refer to short, medium, and long wavelength selective cones. And then we have the rods. Now what happens is that the light comes in, in this case from the bottom if you look at yourself. The light comes in, and it goes through many of the cells in the retina. And it impinges on the receptors, which are facing away from the light against the pigment epithelium.

And as I've mentioned to you, there have been some interesting questions as to why this strange arrangement had emerged. Nobody had predicted that before we had any anatomy, people just thought if there were any receptors, they would face the light. So that's certainly a strange arrangement, an unusual one. And a lot of speculation have been advanced as to why this has happened.

And I will just briefly mention two of those. One is that when these photoreceptors are right against the pigment epithelium, which in diurnal animals is black, and absorbs the photons thereby preventing scattering of light. As a result of which, you can gain high acuity. And that is known by the fact that if you assess the visual capacities of albinos who don't have a black pigment epithelium because they lack pigment-- that's the definition of being an albino-- most people have very poor vision because the photons come into the eye, and they scatter all over the place and activate many photoreceptors rather than just those which the incoming photon would hit directly.

So that is the arrangement for these. Another factor, which I don't think I may not have mentioned is that what you have in these photoreceptors are little packets, if you talk about the rods in particular, you have little packets, each of which has the molecules, which are sensitive to the incoming light. And each of these packets-- there are about 1,000 packets in each of these rods. And each of those 1,000 packets is about 10,000 molecules.

So we're talking about gigantic numbers. Now what happens is that given this 1,000 of-- about 1,000 packets, they are not there for your life. What happens is that the packets gradually disintegrate and get replaced by a new one. It's about once every 10 days you lose a packet, and you gain a new one.

Now that means that some of this stuff gets sloughed off. And one of the reasons people think that the photoreceptors ended up facing away from the light is that they could be close to this inner part of the retina where anything that's sloughed off can be absorbed rather than being just thrown into the eye itself, into the vitreous, because if that were to happen over many, many years, the vitreous would become cloudy, and you couldn't see.

So those are two possible reasons why this strange arrangement has evolved. And you see this in virtually all species. There are just a few species that have-- and most of those are actually in the sea-- who have photoreceptors that face towards the light.

All right, and then if you proceed here, the other amazing thing that had been discovered is that all the photoreceptors hyper polarized to light. Again, they do the opposite of what people had thought.

You'd think that when photons come in, they would activate the photoreceptors, and they would send the signal down the stream through the eye. Turns out the opposite happens that the discharge on the neurotransmitter here occurs when there's a darkening rather than an increase in light. That's an important factor to remember.

That's true for all the forests accept as all photoreceptors hyperpolarized to light. You know this well already. I must have said that about 10 times by now. Now the amazing thing is that when you come to the bipolar cells, the next set of receptors here, it was discovered that two basic types of several different types like from for the major parasol cells, but there are two basic types, the on and the off.

And this is accomplished by having two kinds of synapses in the on and off bipolar cells, sign conserving ones and sign inverting ones. This is accomplished in the on bipolars by virtue of the [INAUDIBLE] six receptor site and the [INAUDIBLE] one and two for the off bipolar cells.

So that means now that you have signals in some of these cells when there's an increase in light and the signals in some where there's a decrease in light. So that's the situation for the on and off bipolars. And then when you come into the level of the ganglion cells, two major classes of ganglion cells are the on and the off.

Now I'll talk about that in more detail in just a minute. Now the other interesting, curious thing is that when you look at the rods, the rods-- and they connect to their bipolar cells. They are all sign inverting synapses. They only come in one type, at least in humans and in primates.

So what happens then to create on and off section done in the inner retina by virtue of having a synapse here [INAUDIBLE] to amacrine cell, which is a glycinergic synapse, and it also makes connections to the on bipolar, which is a gap junction. And this way, it becomes a double ended system for the rods as well as for the cones.

So hopefully you guys all remember this. I know it's complicated. But that is something that one doesn't have a choice about. That's how it simply is.

All right, so now we move on. And we are going to look at the lateral geniculate nucleus. Here's a cross section of it. I've shown you before.

It was discovered that the lateral geniculate nucleus in central retina-- this is a monkey retina. The human is very similar, so input from a monkey retina to the lateral geniculate nucleus. And the six layers consist of two major types, the parvocellular so called the magnocellular layers. And what was discovered is that the parvocellular layers get input from the midget cells that we'll talk about in just a minute. And the bottom two layers, which are the magnocellular layers get input from the parasol cells.

And then what happens is that when you go from central vision to peripheral vision, you have a huge change in the productive percentage of midget and parasol cells that you have in the retina and in the lateral geniculate nucleus near the fovea. In the foveola itself, you don't have any parasol cells, but in the fovea itself you do. And there's a ratio of about eight to one. And then as you go to the periphery, eventually they're equally in number.

So there's a huge emphasis on the midget system in central vision and the much increased emphasis on the parasol cells in peripheral vision. So that's the arrangement. And this is reflected in the geniculate, which has six layers in central vision after about 18 degrees. And it has four layers in the periphery where the midget and parasol inputs are pretty much equal as reflected by these four layers. So that's the basic arrangements for the lateral geniculate nucleus.

Now if we move on-- let me say one more thing. The receptive field properties of cells in the retina, in retinal ganglion cells, I should say, and in the lateral geniculate nucleus are highly similar. They're virtually identical. You have circular receptive fields with centers around antagonism.

All right, now if you move on and move up to the visual cortex, what happens is that there's a huge change that arises, the beautiful discoveries made by Hubel and Wiesel for which they had received the Nobel Prize. And this is just a quick view of the monkey brain. Here is area V1. I'll come back to the other areas in a minute.

The nice thing about this in the monkey is that this area is [INAUDIBLE] as I had told you. And because of that, it's easy to study the cells and their properties in area V1. All right, so now if one examines the properties of single cells in area V1, it was discovered some major transformations had occurred in the inputs from the lateral geniculate nucleus. And these major transforms can be summarized in just a second.

But I will first tell you that there is a differential input from the parvocellular and magnocellular layers, which project respectively to [? 4C ?] beta and [? 4C ?] alpha. And then there's yet another class of cells that originates in the retina that are project into the inter lamina layers, and they project into the upper portions of the visual cortex.

So now if one looks in detail at the properties of these cells, which we have discussed quite a bit. You can refer to these as transforms. The transforms of the visual input into the cortical cells. So when you're record from these cortical cells, you'll find one big transform is that these cells, the overwhelming majority of these cells, become orientation selective.

Many cells become direction selective, virtually all simple cells and about half the complex cells. So direction selectively becomes very important. We'll talk about that in a bit more detail later on. Then, some cells are spatial frequency selective. Many cells get an input from both eyes. And there's a convergence of input from the on and off channels. This is also true for some of the cells that get a convergent input from the midget and parasol cells. So those are the major transforms that you see in the visual cortex.

All right, so now as a result of having made these discoveries, people came up with a question of how is this organized in the visual cortex. And the first point that I had made is that there's a topographic layout of the visual field in visual cortex. But with much more area allocated for central vision than peripheral vision simply copying the relative percentage of cells already in the retina that exist in central vision and peripheral vision and because the thickness of the gray matter in cortex is about two millimeters roughly. And it's constant. More space has to be allocated for central vision than peripheral vision.

And as a result of these people that studied the spacial arrangement and organization of the visual cortex-- and the initial model that was proposed, if you remember, is the Hubel and Wiesel model, according to which in one direction you have the alternation of left and right eyes. You have column, left, right, left, right. And in the other direction, you have a systematic change in the orientation of cells.

Now this model didn't fare that well because it's not as neat as has been proposed. An alternative model was the Raidial model. And the last one I'm showing here, which I call the Swirl model is not really a model because some very clever experiments that have been carried out by [INAUDIBLE] actually did optical recording and demonstrated that the visual cortex from the top looks something like this where you have indeed systematic arrangement of orientations in left and right eye columns. But it's not a straight linear factor, but it's kind of a swirly arrangement. So that then established what is the layout of the primary visual cortex.

OK now the other important thing that we had emphasized is that contrary to some of the popular ideas that people have had that the cells in the brain are feature selective, meaning that they'll extract specific features from the visual scene like say one cell extracts color, another cell extracts a particular face, and so on. It turns out that that's a false impression that people had gained.

And instead what is happening that any given one cell processes many different kinds of visual information. And it's the activity of thousands and thousands of cells in a network that can come up with the percepts that you perceive. Now that's extremely complicated, 10 times more complicated than any computer. And it is something that to a larger extent still has not been solved.

You don't know how does a person recognize a face. You can tell oh, it takes place in various parts of the brain and so on. But exactly physically how that's done is something that's still remains largely a mystery.

All right, so now let's move on and talk about extrastriate cortex. And extrastriate cortex-- here's a diagram of the monkey brain again. Now I'll point out here's area V1. And once you get close to the lunate sulcus here, V2 begins and folds under. And then inside there we have V3. And then actually make folds back out again. You have area V4 here.

And then you have areas MT and MST right here. And then, in addition, you have, of course, your infertemporal cortex area, which plays a very important role in complex analyses such as faces. And then you come to the frontal lobe here, in which you have the frontal eye fields and medial eye fields that also process visual information but mostly for eye movements that I will talk about later on.

So that then is in a nutshell the arrangement. And much of the work that has been done in the past dozen years or so was to examine what these extrastriate areas do for vision. And I'll come back to that when we talk about higher level visual processing.

Now basically the fact is that there are more than 30 visual areas and that there are more than 300 interconnections among them. Initially the idea was-- the feature detection idea-- that each of these areas is specific for analyzing a particular type of percept. But then it became more evident, increasingly more evident, that these areas tremendously interact with each other and perform these very complex analyses based on networks being active.

Now the basic major cortical visual areas, V1 I just talked about, V2 I mentioned, V3, V4, MT. Then when you come to the temporal cortex, you come to infertemporal region that I just mentioned. And then in the parietal cortex, we have the lateral parietal sulcrus, the ventral interparietal, and the medial superior temporal sulcrus.

So those are some of the major areas. And then as I've already noted, in the frontal cortex, we have the frontal eye fields. And then even we had the medial eye fields, which are not listed here that also play a role in eye movements, perhaps a lesser extent in visual analysis as such. But many of the cells there too have visual receptive fields, although they are very hard to discern. They much more clearly have motor fields than visual field.

All right so now what we are going to do is we are going to go back to the beginning and look at the so-called on and off channels briefly. We talked about that a lot. Again, to reemphasize, all photoreceptors hyperpolarize to light. And then because the two major classes of neurotransmitter receptor sites in the bipolar cells, you create a double ended system from a single ended one creating the so-called on and off.

Now these systems were discovered initially by Keffer Hartline, who received the Nobel Prize for that remarkable discovery. And he thought at the time that the on system signaled when a stimulus came on. And the off channel signaled when it went off. That was his idea, which turned out to be all wrong because that's not what these cells are about.

What these cells are about, as I've pointed out repeatedly, is that they can process both light increment and light decrement with an excitatory response. That means because of the nature, the physical nature, of light that some objects in the world reflect light, and some objects in the world absorb light.

Because of this, as you look around, some objects look black, and some objects look white, or whatever. And because of that, to be able to rapidly process something that is a dark object as well as a light object, you need to have excitatory signals to go to the central nervous system to process that. So therefore, we can say, first of all, that we have these cell types. And they won't have sensor surround antagonism.

And let me add one more fact here is that they're comfortable with adaptation, that the average firing rate, average maximum firing rate, of a retinal ganglion cell is maybe about 400 to 600 hertz. And that is a rather limited frequency range. And yet, you have to analyze practically over 10 log units of light information. And because of that, the sensor surround antagonism has evolved so that these cells always look at local contrast changes rather than absolutes.

So then, if you look at the on and off cells, I've told you, in accordance with the sensor surround antagonism, if you split a small spot of light in the center of receptive fields, on cells fire when you increase it. Off cells fire when you decrease it. But when you use a much larger spot, you get a lesser response because of the surround antagonism. So that's the basic principle of these two types of cells.

And then, I told you about these experiments, in which two 2-amino-4-phosphonobutyrate had been used, which is for brief purposes, called APB. And I told you about two types of experiments, one doing single cell recordings in various parts of the brain and the others to do behavioral studies. And what the signal cell recordings had shown is that the-- let me first say what APB does.

APB is what? Anybody remember? It's a neurotransmitter analog. And what neurotransmitter is it?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Very good. Glutamate.

All right so what you do is when you inject this substance into the eye-- this is an artificial substance-- it blocks the on cells from being able to respond to incoming light but does nothing to the off cells. So if you do this and study the responses of single neurons in various parts of the brain-- there have been all these different hypotheses as to why we had the on and off channels.

One of them was to create sensor surround antagonism. And the other one was to create orientation direction selectivism in the cortex. But it turned out that when you injected APB into the eye, and you blocked the on channel, the off input to the cells, and the off cells therefore, still had sensor surround antagonism. And the cells in the cortex still had orientation and direction selectivities. So these two systems did not arise for the purpose of creating those basic attributes, which are so central for being able to analyze the visual scene.

Now the second important finding was that when you did a behavioral study and asked monkeys to detect light increment and detect light decrement, there was a huge deficit in detecting light increments but no deficit in detecting light decrement. So these observations and many other studies analyzing why there are on and off channels came up with the conclusion, which I think is quite valid that these two systems have evolved to enable organisms to quickly respond to both light detrimental and light incremental input.

And you probably remember the little quick movie I showed you that you have a fish in the ocean. Fish also have on and off channels of course. If there's a bird in the sky, like an osprey that is seen by virtue of light decrement, your off system tells that fish, oh, there's a bird up there so it can escape. And if a predator from below that is lit up by the sunshine, the on system in response to that and enables this fish to escape. So that's one example of the function, the prime function, of the on and off channels.

All right so that's the basic fact then. So to conclude then, the on and off channels have emerged in the course of evolution to enable organisms to process both light incremental and light decremental information rapidly and effectively. So that's the conclusion then in a nutshell of the on and the off channels.

Now we can move on and look at the so-called midget and parasol cells that had been discovered initially in the cat. And they were called the x and y cells. In the monkey, it's called midget and parasol because when you look at them anatomically, the midget cells are small. And they're very small dendritic arbors where the parasols cells are much bigger and have much larger dendritic arbors that look like an umbrella.

So those two systems were discovered, and statistical analysis revealed that they are totally separate types of cells. They're not a continuum. So the question then became why did these two systems evolve? And why did nature go to such trouble as to make sure that they were separate in retina and separate in the geniculate to the monkey?

And then, in the cortex, sometimes it remains separate-- sometimes the two systems remain separate-- and sometimes they converge as I had noted in those transforms in area V1. So now if you look at that, you've seen this several times now.

The midget system, the center in central retina consists of just a single cone. And therefore, this system should be able to tell you about color whereas the parasol system has mixed inputs both in the center and the surround. Furthermore, the parasol system response much more trangently than the midget system. So temporal information can be processed more effectively by the parasol system than the midget system. So those are the initial observations at the single cell level.

And then behavioral studies were carried out in which either the midget or the parasol system were selectively blocked. And then performance was tested where those systems had been blocked and where the systems were intact. And when this was done, some major findings emerged.

Before I tell you about that, let me just reiterate again what these connections are. Here we have the midget and the parasol cells as well as the cornea cellular cells. They project through the geniculate up to the visual cortex.

And then from there, there has been lots of debate as to what is the nature of the connections to higher areas in the brain. And we talked about that quite a bit. And some beautiful studies had been carried out showing that the input to area MT in the parietal lobe is dominated by the parasol system, but the input to V4 in the temporal lobe is about equal for the two systems. So that was the basic factor then.

And so now the question then comes up, what is the contribution of these two systems, the midget and the parasol? And so experiments are carried out where lesions are made in either parvocellular or magnocellular geniculate. And then the monkey was tested, as I've said, in intact areas, in areas where the midget system and areas where the parasol system had been blocked.

Now one additional fact is that when you block both of these by lesion in the lateral geniculate nucleus, for the most part, the monkey becomes blind. OK, so these two systems are really central for being able to process visual information.

All right so now, if one looks at what kinds of deficits arise, a monkey can be trained in a whole bunch of different tasks. It talked about these. Color vision, texture perception, pattern perception, shape perception, brightness, [INAUDIBLE] scotopic vision, contrast sensitivity, stereopsis, motion perception, flicker perception. We'll talk about those first.

So it was found that there was severe deficits after a parvocellular lesion, meaning when the midget system was blocked in color vision, and texture perception, pattern perception, and shape perception. Also in contrast sensitivity and severe in stereopsis.

None of those cause a deficit with the magnocellular region, mean eliminating the parasol system. But when examine motion perception and flicker perception, there was a moderate to major deficit where that system was missing. So that's then established. I'll come back to these later. Established at least in some people's mind why these two systems have emerged in the course of evolution.

And so a summary statement to that effect is shown here. If you look at the ability to process spatial frequency by the midget and parasol system, the midget system can process it up to much higher spatial frequencies. The obverse is the case when it comes to temporal frequency. The parasol system can process to much higher levels of rapid motion or flicker, as you can see in this little diagram.

So the midget system extends the range of vision in the spatial frequency wavelength range. And the parasol system extends it in the temporal frequency range. So that's why these two systems have evolved.

And then if you look at this in terms of the relative number of cells in the retina that are devoted to these two attributes-- I told you that in the foveola, there no input at all to the parasol system. So therefore, what about this fine vision that the fovea makes possible for you is due to the fact that area is dominated by the midget system.

Then as you go progressive to the periphery, that ratio changes as I had just shown you because increased emphasis has to be placed a seeing motion and rapid changes in the periphery. So that's what happens with the parasol system's increased number of cells in the periphery that can handle that requirement.

So now we're going to move on and talk about various aspects of visual processing. And we'll start first with color vision and adaptation. As I've shown you before, one of the beautiful advances that had been made initially actually, believe it or not, by Newton-- I mentioned that I think-- was the discovery of-- I shouldn't say discovery-- the invention on the color circle.

Now this invention arose in part because it was established-- it's a well-known fact-- that we don't have opposites along these axes. You don't have a yellowish blue color. You don't have a reddish green color. But anything that's not an opposite in this color circle, you do have. So you have yellowish red, or you have yellowish green, and so on.

So the color circle was then elaborated upon over many years. This is a slightly modified version from what Newton had invented. And this is set up in such a fashion that when you go around this circle-- I should say disk I suppose-- you change the hue of course. And then when you go from the center of the periphery, you increase saturation.

This is not the perfect display, especially because the projector isn't perfect. But the center is supposed to be white. And all this is fairly equal luminent. And so you go from unsaturated to saturated.

Now I will say already at this point another very important factor in appreciating the beauty of the color circle is that when you analyze after images, it was found that if you adapt to something that's yellow, you adapt the eye to this wavelength. And then you shift it to white, then you get an after-effect, which is blue. And if you do that for red, the after-effect is green. And the same thing is all the way around the circle. If you have this one, the after-effect is here.

So the color circle perfectly predicts what you're after images are due to adaptation, which occurs as a result of having bleached selectively the molecules in the various cone types that we have, the three cones, red, green, and blue. So that's the basic rule of the color circle, which can be used extensively. And I think you yourself can have a lot of fun studying this in your off time, which you don't have too much of I'm sure. But it's really a wonderful thing to play around with.

Since this course is rather heavily fact oriented, I want you to remember these basic facts that I had listed before. I just noted along the color circle, you have three attributes, hue, brightness, and saturation. And then I also mentioned to you that there is a distinction between the psychological and the physical attribute of images. And this arrangement is such that I gave you an example of.

For example, when a tree falls in the forest, is there a sound when there's nobody around? And the answer is a distinctive no. Why? Because sound is a psychological attribute. If you're on the other hand, you'd have said well, if a tree falls in the forest, through some wavelength result that are in the range of hearing. And that, of course, yes. But if you say sound, that's something you hear. It's not something that's a physical thing.

So that applies to many aspects of vision, as well as audition and many other senses that you must make a distinction between what your psychological disposition is as opposed to what a physical fact is.

The next thing here is that we have three types of photoreceptors, a short, medium, and long wavelength for the cones. And then we have also a different wavelengths peak for the rods. All of these peaks are broadly tuned to enable you to-- some are in the brain-- examine the relative amounts of information from these three wavelengths.

That then enables you to perceive many, many other colors partly because of color [INAUDIBLE] and partly because of variety of amount of activity of them. And then I mentioned Grassman's laws. Every color has a complementary, which when mixed properly yields gray. That should do with again with the color circle.

So in other words, to go back to that, if you mix yellow and blue in equal amounts, you get white or gray I should say. And the same thing for anything that's and opposite. But then if you mix things which are not at diagonals to each other, then you get an in between color. So if you mix yellow and green, you get yellowish green.

All right so that's Grassman's laws. So if you have non complementary colors, you get intermediate. And if you get complementary colors, you get gray. Again to make sure that you understand this, complementary means this and that, this and that, this and that, which are on the opposites on the lines that intersect the center of the color circle.

We move on. And we talk about Abney's law. That is not very important. And you don't have to remember that. The luminance of a mixture of differently colored lights is equal to the sum of the luminance of its components. That another fact, but you don't need to know that.

The last thing that I want to mention is so-called metamers, which are stimuli which look the same but are the product of different subcompositions of wavelengths. So because we only have three different cone photoreceptors, you can, in a sense, if you will, fool them a little bit by very carefully mixing things up with different wavelengths to activate them equally. So that is what is called a metamer when you can't tell the difference between two stimuli. They look identical even though their wavelength compositions are different.

OK so now another factor that I should note here when it comes to color vision is that when you look at the response characteristics of cells in the retina-- and I'm talking about the retinal ganglion cells-- when we look at the cells in the geniculate, which is here, what you find actually is just a few major categories.

This is a color circle here. And one presents the stimuli around the circle and see how the cell responds. And what you're see here is one cell, which is a blue on cell, a green off cell, a yellow on cell, and a green on cell. Now it turns out if you record from hundreds and hundreds of cells, you only get these categories. You don't ever get any cells which are at the diagonals. So to see the diagonals, something has to be taking place in the cortex on the basis of what is coming in from the retina and the lateral geniculate nucleus.

Now when you come to adaptation, we talk about that quite a bit. And also with after images, I'll come back to that in a minute. It was discovered in some very nice example that you take a cell, and you adapt it to various levels of overall illumination, and then see how the cell responds to it, and then you stimulate the receptive field. What you find is that here's the same cell. Here's a background illumination of a huge range over five log units.

And what you find is that when most of the cell is adapted, it responds always the same. So it's looking not at over all levels of illumination, it's looking at differences in illumination. It's looking at contrast.

Now how many of you remember the formula for contrast? Anybody? All right, I think that's a really good thing to remember. I'm sure when you go to a party, people would be fascinated by you knowing the formula for contrast.

OK, so contrast equals-- you take the stimulus, which is call it x. And you take the background, which you called y. You subtract one from the other. Then you add the two up. And then you multiply this by 100. So that your contrast.

What this formula means that this applies to endless levels of overall illumination. You can do this in the sunshine. You can do this in the moonshine because you're looking at the differences between the background and the stimulus itself.

AUDIENCE: What is x and y?

PROFESSOR: As I've said, x is the illumination level for the target. Suppose you take a cell, and you shine a spot of light on it like that. Then you remove it, and you measure the background level. And so x is a visual stimulus. y is the background.

We talk about light adaptation. Again, I want you to know a few basic facts. The overall level of illumination is close to 10 log units. But in contrast to that, if you just look at reflected light, that varies over a much smaller range because on the very bright illumination conditions, even a black object will reflect some light.

So you're talking about direct light versus reflected light. So when you do reflected light, you get a smaller range. Now the pupil plays a role in the amount of light it controls getting into the eye. But it can only do that over a range of 16 to one. Now because of that, the major role of adaptation has to do with the photoreceptors in your rods and your cones.

And the way that works is, if you remember, is that you can think of your molecules in your photoreceptors as existing in two forms, bleached and unbleached. And because of the millions and millions of them that I told you already about, 1,000 times 10,000 in just a single cone, there's a relative percentage of bleached and unbleached molecules in each cone and in each rod.

And so what is happening is that during dark adaptation, there's a huge increase in the unbleached, and during light adaptation, there's an increase in the bleached molecules. So therefore, any increase in the rate of at which quanta are delivered to the eye is also in the proportion of decrease in the number of pigment molecules available to absorb those quanta.

Retina ganglion cells are selected sensitive to local contrast differences not absolute levels of illumination. I've said that many times over again. OK, and that's why this formula, this contrast formula, is one that's the most useful in being able to depict what kind of input these cells are getting. So that then is the arrangement about light adaptation.

Now let's move on to depth perception, which is one the most intriguing capacities that we have since our retinae essentially are like a two dimensional surface. So whatever comes onto the retina, some mechanisms have to be able to tell you where things are in depth. And because it's such a complex problem, quite a number of different mechanisms have evolved to make it possible for you to do that.

And that means that first we have oculomotor cues. We don't need to talk about those. But we have visual cues, which have binocular cues, stereopsis we talked about quite a bit. And the binocular cues are motion parallax, shading, interposition size, and perspective. All these cues we can utilize to tell us where things are in depth.

So now if we look at stereopsis-- I've handed out to you some of these autostereograms. If you look at these, you can't see it looking at that. You have to do it on those sheets that I handed out to you. You can see something in depth. And this arises by virtue of the fact that stimuli are arranged in such a fashion that they selectively activate neurons in the visual cortex that code depth by virtue of the fact that they get disparity inputs from the two eyes.

Now another central mechanism-- I should add one more thing about stereopsis. I think I've mentioned it. In fact, 10% of the population in the United States lack stereopsis in most cases due to either misalignment of the two eyes or do to ambliopia, meaning one eye doesn't see as well as the other. But those people can still do many things and do depth quite well. They can't thread needles, but they can do courser depth quite well.

And one of those is due to motion parallax. Now the basic rule about motion parallax that cause the brain to evolve to analyze it is that when objects are different distances from the eyes as depicted here, the objects that are closer to the eye when this object moves over a greater range on the retinal surface than those that are further apart. You can see the green versus the red.

So therefore, the system is such that it has evolved to be able to see small differences in the relative motion of objects in the retinal surface. And I showed you an example of that. And I'll show it to you again because this is fun. This is essentially similar to the random autostereogram except it's just a single bunch of random dots.

And as soon as I set this in motion, you see them in three dimensions beautifully because these move over a greater range than these. And these move even less so. So this differential motion commands you to see it in depth.

So that's quite a remarkable ability. And monkeys are even better at it than we are. And even fish have this kind of capacity as do many, many other species. It's so central to our ability to process depth. OK

Studies have been carried out to determine where and how these are analyzed. And when we came to where, here's an example of looking at a brain in a normal stereo-blind subject. When you only present motion parallax, you only presents stereo. And when you do the stereo, monocularly, you don't see depth, and the brain is not active. So this tells you which part of the brain is active and involved in analysis of stereopsis and which one is involved in the analysis of motion parallax.

I showed you this picture and several others telling you which areas it is. The limitation of that is that it can tell you where it takes place in the brain, but it doesn't tell you how it takes place. So because of that, many studies have been carried out doing single cell recordings in these cortical areas. And it was discovered that the there are indeed cells already in area 17 that get disparate input from the two eyes. Beautiful work by [INAUDIBLE] showing this.

And establish therefore that already in area 17, you have neurons that tell you about stereoscopic depth. And then it was also discovered, especially in area MT, to a lesser extent already in V1 also, that you have cells that respond to differential motion. And so those cells are presumptively involved in the processing of depth information based on motion parallax.

Now another mechanism involved that we talked about is shading. Light coming from above like from the sun had been incorporated into the visual system to tell you whether an object is towards you or away from you. And this is an example of that. Here the light is from above, and the darkness is below. This is reversed here. And because of that, you see this as protruding, and you see this as receding. And so I showed you several examples, some in the handout, of the fact that even shading is a hue that's used quite extensively in depth perception.

Now we come to form perception. I'll talk about this briefly. I mentioned three kinds of theories. One is that the former is due to the fact that neurons respond selectively to line segments of different orientations in V1. Another theory was that they have a spatial mapping of the stimuli on to the visual cortex since you have topography. And the third one is that form perceptions are accomplished by Fourier analysis.

We talked about each of these. And I pointed out to you that even when there are no orientation segments in the display, you can still see and identify faces quite well, as seen in the Wall Street Journal where these kinds of pictures appear every day in the paper itself.

Now then if you move on, and you look at the layout of how the cortexes-- this is a monkey cortex here. This is a visual field. If you present these three stimuli in the visual field, this is the area that's activated in the cortex because more area is allocated to central vision and peripheral vision. And so you say oh, this is much bigger than those. But that's not the case. You can tell that they're identical.

Now even more dramatic is the fact that if you put these three disks centered-- OK, so half of it goes to the ipsilater and half of it to the contralateral visual hemisphere. What you get are a bunch of half circles like that. And it doesn't look anything like that.

So the idea that somehow images are laid down in the visual cortex, and the mind then looks at it is totally wrong. It's wrong to the extent that it's ridiculous. The last analysis theory is accepted by some people. And doing computer analyses has revealed that system actually can be mimicked extremely well based on what we know about the organization of the visual cortex.

It has all the basic attributes that you need, orientation, direction selectivity, and phase that enable you to break down the visual scene in an analytical fashion, which is kind of foreign to our thinking, namely Fourier analysis.

Then we spent some time talking about prosthesis, which you're going to hear quite a bit about actually when Chris Brown is going to lecture because that has been so successful in the auditory system with the cochlear implant, which is a remarkable achievement. We have many more than 50,000 by now in the United States who have cochlear implants. And they can talk and do all kinds of remarkable things.

We don't have this in the visual system. And I've mentioned to you that one of the big differences is that in the retina, there are more than a million fibers in each eye that come from the ganglion cells that project into the brain, whereas when you talk about the auditory system, you only have about 30,000 fibers. So the magnitude is much less.

But also there are other factors, namely the retina is a very difficult structure to work with. And also when people become blind, most cases the retina degenerates. So you can't put a device into the eye very effectively in most blind people to create vision.

So another alternative is to try some other regions. Some people have advocated to do this in the visual cortex. And the problem there is we have the huge magnification factor. So if you put 256 stimuli like this in the visual scene, this is the actual physical activation. And once you know what this layout is, then you can put electrodes in, which are spaced like this. Then, if you were to stimulate these, then you would create an image, which is at least moderately similar to this that would be in slightly different washed out colors, but which would still essentially be a square.

So if you do that then, and you take a camera, and you take the input to the camera and connect it selectively to this proportional implant. If you put the word, fiat lux-- remember what that is? Let there be light. You get a pretty good reproduction of what has been put in there. But by contrast, if you take a ray of electrodes, which are equally spaced-- then, if you activated all those, you would get a butterfly image.

And if you then put in the fiat lux to the camera, it would look like that. So that would mean that they would get a pretty false impression of the world, and you wouldn't be able to even read. So therefore, it would be very important to take into account the functioning of the visual system, as well as functioning of the individual neurons if you are going to create a prosthetic device.

So now we will move on. And I will say a few words about illusions. We talked about quite a few illusions, and you got some of those in the hand out. The one I mentioned to you that I think all of you enjoyed is the Hermann grid illusion, which shows the smudges at the smudges at the intersections. And the famous theory that was advanced by Baumgartner is that it's due to the fact that if you have a cell that is centered around here, as opposed to not at the intersections, this cell would be inhibited more than this one.

So this hypothesis had been accepted by many people a few years back, and it has appeared in many, many textbooks. It turns out this theory is all wrong if you remember because first of all, here you just make a small change in the physical layout of the lines. And you don't get the effect at all, even though if you put a cell here, as opposed to here, the arrangement is still the same. So consequently, that theory is wrong.

And it's even further proven by the fact that when you analyze it physically to see what the number of cells is in this area here-- and this is for parasol cells. And this is for the midget cells. You have a huge number of cells. And this is shown only for the on cells. You can double that for the off cells.

You would activate this teeny area here, five degrees from the fixation. You would activate 365 midget cells and 50 parasol cells, half of which would be on and half of which would be off. So this theory is just incorrect. And so alternative ideas have been developed still sort of under debate. And one is that this takes place because of the simple cells in the visual cortex as we have talked about it.

Now then, another set of illusions we talked about are the after-effect illusions. And the experiments that we so informally did here asked if you look at a particular display, you fix it for a while, and then you change the display, you have an after-effect, right? A very dramatic after-effect. And one of those was the rotating dots in the circle.

And I showed it to you. The experiment was that you adapt to it with one eye, and then you look at the display afterwards with the other eye, and then you would have no effect, which proves that this takes place in the retina and proves it is due to the adaptation that takes place in the photoreceptors. All right so those were the so-called interlocking experiments we had discussed.

So now let me move on and talk some more about the deficits in vision arise as a function of lesions. And I already showed you a whole set of those when we talked about the lesions of the midget and parasol systems. And now if you look at this in more detail, we add to this, what happens when you remove V4 and remove MT?

And it's quite striking that the deficits are far, far less when you take out the midget system. You have very mild deficits with V4 lesions for most of these up here. These are basic visual capacities. But MT lesions do give you pretty much the same deficits as a magnocellular lesion that blocks the parasol system.

Now then when higher level visual capacities we have now analyzed-- I showed you those as well-- you found that there was some dramatic deficits with V4 lesions when monkeys had to choose less a stimuli and had to learn visual percepts, they had severe deficits with a V4 lesion. So that suggests that an area like V4 plays a very important role in higher level visual processing. Yes?

AUDIENCE: What does the pronounced mean? Is that--

PROFESSOR: Pronounced? It means like a strong deficit.

AUDIENCE: So more than severe?

PROFESSOR: No, no. You can see by the color also. Severe is the strongest. Pronounced is strong. Moderate is weaker. And mild is mild. [CHUCKLES]

OK so now, next I want to turn to eye movement control. And when we do that I want to remind you that the many cortical areas as well as the subcortical areas that play a significant role in eye movement control. And one way to test this is to electrically stimulate various regions in the brain and see if you get any eye movements. And this happens in many areas.

The ones we have here are superior colliculus, of course, we talked a lot about. V1, LIP, the medial eye fields, and the frontal eye fields. Now in all but one of these you get a constant vector [INAUDIBLE] at any sight where you stimulate, meaning no matter where the eye is looking when you stimulate, you've got a particular vector as depicted here.

The exception to that is the medial eye fields where you have a place code. The result of stimulating any given area is to bring the eye, normally where the eye is, into that motor field. Now different regions obviously in these areas have different vectors. So that's the basic layout.

And then the question arose how do these get down to the brain stem ocularmotor complex that drives the eye muscles that we had talked about. Well, the way the experiment was done then is to remove the superior colliculus. And when that was done, what you found was really quite dramatic, namely that you could no longer drive cells from the posterior part of the cortex, but you could still drive them from the anterior part.

This led to the idea that you have two major systems in saccadic eye movement generation, the so-called posterior system and the anterior system. And then when people looked at the question of well, we have these two systems. What do they do? It was discovered that the posterior system is very important for generating quick saccades, especially express saccades, because when you remove the colliculus, you never got an express saccade again.

And the anterior system plays a very important role in stimulus selection and the sequencing of eye movements because you make so many eye movements in rapid succession, you have to make plans ahead to decide where you're going to look in a sequence. And that was found to be very important for the frontal eye fields because when you remove that, there was a major deficit in target selection and in sequencing.

So then when this was done, we also examined, if you remember the question, of what is the role of these various areas when you block inhibition or you increase inhibition. So we use muscimal and bicuculline to do that as shown here. And it showed that with V1, you get a strong interference with both, and you also get a strong deficit in visual discrimination because to be able to analyze the visual scene, you need to have interaction between excitation and inhibition both for eye movements and for visual discrimination.

And then with a frontal eye field lesion, you've got facilitation as you did in the colliculus when you put in bicuculline, which eliminates inhibition. The monkey couldn't help but makes saccades. But you've got interference with muscimal. LIP had no effect.

So that's then in a summary was what we had discussed. And this is something, of course, you need to go over again in your notes, and in Stellar, and in the assigned readings so that you can remember this for the exam.

OK and then I pointed out to you that even though we never think of eye movements, we have an incredible number of structures and a number of tasks to be able to make each eye movement. We have to select a target. We have to decide what each-- every time you move your eye, there are dozens of targets. We have to select one of those. Then we have to decide what they are. Then we have to decide which one to look at which one not to look at. And then we have to use our system, which is a spatial organization of the motor fields to eventually generate an eye movement.

Now in reality then what happens is that many other systems-- I showed you this before, too-- many other systems are involved in generational eye movements, hearing, touch, so on. And we had generated all sorts of systems to enable for you to do this. the so-called anterior and the posterior systems that reach the brain stem through various channels here. This is available for you on the internet. It's also available to you on the assigned reading.

So now lastly we'll turn to motion perception. And when we talked about motion perception, I pointed out to you that in the area of V1, we have simple cells, and complex cells, several different classes of simple cells. And almost all of these cells, if you look at their responses to light increment and light decrement, meaning light edges and dark edges, almost every one of these cells is direction selective. And it's also true for most complex cells. About half the complex cells, maybe more, are also direction selective.

So direction selectivity is one of the most central features in the visual system that we use extensively not just to analyze motion but also to be able to see depth by virtue of motion. Paradox

So now we can say, because of all those little experiments I had shown you, that the parasol system and because of the lesion experiments plays a central role in motion analysis. And when we do those experiments with a apparent motion, when we moved little spots in color or in small differences in shape, color and small difference in shape didn't matter, indicating that the parasol system plays a central role in us seeing apparent motion the way we see it.

All right. So now, last very briefly I want to say here is about the accessory optic system because this is what you're going to be writing about. And I just wanted to remind you that the basic discovery was that in the retina, the cells of [INAUDIBLE] that feed into the accessory optic system come in three different direction selectivities as shown here and that these three direction selectivities correspond to the direction selectivity of semicircular canals.

That's quite a remarkable discovery. And this then enables the organism through the system, which by the way these cells respond to all the slow movements, that's prime function is to-- so they claim, and I think that's correct-- is to stabilize the eye with respect to the world.

So when you walk around, what happens is you can still see the world very clearly with no blurring because the accessory optic system adjusts the eye to keep it stable with respect to the world out there. And in fact, I can't remember if I told you this story. Way back when in Germany when some people were treated for pneumonia, they used the drug-- I forget the name of it right now-- that caused malfunctioning in the semicircular canals.

As a result of that, that system, which co-exists with the accessory optic system no longer was able to stabilize the eyes. And so here was this guy in Munich living in a neighborhood where he had lived for many, many years. And he realized that he can't see anything clearly when he's walking on the street. Everything was blurry.

And so he said oh, my god. I won't be able to recognize my friend. I won't be able to say hello to him. And so what he learned to do is this. Hi, Joe. Like that. He stabilized his head by holding it.

So that highlights for you the fact that this system of stabilizing the eye through the accessory optic system plays a very important central role in your being able to move around in the world and being able to analyze the visual scene in spite of the fact that you're moving around.

So the last thing I wanted to show you-- first, I'm going to show you one more picture. But first, let me say a couple of words about the exam again. I've told you the exam is a multiple choice exam, probably something about 100 or so questions. Almost all of them deal with basic facts, I should say basic facts. So you've got to know your facts.

And what you need to do is read each question, circle the choice. You don't get punished extra for being wrong. If you're wrong, you're wrong. But I'm not going to subtract on top of that the wrong answers from the right answers. So choose an answer, even if you don't know it for every question.

And you'll have a probability of one in four, maybe one in five of getting the right answer if you're totally ignorant. So that's what the exam is. It's going to take about an hour or so, hour and a half maybe, depending on how fast you read and how fast you make decisions. And that's going to take place this coming Wednesday right in this room.

Now the last thing I wanted to show you is-- I mean I seem to be so certain about everything being right and wrong here. I just wanted to tell you one thing, a note of caution. And the caution is this one here. This is a wonderful sculpture by Naum Gabo. I don't know if you've ever heard of Naum Gabo.

Anyway, this is obviously you can almost instantly say it's an upper body and a face, right? But the fact is that, as I say here, as many scientific hypotheses of brain function are appealing but a far cry from the real McCoy. So we are still groping. And yes, we are a long way from phrenology, but still many of the hypotheses and ideas that we have are wrong and are more like a cartoon of what it really is like.

And of course, the further up, in my opinion, you go from the retina, the higher the fancifulness of the ideas. At least when it comes to the retina, I think we are reasonably comfortable that we know a lot about the photoreceptors and how they interact. And that's fairly close to the way it really is. So that may be more like a photograph of Obama. But when it comes to the cortex of the higher areas, things are a bit more like that.

So that's the end of it then. Thank you very much. And I wish you the best of luck on your exam on Wednesday.

[APPLAUSE]

Oh, thank you. Thank you. That's very nice. I appreciate it.