Lecture 6: Column space and nullspace

Related Resources

Readings
Table of Contents

 

• Download English-US transcript (PDF)

Okay. This is lecture six in linear algebra, and we're at the start of this new chapter, chapter three in the text, which is really getting to the center of linear algebra.

And I had time to make a first start on it at the end of lecture five.

But now is lecture six is officially the lecture on vector spaces and subspaces.

And then especially -- there are two subspaces that we're specially interested in.

One is the column space of a matrix, the other is the null space of the matrix.

So, I got to tell you what those are.

Okay. So, first to remember from lecture five, what is a vector space?

It's a bunch of vectors that -- where I'm allowed -- where I can add -- I can add any two vectors in the space and the answer stays in the space.

Or I can multiply any vector in the space by any constant and the result stays in the space.

So that's -- in fact if I combine those two into one, you can see that -- if I can add and I can multiply by numbers, that really means that I can take linear combinations.

So the quick way to say it is that all linear combinations, C -- any multiple of V plus any multiple of W stay in the space.

So, can I give you examples that are vector spaces and also some examples that are not, to make that point clear?

So, suppose I'm in three dimensions.

Then one way to get us one space is the whole three dimensional space.

So the whole space R^3, three dimensional space, would be a vector space, because if I have a couple of vectors I can add them and I'm certainly okay and they follow all the rules.

So R^3 is easy.

Now I'm interested also in subspaces.

So there's this key word, subspaces.

That's a space -- that's some vectors inside the given space, inside R three that still make up a vector space of their own.

It's a vector space inside a vector space.

And the simplest example was a plane.

So, like, can I just sketch it -- there is a plane.

It's got to go through the origin, and of course it goes infinitely far.

That's of that's a subspace now.

Do you see that if I have two vectors on the plane and I add them, the result stays in the plane.

If I take a vector in the plane and I multiply by minus two, I'm still in the plane.

So that plane is a subspace.

So let me just make that point.

Plane through zero, through that zero zero zero is a subspace.

Okay. And also, another subspace would be a line.

A line through zero zero zero -- yeah, the line has to go through the origin.

All subspaces have got to contain the origin, contain zero -- the zero vector.

So this line is a subspace.

Really, if I want to say it really correctly, I should say a subspace of R^3. That of R^3 was, like, understood there.

Now -- so let me call this plane P. And let me call this line L.

And let me ask you about other sets of vectors.

Suppose I took -- yeah -- so here's a first question.

Suppose I take two subspaces, like P and L.

And I just put them together, take their union, take all the vectors -- so now you've got P and L in mind, here.

So I have two subspaces.

I have two subspaces and, for example, P -- a plane and L a line.

Okay.

Now I want to ask you about the union of those.

So P union L.

This is all vectors in P or L or both.

Is that a subspace?

Is this a subspace?

This is or is not a subspace?

Because we're -- I just want to be sure that I've got the central idea.

Suppose I take the vectors in the plane and also the vectors on that line, put them together, so I've got a bunch of vectors, is it a subspace?

Can you give me, like, so the camera can hear it or maybe the tape.

Can you say yes or no?

Do I have a subspace if I put -- if I take all the vectors on the plane plus all -- and all the ones on the line and just join them together -- but I'm not taking this guy that's -- actually, I'm not taking most of them, because most vectors are not on the line or the plane, they're off somewhere else.

Do I have a subspace?

STUDENTS: No.

STRANG: Right.

Thank you.

No. Because -- why not?

Because I can't add.

Because if I that requirement isn't satisfied.

If I take one vector like this guy and another vector that happens to come from L and add, I'm off somewhere else.

You see that I've gone outside the union if I just add something from P and something from L, then normally what'll happen is I'm outside the union -- and I don't have a subspace.

So the correct answer is -- is not.

Okay.

Now let me ask you about -- the other thing we do is take the intersection.

So what does intersection mean?

Intersection means all vectors that are in both P and L.

Is this a subspace.

Yeah, so I guess I want to go back up to the same question.

This is or is not a subspace?

And you can answer me -- answer the question first for this particular example, this picture I drew.

What is P intersect L for this case?

STUDENT: It's only zero.

STRANG: It's only zero.

At least, sort of this was the artist's idea as he drew it that, that that line L was not in the plane and, went off somewhere else -- and then the only point that was in common was the zero vector.

Is the zero vector by itself a subspace?

STUDENT: Yes.

STRANG: Yes, absolutely.

And what about, if I don't have this plane and this line but any subspace and any other subspace?

So now -- can I ask that question for any two subspaces?

So maybe I'll write it up here.

If I'm strong enough.

Okay.

So this is the general question.

I have subspaces, say S and T.

And I want to ask you about their intersection S intersect T and I want -- it is a subspace.

Do you see why?

Do you see why if I take the vectors that are in both one- th- that are in both of the subspaces -- so that's like a smaller set of vectors, probably, because it's -- we've added requirements.

It has to be in S and in T.

How do I know that's a subspace?

Can we just think through that abstract stuff and then I get to the examples.

Okay.

So why? Suppose I take a couple of vectors that are in the intersection.

Why is the sum also in the intersection?

Okay, so let me give a name to these vectors, say V and W.

They're in the intersection.

So that means they're both in S.

Also means they're both in T.

So what can I say about V plus W?

Is it in S?

Yes. Right?

If I take two vectors, V and W that are both in S, then the sum is in S, because S was a subspace.

And if they're both in T and I add them, then the result is also in T, because T was a subspace.

So the result V plus W is in the intersection.

It's in both and requirement one is satisfied.

Requirement two's the same.

If I take a vector that's in both, I multiply by seven.

Seven times that vector is in S, because the vector was in S.

Seven times that vector's in T because the original one was in

T. So seven times that vector is in the intersection.

In other words, when you take the intersection of two subspaces, you get probably a smaller subspace, but it is a subspace.

Okay.

So that's like sort of just emphasizing what these two requirements mean.

Again -- Let me circle those, because those are so important.

The sum and the scale of multiplication which combines into linear combinations.

That's what you have to do inside the subspace.

Okay. On to the column space.

Okay. So my lecture last time started that and I want to continue it.

Okay.

Column space of a matrix.

Of A.

Okay.

Can I take an example?

Say one two three four.

One one one one.

Two three four five.

Okay.

That's my matrix A.

So, it's got columns, three columns.

Those columns are vectors, so the column space of this A, of this A -- let's stay with this example for a while.

The column space of this matrix is a subspace of R -- R what?

So what space are we in if I'm looking at the columns of this matrix?

R^4 , right?

These are vectors in R^4, they're four dimensional vectors.

So it's this column space of A is a subspace of R^4 here, because A was four by -- A is a four by three matrix.

This tells me how many rows there are, how many components in a column, and so we're in R^4. Okay, now what's in that subspace?

So the column space of A, it's a subspace of R^4. I call it the column space of A, like that.

So that's my little symbol for some subspace of R^4. What's in that subspace?

Well, that column certainly is.

One two three four.

This column is in.

This column is in, and what else?

So it's got the columns of A in it, but that's not enough, certainly.

Right?

I don't have a subspace if I just put in three vectors.

So how do I fill that out to be a subspace?

I take their linear combinations.

So the column space of A is all linear combinations -- combinations of the columns.

And that does give me a subspace.

It does give me a vector space, because if I have one linear combination and I multiply by eleven, I've got another linear combination.

If I have a linear combination, I add to another linear combination I get a third combination.

So that -- this is like the smallest space -- like, it's got to have those three columns in it, and it has to have their combinations and that's where we stop.

Okay. Now I'm going to be interested in that space.

So I, like -- get some idea of what's in that space.

How big is that space?

Is that space the whole four dimensional space?

Or is it a subspace inside?

Can you -- let me just see if we can get a yes or no answer sometimes without being ready for the complete proof.

What do you think?

Is the subspace that I'm talking about here, the combinations of those three guys, does that fill the full four dimensional space?

Maybe yes or no on that one.

No. No. Somehow our feeling is, and it happens to be right, that if we start with three vectors and take their combinations, we can't get the whole four dimensional space.

Now -- so somehow we get a smaller space.

But how much smaller?

That's going to come up here.

That's not so immediate.

Let me first make this critical connection with -- with, linear equations, because behind our abstract definition, we have a purpose.

And that is to understand Ax=b.

So suppose I make the connection -- w- w- does A x=b always have a solution for every b?

Have a solution for every right-hand side?

I guess that's going to be a yes or no question.

And then I'm going to ask which right-hand sides are okay?

That's really the question I'm after, is which right-hand sides (b) do make up -- you can see from the way I'm speaking what the question -- As it is.

The answer is no.

A x=b does not have a solution for every b.

Why do I say no?

Because A x=b is -- like, this is four equations, and only three unknowns.

Right?

X is -- let me right out that whole -- what the whole thing looks like.

Yeah. Let me write out A x=b.

A x is -- these columns are one two three four.

One one one one and two three four five.

Then x, of course, has three components, x1, x2, x3. And I'm trying to get the -- hit the right-hand side, b1,b2,b3 and b4. So my first point is, I can't always do it.

In a way, that just says again what you told me five minutes ago -- that the combinations of these columns don't fill the whole four dimensional space.

There's going to be some vectors b, a lot of vectors b, that are not combinations of these three columns, because the combinations of those columns are, like, going to be just a little plane or something inside -- inside R^4. Now, so and you see that I do have four equations and only three unknowns.

So, like anybody is going to say, no you dope, you can't usually solve four equations with only three unknowns.

But now I want to say sometimes you can.

For some right-hand sides, I can solve this.

So that's the bunch of right-hand sides that I'm interested in right now.

Which right-hand sides allow me to solve this?

This is the question for today.

It's going to have, like, a nice clear answer.

So my question is -- is which bs, which vectors b, allow this system to be solved?

And I want to ask you -- so that's, like, gets two question marks to indicate that's -- this is the important question.

Okay, first, before we give a total answer, give me just a partial answer.

Tell me one right-hand side that I know I can solve this thing for.

So -- all zeroes. Okay. That's the, like, guaranteed.

If these were all zero, then I know I can solve it, let the x-s all be zero, no problem.

So that's always a -- okay.

Okay.

A x=0 I can always solve.

Now tell me another right-hand side, just a specific set of numbers for which I can solve these three -- these four equations with only three unknowns, but if you give me a good right-hand side, I can do it.

So tell me one?

STUDENT: 1 2 3 4.

STRANG: 1 2 3 4? If I -- can I solve -- is that a good right-hand side?

Can you solve -- can you find a solution that -- X one plus X two plus two X three is one, two X one plus X two plus three X three is two and two more equations -- so I'm asking you to solve in your head in -- within five seconds, four equations and three unknowns, but you can do it, because the right-hand side is, like, showing up here is -- it's one of the columns.

So tell me what's the X that does solve it?

One zero zero.

One zero zero solves it, because -- well, so you can multiply this out by rows, but oh God, it's much nicer to say -- okay, this is one of this column, zero of this, zero of this, so it's one of that column, which is exactly what we wanted.

Okay.

So there is a b that's okay.

Now tell me another B that's okay, another right-hand side that would be all right?

Well -- all ones?

Actually -- and then what's the solution in that case?

0 1 0, thanks.

And, in fact, it's much e- like, one way to do it is think of a solution first, right, and then just see what b turns out to be.

What b turns out to be, right.

Okay.

So I think of a solution -- so I think of an x, I think of any -- x1, x2, x3, I do this multiplication and what have I got?

Now I'm ready to answer the big question.

I can solve A x=b exactly when the right-hand side B is a vector in the column space.

Good. I can solve A x=b when b is a combination of the columns, when it's in the column space -- so let me write that answer down.

I can solve Ax=b exactly when B is in the column space.

Let me just say again why that is.

Because it -- the column space by its definition contains all the combinations.

It contains all the Ax-s. The column space really consists of all vectors A times any X.

So those are the bs that I can deal with.

If b is a combination of the columns, then that combination tells me what X should be.

If b is not a combination of the columns, then there is no x.

There's no way to solve A x equal b.

Okay.

So the column space -- that's really why we're interested in this column space, because it's the central guy.

It says when we can solve, and that -- we got to understand this column space better.

Let's see.

Do I want to think -- yeah, somehow -- oh, well, let's just -- as long as we've got it here, what do I get for this particular example?

If I take combinations of this and this and this, I'll tell you the question that's in my mind.

It's not even proper to use this word yet, but you'll know what it means.

Are those three columns independent?

If I take the combinations of the three columns -- does each column contribute something new or now?

So that if I take the combinations of those three columns, do I, like, get some three dimensional subspace -- do I have three vectors that are, like, you know, independent, whatever that means?

Or do I -- is one of those columns, like, contributing nothing new -- So that actually only two of the columns would have given the same column space?

Yeah -- that's a good way to ask the question.

Finally I think of it.

Can I throw away any columns -- and have the same column space?

STUDENT: Yes.

STRANG: Yes.

And which one do you suggest I throw away?

STUDENT: Column three -- three.

STRANG: Well, three is the natural, like, guy to target.

So if I -- and why?

Because -- what's so bad about three here?

Column three?

It's the sum of these, right?

So it's not -- if I'm taking -- if I have combinations of these two and I put in this one, actually, I don't get anything more.

So later on I will call these pivot columns.

And the third guy will not be a pivot column in this -- with those numbers.

Now actually -- honesty makes me ask you this question.

Could I have thrown away column one?

Yes, I could.

I could.

So when I say pivot columns, my convention is, okay, I'll keep the first ones as long as they're not

dependent. So I keep this guy, he's fine, he's a line.

I keep the second guy.

It's in a second direction.

But the third one, which is in the same plane as the first two gives me nothing new.

It's dependent in the language that we will use and I don't need it.

Okay. So I would describe the column space of this matrix as a two dimensional subspace of R^4. A two dimensional subspace of R^4. Okay. So you're seeing how these vector spaces work and you -- you're seeing that we -- some idea of dependence or independence is in our future.

Okay.

Now I want to speak about another vector space, the null space.

So again I'm getting a little ahead because it's in section three point two, but that's okay.

All right.

Now I'm ready for the null space.

Let me keep the same matrix.

And this is going to be a different -- totally different subspace.

Totally different.

Okay.

Now -- so let me make space for it.

Now -- here comes a completely different subspace, the null space of A.

What's in it?

It contains not right-hand sides b.

It contains x-s. It contains all x-s that solve -- this word null is going to -- I mean, that's the key word here, meaning zero.

So this contains -- this is all solutions x, and of course x is our vectors, x1, x2 and x3, to the equation A x=0. Well, four equations, because we've got -- so, do you see what I'm doing?

I'm now saying, okay, columns were great, the column space we understood.

Now I'm interested in x-s. I'm not -- the only b I'm interested in now is the b of all zeroes. The right-hand side is now zeroes. And I'm interested in solutions.

x-s. So t- where is this null space for this example?

These x-s are -- have three components.

So the null space is a subspace -- we still have to show it is a subspace -- of R^3. So this is -- and we will show -- these vectors x, this is in R^3, where the column space was in R^4 in our example.

For an m by n matrix, this is m and this is n, because the number of columns, n, tells me how many unknowns, how many x-s multiply those columns, so it tells me the big space, in this case R three that I'm in.

Now tell me -- why don't we figure out what the null space is for this example, just by looking at it.

I mean, that's the beauty of small examples, that my official way to find null spaces and column spaces and get all the facts straight would be elimination, and we'll do that.

But with a small example, we can see that -- see what's going on without going through the mechanics of elimination.

So this null space -- so I'm talking about -- again, the null space, and let me copy again the matrix.

One two three four, one one one one and two three four five.

What's in the null space?

So I'm taking A times x, so let me right it again, and I want you to solve those four equations.

In fact, I want you to find all solutions to those four equations.

Well, actually, just first of all find one.

Why should I ask you for all of them?

Tell me one -- well, tell me one solution that y- you don't even have to look at the matrix to know one solution to this set of equations.

It is zero vector.

Whatever that matrix is, its null space contains zero -- because A times the zero vector sure gives the zero right-hand side. So the null space certainly contains zero.

A- so it's got a chance to be a vector space now, and it will turn out it is.

Okay. Tell me another solution.

So this particular null space -- and of course I'm going to call it N(A) for null space -- this contains-- well we've already located the zero vector, and now you're going to tell me another vector that's in the null space, another solution, another x, another -- you see what I'm asking you for is a combination of those columns.

That's what I'm always looking at combinations of columns, but now I'm looking at the weights, the coefficients in the combination.

So tell me a good set of numbers to put in there.

One one -- STUDENTS: Minus one.

STRANG: One one minus one.

Thanks.

One one minus one.

So there's a vector that's in it.

Okay.

But have I got a subspace at this point?

Certainly not, right?

I've got just a couple of vectors.

No way they make a subspace.

Tell me -- actually, why don't I jump the whole way now?

Tell me -- well, tell me one more solution, one more X that would work.

Student: 2 2 -2.

STRANG: 2 2 -2?

Oh, well, tell me all of them, that would have been easier.

Tell me the whole lot, now.

What is the null space for this matrix?

It's all vectors of the form -- what could this be?

It could be one one minus one, it could be it could be any number C, any number -- the same number again and -- STUDENTS: Minus.

STRANG: Minus C.

In other words -- actually, any multiple of this guy.

Oh, that's the perfect description, because now the zero vector's automatically included because C could be zero.

The vector I had is included, because C could be one.

But now any vector.

And that's actually it.

And do I have a subspace?

And what does it look like?

It's in -- how would you describe this, the null space, this -- all these vectors of this form C C minus C, like, seven seven minus seven.

Minus eleven minus eleven plus eleven.

What have I got here?

If -- describe that whole null space of -- what -- if I drew it, what do I draw?

A line, right?

The null space is a line.

It's the line through -- in R^3 and the vector one one negative one maybe goes down here, I don't know where it goes, say, down here.

There's the vector one one negative one that you gave me.

And where is the vector C C negative C?

It's on this line.

Of course, there's zero zero zero that we had.

And what we've got is that whole -- oops -- that whole line, going both ways, through the origin.

The null space is a line in R^3. Okay.

For that example, we could find all the combinations of the columns that gave zero at sight.

Now, can I just take one more time, to go back to the definition of subspace, vector space, and ask you -- how do I know that the null space is a vector space?

How I entitled to use this word space?

I'll never use that word space without meaning that the two requirements are satisfied.

Can we just check that they are?

So I'm going to check that -- can I just continue here?

Check that -- that the solutions to A x=0 always give a subspace.

And, of course, the key word is that= "Space." So what do I have to check?

I have to show that if I have one solution, call it x, and another solution, call it x*, that their sum is also a solution, right?

That's a requirement.

To use that word space, I have to say -- I have to convince myself that if A x is zero and also -- and A x* is zero, or maybe I should have said if A v is zero and A w is zero, then what about v plus w?

Shall I -- let me use those letters.

If A v is zero and A w is zero, then what -- if that and that, then what's my point here?

That A times (v+w) must be zero.

That says that if v is in the null space and w's in the null space, then their sum v+w is in the null space.

And of course, now that I've written it down, it's totally absurd, ridiculously simple -- because matrix multiplication allows me to separate that out into A v plus A w.

I shouldn't say absurdly simple.

That was a dumb thing to say.

Could -- we've used, here, a basic law of matrix multiplication.

Actually, we've used it without proving it, but that's okay.

We only live so long, we just skip that proof.

I think it's called the distributive law that I can split these -- split this into two pieces.

But now you see the point, that A v is zero and A w is zero so I have zero plus zero and I do get zero.

It checks.

And, similarly, I have to show that if A v is zero, then A times any multiple, say 12v is also zero.

And how do I know that?

Because I'm allowed to s- bring that twelve outside.

A number, a scaler can move outside, so I have twelve A vs, twelve zeroes -- I have zero.

Okay.

Just to -- it's really critical to understand the -- oh yeah.

Here -- I was going to say, understand what's the point of a vector space?

Let me make that point by changing the right-hand side.

Oops.

Okay. Let me change the right-hand side to one two three four.

Oh, okay.

Why don't we do all of linear algebra in one lecture, then we -- okay.

I would like to know the solutions to this equation.

For those four equations.

So I have four equations.

I have only three unknowns, so if I don't have a pretty special right-hand side there won't be any solution at all.

But that is a very special right-hand side.

And we know that there is a solution, one zero zero.

Were there any more solutions?

And did they form a vector space?

Okay.

So I'm asking two questions there.

One is, do -- so my right-hand side now is not zero anymore.

I'm not looking at the null space because I changed from zeroes. So my first question is, do the solutions, if there are any and there are, do they form a subspace?

Let's answer that question first.

Yes or no.

Do I get a subspace if I look at the solutions to -- let me go back to x1 x2 x3. I'm looking at all the x-s, at all those vectors in R^3 that solve A x -b.

The only thing I've changed is b isn't zero anymore.

Do the x-s, the solutions, form a vector space?

The solutions to this do not form a subspace.

The solutions don't, because -- how shall I see that?

The zero vector is not a solution, so I never even got

started. The zero vector doesn't solve this system.

I can't -- solutions can't be a vector space.

Now what are they like?

Well, we'll see this, but let's do it for this example.

So one zero zero was a solution.

You saw that right away.

Are there any other solutions?

Can you tell me a second solution to this system of equations?

STUDENTS: 0 -1 1 STRANG: 0 -1 1. Boy, that's -- 0 -1 1. Yes.

Because that says I take minus this column plus this one and sure enough.

That's right.

So there are -- there's a bunch of solutions here.

But they're not a subspace.

I'll tell you what it's like.

It's like a plane that doesn't go through the origin, or a line that doesn't go through the origin.

Maybe in this case it's a line that doesn't go through the origin, if I graft the solutions to A x equal B.

So you -- I think you've got the idea.

Subspaces have to go through the origin.

If I'm looking at x-s, then they'd better solve Ax=0. In a way I've got -- my two subspaces that I -- talking about today are kind of the two ways I can tell you what a -- about subspace.

If I want to tell you about the column space, I tell you a few columns and I say take their combinations.

Like I build up this subspace.

I put in a few vectors, their combinations make a subspace.

Now, when I went to -- let me come back to the one that is a subspace here.

Here, when I talked about the null space, I didn't tell you what's in it.

We had to figure out what was in it.

What I told you was the equations that I'm -- that has to be satisfied.

You see those -- like, those are the two natural ways to tell you what's in a subspace.

I can either give you a few vectors and say fill it out, take combinations -- or I can give you a system of equations, the requirements that the x-s have to satisfy.

And both of those ways produce subspaces and they're the important ways to construct subspaces.

Okay, so today's lecture actually got, the essentials of three point two, the idea of the null space.

Now we have to tackle, Wednesday, the job of how do we actually get hold of that subspace in an example that's bigger and we can't see it just by eye.

Okay.

See you Wednesday.

Thanks.

Free Downloads

Free Streaming

• VideoLectures.net

Video

• iTunes U (MP4 - 101MB)
• Internet Archive (MP4 - 185MB)

Caption

• English-US (SRT)