Lecture 11: Matrix spaces; rank 1; small world graphs

Related Resources

Readings
Table of Contents

• Download English-US transcript (PDF)

OK. This is linear algebra lecture eleven. And at the end of lecture ten, I was talking about some vector spaces, but they're -- the things in those vector spaces were not what we usually call vectors.

Nevertheless, you could add them and you could multiply by numbers, so we can call them vectors.

I think the example I was working with they were matrices.

So the -- so we had like a matrix space, the space of all three by three matrices.

And I'd like to just pick up on that, because -- we've been so specific about n dimensional space here, and you really want to see that the same ideas work as long as you can add and multiply by scalars.

So these new, new vector spaces, the example I took was the space M of all three by three matrices.

OK.

I can add them, I can multiply by scalars.

I can multiply two of them together, but I don't do that.

That's not part of the vector space picture.

The vector space part is just adding the matrices and multiplying by numbers.

And that's fine, we stay within this space of three by three matrices.

And I had some subspaces that were interesting, like the symmetric, the subspace of symmetric matrices, symmetric three by threes.

Or the subspace of upper triangular three by threes.

Now I, I use the word subspace because it follows the rule.

If I add two symmetric matrices, I'm still symmetric.

If I multiply two symmetric matrices, is the product automatically symmetric?

No.

But I'm not multiplying matrices.

I'm just adding.

So I'm fine.

This is a subspace.

Similarly, if I add two upper triangular matrices, I'm still upper triangular.

And, that's a subspace.

Now I just want to take these as example and ask, well, what's a basis for that subspace?

What's the dimension of that subspace?

And what's bd- dimension of the whole space?

So, there's a natural basis for all three by three matrices, and why don't we just write it down.

So, so M, a basis for M.

Again, all three by threes.

OK.

And then I'll just count how many members are in that basis and I'll know the dimension.

And OK, it's going to take me a little time.

In fact, what is the dimension?

Any idea of what I'm coming up with next?

How many numbers does it take to specify that three by three matrix?

Nine. Nine is the, is the dimension I'm going to find.

And the most obvious basis would be the matrix that's that matrix and then this matrix with a one there and that's two of them, shall I put in the third one, and then onwards, and the last one maybe would end with the one.

OK. That's like the standard basis.

In fact, our space is practically the same as nine dimensional space.

It's just the nine numbers are written in a square instead of in a column.

But somehow it's different and, and ought to be thought of as -- natural for itself.

Because now what about the symmetric three by threes?

So that's a subspace.

Just let's just think, what's the dimension of that subspace and what's a basis for that subspace.

OK. And I guess this question occurs to me.

If I look at this subspace of symmetric three by threes, well, how many of these original basis members belong to the subspace?

I think only three of them do.

This one is symmetric.

This last one is symmetric.

And the one in the middle with a, with a one in that position -- in the two two position, would be symmetric.

But so I've got three of these original nine are symmetric, but, so this is an example where -- but that's, that's not all, right?

What's the dimension?

Let's put the dimensions down.

Dimension of the, of M, was nine.

What's the dimension of -- shall we call this S -- is what?

What's the dimension of this?

I'm sort of taking simple examples where we can, we can, spot the answer to these questions.

So how many -- if I have a symmetric -- think of all symmetric matrices as a subspace, how many parameters do I choose in three by three symmetric matrices?

Six, right.

If I choose the diagonal that's three, and the three entries above the diagonal, then I know what the three entries below.

So the dimension is six.

I guess what's the dimension of this here?

Let's call this space U for upper triangular.

So what's the dimension of that space of all upper triangular three by threes?

Again six.

Again six.

And, but we haven't got a -- we haven't seen -- well, actually, maybe we have got a basis here for, the upper triangulars.

I guess six of these guys, one, two, three, four, and a, and a couple more, would be upper triangular.

So there's a accidental case where the big basis contains in it a basis for the subspace.

But with the symmetric guy, it didn't have.

The symmetric guy the, basis -- so you see -- a basis is the basis for the big space, we generally need to think it all over again to get a basis for the subspace.

And then how do I get other subspaces?

Well, we spoke before about, the subspace the symmetric matrices and the upper triangular.

This is symmetric and upper triangular.

What's the, what's the dimension of that space?

OK.

Well, what's in that space?

So what's -- if a matrix is symmetric and also upper triangular, that makes it diagonal.

So this is the same as the diagonal matrices, diagonal three by threes.

And the dimension of this, of S intersect U, right -- you're OK with that symbol?

That's, that's the vectors that are in both S and U, and that's D.

So S intersect U is the diagonals.

And the dimension of the diagonal matrices is three.

And we've got a basis, no problem.

OK, as I write that, I think, OK, what about putting -- so this is like, this intersection -- is taking all the vectors that are in both, that are symmetric and also upper triangular.

Now we looked at the union.

Suppose I take the matrices that are symmetric or upper triangular.

What -- why was that no good?

So why is it no -- why I not interested in the union, putting together those two subspaces?

So this, these are matrices that are in S or in U, or possibly both, so they, the diagonals included.

But what's bad about this?

It's not a subspace.

It's like having, taking, you know, a couple of lines in the plane and stopping there.

A line -- this is -- so there's a three dimensional subspace of a nine dimensional space, there's -- ooh, sorry, six.

There's a six dimensional subspace of a nine dimensional space.

There's another one.

But they, they're headed in different directions, so we, we can't just put them together.

We have to fill in.

So that's what we do.

To get this bigger space that I'll write with a plus sign, this is combinations of things in S and things in U.

OK.

So that's the final space I'm going to introduce.

I have a couple of subspaces.

I can take their intersection.

And now I'm interested in not their union but their sum.

So this would be the, this is the intersection, and this will be their sum.

So what do I need for a subspace here?

I take anything in S plus anything in U.

I don't just take things that are in S and pop in also, separately, things that are in U.

This is the sum of any element of S, that is, any symmetric matrix, plus any in U, any element of U.

OK. Now as long as we've got an example here, tell me what we get.

If I take every symmetric matrix, take all symmetric matrices, and add them to all upper triangular matrices, then I've got a whole lot of matrices and it is a subspace.

And what's -- it's a vector space, and what vector space would I then have?

Any idea what, what matrices can I get out of a symmetric plus an upper triangular?

I can get anything.

I get all matrices.

I get all three by threes.

It's worth thinking about that.

It's just like stretch your mind a little, just a little, to, to think of these subspaces and what their intersection is and what their sum is.

And now can I give you a little -- oh, well, let's figure out the dimension.

So what's the dimension of S plus U?

In this example is nine, because we got all three by threes.

So the original spaces had, the original symmetric space had dimension six and the original upper triangular space had dimension six.

And actually I'm seeing here a nice formula.

That the dimension of S plus the dimension of U -- if I have two subspaces, the dimension of one plus the dimension of the other -- equals the dimension of their intersection plus the dimension of their sum.

Six plus six is three plus nine.

That's kind of satisfying, that these natural operations -- and we've -- this is it, actually, this is the set of natural things to do with, with subspaces.

That, the dimensions come out in a good way.

OK. Maybe I'll take just one more example of a vector space that doesn't have vectors in it.

It's come from differential equations.

So this is a one more new vector space that we'll give just a few minutes to.

Suppose I have a differential equation like d^2y/dx^2+ y=0. OK.

I look at the solutions to that equation.

So what are the solutions to that equation?

y=cos(x) is a solution. y=sin(x) is a solution.

y equals -- well, e to the (ix) is a solution, if you want, if you allow me to put that in.

But why should I put that in?

It's already there.

You see, I'm really looking at a null space here.

I'm looking at the null space of a differential equation.

That's the solution space.

And describe the solution space, all solutions to this differential equation.

So the equation is y''+y=0. Cosine's, cosine's a solution, sine is a solution.

Now tell me all the solutions.

They're -- so I don't need e^(ix). Forget that.

What are all the complete solutions?

Is what?

A combination of these.

The complete solution is y equals some multiple of the cosine plus some multiple of the sine.

That's a vector space.

That's a vector space.

What's the dimension of that space?

What's a basis for that space?

OK, let me ask you a basis first.

If I take the set of solutions to that second order differential equation -- there it is, those are the solutions.

What's a basis for that space?

Now remember, what's the, what question I

asking? Because if you know the question I'm asking, you'll see the answer.

A basis means all the guys in the space are combinations of these basis vectors.

Well, this is a basis. sin x, cos x there is a basis.

Those two -- they're like the special solutions, right?

We had special solutions to Ax=b.

Now we've got special solutions to differential equations.

Sorry, we had special solutions to Ax=0, I misspoke.

The special solutions were for the null space just as here we're talking about the null space.

Do you see that here is a -- those two -- and what's the dimension of the solution space?

How many vectors in this basis?

Two, the sine and cosine.

Are those the only basis for this space?

By no means. e^(ix) and e^(-ix) would be another basis.

Lots of bases.

But do you see that really what a course in differential -- in linear differential equations is about is finding a basis for the solution space.

The dimension of the solution space will always be -- will be two, because we have a second order equation.

So that's, like there's 18.03 in -- five minutes of 18.06 is enough to, to take care of 18.03. So there's a -- that's one more example.

OK.

And of course the point of the example is these things don't look like vectors.

They look like functions.

But we can call them vectors, because we can add them and we can multiply by constants, so we can take linear combinations.

That's all we have to be allowed to do.

So that's really why this idea of linear algebra and basis and dimension and so on plays a wider role than -- our constant discussions of m by n matrices.

OK.

That's what I wanted to say about that topic.

Now of course the key, number associated with matrices, to go back to that number, is the rank.

And the rank, what do we know about the rank?

Well, we know it's not bigger than m and it's not bigger than n.

So but I'd like to have a little discussion on the rank.

Maybe I'll put that here.

So I'm picking up this topic of rank one matrices.

And the reason I'm interested in rank one matrices is that they ought to be simple.

If the rank is only one, the matrix can't get away from

us. So for example, let me take -- let me create a rank one matrix.

OK. Suppose it's three -- suppose it's two by three.

And let me give you the first row.

What can the second row be?

Tell me a possible second row here, for, for this matrix to have rank one.

A possible second row is?

Two eight ten.

The second row is a multiple of the first row.

It's not independent.

So tell me a basis for the -- oh yeah, sorry to keep bringing up these same questions.

After the quiz I'll stop, but for now, tell me a basis for the row space.

A basis for the row space of that matrix is the first row,

right? The first row, one four five.

A basis for the column space of this matrix is?

What's the dimension of the column space?

The dimension of the column space is also one,

right? Because it's also the rank.

The dimension -- you remember the dimension of the column space equals the rank equals the dimension of the column space of the transpose, which is the row space of A.

OK, and in this case it's one, r is one.

And sure enough, all the columns are -- all the other columns are multiples of that column.

Now there's -- there ought to be a nice way to see that, and here it is.

I can write that matrix as its pivot column, one two, times its -- times one four five.

A column times a row, one column times one row gives me a matrix, right?

If I multiply a column by a row, that, g- that's a two by one matrix times a one by three matrix, and the result of the multiplication is two by three.

And it comes out right.

So what I want to -- my point is the rank one matrices that every rank one matrix has the form some column times some row.

So U is a column vector, V is a column vector -- but I make it into a row by putting in V transpose.

So that's the -- complete picture of rank one matrices.

We'll be interested in rank one matrices.

Later we'll find, oh, their determinant, that'll be easy, their eigenvalues, that'll be interesting.

Rank one matrices are like the building blocks for all matrices.

And actually maybe you can guess.

If I took any matrix, a five by seventeen matrix of rank four, then it seems pretty likely -- and it's true, that I could break that five by seventeen matrix down as a combination of rank one matrices.

And probably how many of those would I need?

If I have a five by seventeen matrix of rank four, I'll need four of them, right.

Four rank one matrices.

So the rank one matrices are the, are the building blocks.

And out -- I can produce every, I can produce every five by -- every rank four matrix out of four rank one matrices.

That brings me to a question, of course.

OK.

Would the rank four matrices form a subspace?

Let me take all five by seventeen matrices and think about rank four -- the subset of rank four matrices.

Let me -- I'll write this down.

You seem I'm reviewing for the quiz, because I'm asking the kind of questions that are short enough but -- that bring out do you know what these words mean.

So I take -- my matrix space M now is all five by seventeen matrices.

And now the question I ask is the subset of, of rank four matrices, is that a subspace?

If I add a matrix of -- so if I multiply a matrix of rank four by -- of rank four or less, let's say, because I have to let the zero matrix in if it's going to be a subspace.

But, but that doesn't just because the zero matrix got in there doesn't mean I have a subspace.

So if I -- so the, the question really comes down to -- if I add two rank four matrices, is the sum rank four?

What do you think?

If -- no, not usually.

Not usually.

If I add two rank four matrices, the sum is probably -- what could I say about the sum?

Well, actually, well, the rank could be five.

It's a general fact, actually, that the rank of A plus B can't be more than rank of A plus the rank of B.

So this would say if I added two of those, the rank couldn't be larger than eight, but I know actually the rank couldn't be as large as eight anyway.

What -- how big could the rank be, for, for the rank of a matrix in M?

Could be as large as five, right, right.

So they're all sort of natural ideas.

So it's rank four matrices or rank one matrices -- let me, let me change that to rank one.

Let me take the subset of rank one matrices.

Is that a vector space?

If I add a rank one matrix to a rank one matrix?

No.

It's most likely going to have rank two.

So this is -- So I'll just make that point.

Not a subspace.

OK. OK. Those are topics that I wanted to, just fill out the, the previous lectures.

The I'll ask one more subspace question, a, a more, a more, likely example.

Suppose I'm in -- let me put, put this example on a new board.

Suppose I'm in R, in R^4. So my typical vector in R^4 has four components, v1, v2, v3, and v4. Suppose I take the subspace of vectors whose components add to zero.

So I let S be all v, all vectors v in four dimensional space with v1+v2+v3+v4=0.

So I just want to consider that bunch of vectors.

Is it a subspace, first of all?

It is a subspace.

It is a subspace.

What's -- how do we see that?

It is a subspace.

I -- formally I should check.

If I have one vector that with whose components add to zero and I multiply that vector by six -- the components still add to zero, just six times as -- six times zero.

If I have a couple of v and a w and I add them, the, the components still add to zero.

OK, it's a subspace.

What's the dimension of that space and what's a basis for that space?

So you see how I can just describe a space and we -- we can ask for the dimension -- ask for the basis first and the dimension.

Of course, the dimension's the one that's easy to tell me in a single word.

What's the dimension of our subspace S here?

And a basis tell me -- some vectors in it.

Well, I'm going to make ask you again to guess the dimension.

Again I think I heard it.

The dimension is three.

Three.

Now how does this connect to our Ax=0? Is this the null space of something?

Is that the null space of a matrix?

And then we can look at the matrix and, and we know everything about those subspaces.

This is the null space of what matrix?

What's the matrix where the null space is then Ab=0. So I want this equation to be Ab=0. b is now the vector.

And what's the matrix that, that we're seeing there?

It's the matrix of four ones.

Do you see that that's -- that if I look at Ab=0 for this matrix A, I multiply by b and I get this requirement, that the components add to zero.

So I'm really when I speak about S -- I'm speaking about the null space of that matrix.

OK.

Let's just say we've got a matrix now, we want its null space.

Well, we -- tell me its rank first.

The rank of that matrix is one, thanks.

So r is one.

What's the general formula for the dimension of the null space?

The dimension of the null space of a matrix is -- in general, an m by n matrix of rank r?

How many independent guys in the null space? n-r, right? n-r. In this case, n is four, four columns.

The rank is one, so the null space is three dimensions.

So of course y- you could see it in this case, but you can also see it here in our systematic way of dealing with the four fundamental subspaces of a matrix. So what actually what, what are all four subspaces

then? The row space is clear.

The row space is in R^4. Yeah, can we take the four fundamental subspaces of this matrix?

Let's just kill this example.

The row space is one dimensional.

It's all multiples of that, of that row.

The null space is three dimensional.

Oh, you better give me a basis for the null space.

So what's a basis for the null space?

The special solutions.

To find the special solutions, I look for the free variables.

The free variables here are -- there's the pivot.

The free variables are two, three, and four.

So the basis, basis for S, for S will be -- I'm expecting three vectors, three special solutions.

I give the value one to that free variable, and what's the pivot variable if the -- this is going to be a vector in S?

Minus one.

Now they're always added to -- the entries add to zero.

The second special solution has a one in the second free variable, and again a minus one makes it right.

The third one has a one in the third free variable, and again a minus one makes it right.

That's my answer.

That's the answer I would be looking for.

The -- a basis for this subspace S, you would just list three vectors, and those would be the natural three to list.

Not the only possible three, but those are the special three.

OK, tell me about the column space, What's the column space of this matrix A?

So the column space is a subspace of R^1, because m is only one.

The columns only have one component.

So the column space of S, the column space of A is somewhere in the space R^1, because we only have -- these columns are short.

And what is the column space actually?

I just, it's just talking with these words is what I'm doing.

The column space for that matrix is R^1. The column space for that matrix is all multiples of that column.

And all multiples give you all of R^1. And what's the, the remaining fourth space, the null space of A transpose is what?

So we transpose A.

We look for combinations of the columns now that give zero for A transpose.

And there aren't any.

The only thing, the only combination of these rows to give the zero row is the zero combination.

OK. So let's just check dimensions.

The null space has dimension three.

The row space has dimension one.

Three plus one is four.

The column space has dimension one, and what's the dimension of this, like, smallest possible space?

What's the dimension of the zero space?

It's a subspace.

Zero.

What else could it be?

I mean, let's -- we have to take a reasonable answer -- and the only reasonable answer is zero.

So one plus zero gives -- this was n, the number of columns, and this is m, the number of rows.

And let's just, let me just say again then the, the, the subspace that has only that one point, that point is zero dimensional, of course.

And the basis is empty, because if the dimension is zero, there shouldn't be anybody in the basis.

So the basis of that smallest subspace is the empty set.

And the number of members in the empty set is zero, so that's the dimension.

OK. Good.

Now I have just five minutes to tell you about -- well, actually, about some, some, some, this is now, this last topic of small world graphs, and leads into, a lecture about graphs and linear algebra.

But let me tell you -- in these last minutes the graph that I interested in.

It's the graph where -- so what is a graph?

Better tell you that first.

OK.

What's a graph?

OK.

This isn't calculus.

We're not, I'm not thinking of, like, some sine curve.

The word graph is used in a completely different way.

It's a set of, a bunch of nodes and edges, edges connecting the nodes.

So I have nodes like five nodes and edges -- I'll put in some edges, I could put, include them all.

There's -- well, let me put in a couple more.

There's a graph with five nodes and one two three four five six edges.

And some five by six matrix is going to tell us everything about that graph.

Let me leave that matrix to next time and tell you about the question I'm interested in.

Suppose, suppose the graph isn't just, just doesn't have just five nodes, but suppose every, suppose every person in this room is a node.

And suppose there's an edge between two nodes if those two people are friends.

So have I described a graph?

It's a pretty big graph, hundred, hundred nodes.

And I don't know how many edges are in there.

There's an edge if you're friends.

So that's the graph for this class.

A, a similar graph you could take for the whole country, so two hundred and sixty million nodes.

And edges between friends.

And the question for that graph is how many steps does it take to get from anybody to anybody?

What two people are furthest apart in this friendship graph, say for the US?

By furthest apart, I mean the distance from -- well, I'll tell you my distance to Clinton.

It's two.

I happened to go to college with somebody who knows Clinton.

I don't know him.

So my distance to Clinton is not one, because I don't, happily or not, don't know him.

But I know somebody who does.

He's a Senator and so I presume he knows him.

OK.

I don't know what your -- well, what's your distance to Clinton?

Well, not more than three, right.

Actually, true.

You know me.

I take credit for reducing your Clinton distance to three -- what's your distance to Monica.

Not, anybody below -- below four is in trouble here.

Or maybe three, but, right.

So -- and what's Hillary's distance to Monica?

I don't think we'd better put that on tape here.

That's one or two, I guess.

Is that right?

I don't -- well, we won't, think more about that.

So actually, the, the real question is what are large distances?

How, how far apart could people be separated?

And roughly this number six degrees of separation has kind of appeared as the movie title, as the book title, and it's with this meaning.

That roughly speaking -- six might be a fairly -- not too many people.

If you sit next to somebody on an airplane, you get talking to them.

You begin to discuss mutual friends to sort of find out, OK, what connections do you have, and very often you'll find you're connected in, like, two or three or four steps.

And you remark, it's a small world, and that's how this expression small world came up.

But six, I don't know if you could find -- if it took six, I don't know if you would successfully discover those six in a, in an airplane conversation.

But here's the math question, and I'll leave it for next, for lecture twelve, and do a lot of linear algebra in lecture twelve.

But the interesting point is that with a few shortcuts, the distances come down dramatically.

That, I mean, all your distances to Clinton immediately drop to three by taking linear algebra.

That's, like, an extra bonus for taking linear algebra.

And to understand mathematically what it is about these graphs -- or like the graphs of the World Wide Web.

There's a fantastic graph.

So many people would like to understand and model the web.

What the -- where the edges are links and the nodes are, sites, websites.

I'll leave you with that graph, and I'll see you -- have a good weekend, and see you on Monday.

Free Downloads

Free Streaming

• VideoLectures.net

Video

• iTunes U (MP4 - 101MB)
• Internet Archive (MP4 - 185MB)

Caption

• English-US (SRT)