Session 15: Numerical Methods of PDEs: Collocation and Weighted Residuals

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: All right. So you guys have read all about method of weighted residuals. Yes? Makes perfect sense. Yes? We need to go through things today? A little bit. OK. Just a little bit. All right.

So-- actually, but before we do that, I want to-- [INAUDIBLE] [? Let's ?] put up a list of topics that we're going to talk about today.

So I want to start by just talking about the measurable outcomes for the module. Remind you of what it is we're going to do. Then we're going to introduce the model problem. You read this in the pre-class reading. But it's the steady 1D diffusion. And we'll do a few things in general, but more or less we're going to develop the [INAUDIBLE] by thinking about this specific example. We're going to talk about the idea of approximating the solution with basis functions, which is a fundamentally different way of doing the approximation compared to what you've already seen in finite difference and finite volume. Then we'll look at the collocation method. And then finally, we'll get to the method of weighted residuals.

OK. So what I'm going to do on the board is more or less paralleling what's in the notes. And so I can go quickly on the thoughts that are already clear, and we can spend more time on the parts that you have found to be confusing.

OK. So I'm not going to put the screen down because it'll take a while to get the projector on, but I just want to remind you that there's the measurable outcome list embedded on the MIT site. It's also on the mitosis.mitx.mit.edu site.

And so we're into a new module now. There are four modules in the class. The first one was integration methods for ODEs. The second one was finite difference and finite volume methods. And the third one now, which is finite element methods for PDE.

And so I think it's really helpful for you to go back and-- well to go forward at this point-- and look at the outcomes, and think about specifically what it is you're going to have to be able to do. Because as you are going through all the notes, and I know there are a lot of notes for this section, it will help you think about what are the specific skills that need to take away. There'll be homework problems and a project that will help you do that, too.

But for example, describe how the method of weighted residuals can be used to calculate an approximate solution to the PDE, we're going to cover that today. Then to be able to describe the differences between the method of weighted residuals and the collocation method, and the least squares method for approximating a PDE.

So there's a lot of really specific outcomes. I think there's probably about 12 outcomes that are associated with this module. Go and take a look at them, and I think that will hopefully help you divide up the material, and think about what it is that you need to be able to do.

But let's start off by thinking about our model problem. So a model problem just means a simple problem that' we're going to use to develop things on. And so the model problem again is steady 1D heat diffusion in a rod.

And the PDE should be pretty familiar [INAUDIBLE] you by now. It's d by dx of k [? ct dx ?] equal to minus 2. We're thinking about a 1D domain. So here's our domain. That's the x direction. The domain is going to have links l. So we'll have x in general going from minus l over to plus l over 2. The k sitting in here in general is going to be a function of x, although you'll see that just to make things simple in the beginning we'll at some point make k constant. That's the thermal conductivity of the material that the rod is made out of. And this q on the right hand side also can be a function of x, is the heat source. And it's actually heat source per unit [INAUDIBLE] to get units right.

OK, so simple problem set up. PDE specify heat source. We're also going to need to apply boundary conditions when we actually get to specifying a particular problem.

So there's one particular problem that we will use. One that we can compute the analytical solution for it. Because if we can compute an analytical solution, we'll be able to look at errors and things when we look at the numerical approximation. So for the particular case where k is a constant and equal to 1, the length of the rod is 2, so that x goes from minus 1 to plus 1, and the heat source is q of x being 50 e to the x. And again, this is the same example that's in the notes. And boundary conditions, the temperature at the right end of the rod is set to be 100, and the temperature at the left end of the rod is set to be 100.

OK, so for that particular case we have an analytical solution you can get by integrating the PDE. And that turns out to be t of x is minus 50x to the x, plus 50x the hyperbolic sign of 1, plus 100, plus 50 times the hyperbolic cosign of 1. OK, so that's just what comes out of this analytic.

[INAUDIBLE] and that's maybe a little bit hard to think about, so let's just sketch what t of x looks like as a function of x. So there's x going from minus 1 to 1. We've taken x equal to 0-- I mean x equal to minus 1. Then we should be getting the boundary condition back, t is 100. So if we make this 100 on my t-axis, then the solution looks something like this. And it comes back to 100 at the other end.

OK. So that's an analytic solution that we can compute. And again, we're just doing that because when we start looking at the [? miracle ?] approximations, we're going to want something to compare to. So a simple problem where we can actually and integrate things analytically.

OK? Yep. Think I've told you anything.

All right. So let's now start thinking about the solution approximation. Can I-- again, everything I'm writing is scanned in online, so-- I know it's good to copy stuff down and help follow along, but also you will have copies of the full notes.

So now let's start thinking up this idea of approximating the solution. So first let me ask, if we were going to solve this problem using finite differences, what would we do? We'd run to Alex and ask him to help us? What would we do if we wanted to solve this problem using finite differences?

We'd break up the domain into a bunch of cells, and then what? How would we approximate things?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. Good. So we would decide what kind of a scheme we wanted to use, and we would approximate the derivatives. And that's kind of the key part of [INAUDIBLE], is that we would take this-- we'd take k equal 1 constant. So basically we've got a second derivative the t sitting here. We would approximate it using our favorite finite difference [INAUDIBLE]. So maybe the central difference for the second derivative.

So I think it's really key to keep in mind that finite element departs from that mentality right from the beginning. That with finite elements, we're not going to approximate the derivative here. We're not going to approximate this operator, this d squared by dx squared that's acting on t. With finite elements, we're going to assume a form for the solution t, and approximate the solution. We're going to assume a form for the solution t, and we're going to substitute in and do a bit of mathematical magic, and then solve to find an attribute solution. So it's already quite different to finite differences.

So how do we get started? We start off by-- and let's call this 2a-- we're going to say, to approximate the solution [INAUDIBLE] used a sum of weighted functions. OK. We know that t is a function of x. t varies continuously along the domain here. A function of x. That's an infinite dimensional problem, right? t is a continuous function. Best rather write our approximate solution, which I'm going to call t tilde of x. So this guy here is the approximation of my exact solution, t of x. And I'm going to write it as-- I'm just going to leave a bit of space here-- the sum from i equals 1 to capital N of some ai's times some ci's that are a function of a.

What are these guys here? These things here are no-end functions, they are things that I'm going to specify. And what you'll see is that these things are going to be referred to later on as basis functions. These things are no-end functions of x. So we're going to specify them. The ci's, there's N of them-- capital N of them. And the ai's here, these are unknown [INAUDIBLE].

OK, so what I've said is that t of x, an infinite dimensional problem, t is some continuous, or some function, whether it be continuous, some function of x on the domain. Let's approximate it as a finite sum from i equals 1 to n of some weight, so some coefficient-- ai's-- just constant, times some functions that we're going to specify.

So with-- can you see that we've discretized the problem in a different way than we did in finite differences. We have discretized the problem because now we have how many unknowns? n. We have n degrees of freedom. n [INAUDIBLE] ai's that we can use to create our approximate solution. And I've left a bit a space here because I'm going to write this in. I'm going to write in 100 plus that extension.

Why do you think I wrote the 100? I heard the b word.

AUDIENCE: [INAUDIBLE]

PROFESSOR: The boundary conditions. Yeah. In this particular example that we're considering, the boundary conditions [INAUDIBLE] conditions that specified to be t equal 100 at either end. So let's put the 100 out in front, and then the rest it there, the some of the ai times the ci's is going to satisy 0-- 0 temperature at either end. Right?

OK so in this particular example, this guy-- this 100 is chosen to satisfy a outbound [? preconditions ?] here for our model problem.

OK. So by writing this-- and this sort of approximation of the temperature-- it's a simple equation, but I'm spending a little bit of time because it's really important it's clear in your mind, because this is really kind of one of the-- what's the first key step of making this approximation. We've turned the problem of determining t tilde of x, the approximation, which again is really an infinite dimensional problem, into the problem of determining the coefficients, the a1, a2, up to a n where we have now just capital N unknown. Kevin, yeah?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah, so if you had like 100 here and 200 somewhere else or something? There's various ways that you can handle it. And we'll talk probably in two weeks of times about how to do boundary conditions and the finite element method. But one approach that we could use just with the simple idea here is, you could, for example, have a linear function that went from 100 to 200 and then plus the rest. Right? So you could still get the boundary conditions back. But yeah, we'll talk once we've been through the theory about how to handle boundary conditions more generally. Yep.

Generally speaking we will do something so that those ci's will satisfy 0. [INAUDIBLE].

OK. So now the question that you should be asking yourself is, what are these? I said they were no-end basis functions. They are things that we specify. So you guys have seen this kind of idea, this expanding a solution as a sum of weighted functions. You've seen that before, right?

When have you seen? Fourier series. Good, yes. So-- I mean this is no different really to Fourier series, right? What are the ci's in Fourier series? Sines and cosines of different frequencies. So the idea of decomposing a signal into a sum of harmonic components-- I mean, signal processing-- the entire field of signal processing kind of hinges on that idea.

Here we're not going to use sines and cosines. We're going to use polynomials. So ci of x are going to be polynomials [? in x. ?] In other words, c are going to be constant plus maybe a linear term in x plus maybe a quadratic term in x, and so on. And the reason that we're going to use polynomials is because that sets the groundwork for the finite element method. Finite element method works with polynomials, basic functions. So we're going to start off by thinking about polynomials.

OK? Good? Straightforward, right? Easy.

OK. So let's just be more specific for our particular example of what we want to choose for the ci's. And again, I want to be clear, there are many choices for ci, and this is a decision that you make. So this is [INAUDIBLE] choosing the ci's. So again, here we're go to use polynomials, and that's because we're interested in the finite element method. And you will see on Wednesday exactly what the basis functions look like in finite element. But for now, not yet the finite element, we need basis functions that satisfy the boundary conditions. But because we subtracted it off, or because we added in the 100 here, you can see it's what I was saying earlier, we're going to set these guys now to be 0. So they're going to be 0 at the end of the domain, right?

So if we-- I think we're going to choose polynomials, so what's the simplest polynomial we could choose? We could choose [INAUDIBLE] equal constant. That's not very useful. What's the next one? x plus b. You could use a linear function, but what's the linear function that satisfies this condition?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Zero everywhere. So that's not very useful.

So it turns out that the first one that would give us anything-- what's that?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. The first one that would give us anything interesting would be quadratic, or a parabolic function.

So we'll choose a quadratic function and-- again there is a variety of things we could do-- but here's one-- here's a quadratic function, which may be [INAUDIBLE] well it could be scaled. 1 plus x, 1 minus x, that's a quadratic. So c1, the first basis function in our extension, the function of x, and [INAUDIBLE] is 1 plus x, 1 minus x. That clearly satisfies the boundary conditions, the 0 boundary conditions at either end. Yep.

And then let's also look at cubic. So that's going to be our second one. And there are a variety of cubics we could choose, but let's choose the one that does x, 1 plus x, 1 minus x. And again, clearly if we satisfy [INAUDIBLE] x equal-- if we substitute x equal minus 1, or x equal 1 in there, we're going to get c2 equal to 0. So if we sketched out physically what these guys look like, we sketch c as a function of x, then the quadratic looks something like that, and the cubic looks something like that. So that's c1 of x, and this is c2 of x. Right, that's 0.

OK. So those are the choices we make. And so in this simple example we're going to use only two degrees of freedom to describe the approximate solution. And with those two degrees of freedom, what is our approximation?

Our approximation is t tilde of x being 100, plus that sum, which in this case is going to be a1 times c1 of x, where c1 is 1 plus x, 1 minus x, plus a2 times c2 of x, where c2 is x, 1 plus x, 1 minus x.

OK, so physically we're saying the solution is 100-- so first of all, just shift it up. 100. Then take some amount of a function that looks like this, plus some amount-- some other amount-- of a function that looks like this. Add them together. And now we're going to say, what's the right-- what are the right weightings? What are the a1's and a2's that we should choose so that the function that we get is somehow reasonably a solution of that model problem that looks something like this.

Yep. And that's now what we're going to talk about with numbers 3 and 4. The collocation method and the method of weighted residuals are two different ways that we can come up with the conditions to help us choose a1 and a2 so that in different ways we get a solution that is a good, or just a different kind of an approximation, of the problem we're actually trying to solve.

OK? Yep. All right. Any questions so far, questions based on things that are in the notes that weren't clear? No? All right.

So that's kind of the easy part. So now let's talk about the two different ways that we can come up with conditions that will let us solve for a1 and a2.

And the first one is going to be the collocation method. So number three, collocation. Collocation.

OK. So again, what are we doing here? We're saying for the approximate solution t tilde of x being 100 plus the [? extension ?] [INAUDIBLE] in of ai times the ci of x, where we're, in our simple example, choosing [? n equal 2 ?] and the ci's to be these two guys here, quadratic and the cubic. We need now to determine the ai's. a1, a2, up to an. In this case, just a1 and a2.

OK. So first question, how many conditions do we need to come up with? Two. In general, n conditions. Right? We have n unknowns. We said that over here. We turned our infinite dimensional problem of solving the PDE into a problem with n degrees of freedom. So now we somehow need n conditions. And in this example here where we have just two basic functions, we need two conditions that will let us solve for a1 and a2.

So what is collocation say? Collocation says let's pick n points. We draw my rod back up here from l over 2, minus l over 2, to l over 2. Let's pick some points. In particular, let's take n points, so let's pick two points in the domain, this point and this point. And let's enforce the PDE at those points. Let's make sure that the PDE is satisfied.

Everyone know what the PDE is? d by dx of kdtdx equals minus 2. Let's have the solution satisfy the PDE at those points. OK? And by forcing that-- forcing the PDE to be satisfied with the approximate solution at those two points, we're going to two conditions. Right? Two mathematical conditions. Two equations, two unknowns, we should be able to solve.

So that's the idea with collocation. You guys have seen-- did you guys see collocation in aerodynamics? [INAUDIBLE], yeah. What do you do? You have the collocation point at-- which-- I always forget which way it is-- you have c over 4 and 3c over 4, right? [INAUDIBLE]

AUDIENCE: [INAUDIBLE]

PROFESSOR: At 3, [INAUDIBLE]. So it's the same idea. The collocation point is the point at which you enforce the condition. So it's exactly the same idea.

OK, so how does that work out mathematically? So that's what collcation-- that's what collocation means. We're going to enforce the PDE at n point. OK. How are we going to write that? So let's again write down the example that we're considering, which is the 1d heat equation. And I'll write it in the general form here with the k still in there. d by dx of k dt dx equals to minus q.

OK, so now we're going to define something very important, which is the residual. And this is a really important concept that we're going to use repeatedly all the way through this finite element module.

So what is the residual? We're going to call it capital R. It's going to be a function of our approximate solution, and it's a function of x. And for this 1D heat equation, what's the residual? It's d by dx of [? kdtdx ?] tilde dx plus [? q. ?]

OK, so 1D heat equation, d by dx of of kdtdx equals minus 2. The residual for that equation, which is a function of the approximate solution, t tilde-- so you want to mark that this is your approximate solution to remind yourself-- function of the approximate solution n of x is d by dx of kd t tilde dx plus q.

So does someone want to give me in words, what is the residual? Use some words to describe it.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah, good. So the residual is the amount that the approximate solution doesn't satisfy the PDE. In other words, if we come back to our original PDE, we brought everything to the left hand side. So it would read d by dx of kdtdx plus q equals 0. Then if we take, instead of t, the actual solution, if we take an approximate solution, t tilde, we substituted it in, the residual is how much we get that's different from 0. So it tells us by how much the approximate solution, t tilde, does not satisfy the [INAUDIBLE] equation.

Yes? So someone tell me, how is the residual-- is the residual the same thing as the error? If I define the error to be the difference between the exact solution, t, and the approximate solution t tilde, is the residual the same thing as the error?

Not the same thing. So how is it different?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yes. So let's think about that. So if we define the error to be t minus t tilde, this will be our-- is this thing a function over the whole domain? Is e a function of x?

Yeah. Right. So e is a function of x if it's t of x minus t tilde of x. Yeah?

How about the residual? Is the residual a function of x?

Yeah. So they're actually both-- they're both functions that we could plot-- we could plot over the whole the domain x. We could plot e of x and we could plot r of t tilde, x. So they're both functions that are defined over the whole domain.

But they're measuring different things, right?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. That's exactly right. The error is measuring how wrong is t-- t of x. What's the error in t. But the residual is measuring how far is the equation from being satisfied. What's really amazing about the residual? Yeah?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Exactly right. Which is real-- isn't that cool? And that's kind of-- I mean, I think in a way that's kind of like the amazing breakthrough here, is that yeah we'd love to know the error, but if we knew the error we would know the true solution, so then why would we be doing all of this? The really cool thing about the residual is that we don't need to know the true solution. We can take our approximate solution. We know the equation that should satisfy. And we can measure how far off it is.

Now just because the residual is small, doesn't mean the error is going to be small. And that's a whole kind of field of study. Depending on the PDE that you're studying, for this particular PDE, it turns out the residual and the error will be nicely related. But if you have a nasty PDE, the residual might not be a good indication of what's going on. But it's still a very powerful thing that can give you a sense of how good is your approximate solution without actually knowing the truth. And we're going to use this idea of a residual over and over and over again, because we don't need to know the actual solution.

Sometimes I know the residual can be something that gets people tripped up, because they're kind of-- it's pretty neat, right? Take the approximation, substitute it into the PDE, and see how well you're doing. Not the same thing as the error.

So one thing-- so, no it's not the same as the error, but one thing we do know is that what happens if I substitute an exact solution here, residual is 0 everywhere. Yeah. OK.

All right. So how is that going to help us with our collocation method? So we defined the residual. And what we're going to do-- you can see now what we're going to do with the collocation method, is when I said that we enforce the PDE, in point what we're going to do is we're going to set the residual to be 0 at n point.

So in other words we're going to pick n. In our case 2 values of x. Right? So remember we said that the residual was a function of x, something? We're going to pick two values of x. We're going to set the residual to be 0 at those particular points. Pin them down. And that's going to give us the two conditions that we need to solve for the two coefficients, a1 and a2, that are multiplying our basic function.

OK? That logic is clear? Yep. So we can just work through the math that actually does this. And I won't go in too much detail, but if you want me to back up and do any more of the detail, can do.

So in our example, remember we set k to be 1 and q was 50e to the x. So this is a situation where we do have the exact solution but we don't want to use it. And again, just to remind you that our approximate solution is this expansion that looks like 100 plus some amount, a1 times our quadratic basic function, c1, plus some amount a2, times our cubic basis function, c2. And actually let me write that out. It's 100 plus a1, 1 plus x, 1 minus x, plus a to x, 1 plus x, 1 minus x.

OK. So here's our residual. It's d by dx-- the k is just 1, so it's actually second derivative of the approximate solution plus q. So we're going to need a second derivative of our approximate solution with respect to x. So the 100 is going to go away from this guy. What am I going to get? Only the quadratic terms are going to stick around, right? I'm going to get a minus x squared multiplying a1. So when I differentiate that-- where I differentiate that twice, I'm going to get minus [? 2a1. ?] And then from this guy here I'm going to have the cubic term-- is that me beeping? Oh that's you. OK. It's your shoe beeping?

AUDIENCE: [INAUDIBLE].

PROFESSOR: OK. All right.

Cubic term. We're going to have minus x cubed times a a2. So we're going to get minus [? 6a2 ?] times x coming out, and then the quadratic terms actually drop away in that last one. OK. So there's the second derivative of the approximate solution. And you know, again, we've still got these constants hanging around, because we don't know what they are yet. So we can substitute that in to our expression for the residual, which again is a function of our approximate solution and x. And it's just-- that second derivative is 1 plus q. So it's going to be minus 2a 1, minus 6a2 times x, plus q, which is 50e to the x.

OK. So there's the expression for the residual for this problem.

So you look at this and you immediately see, well, there's no way the residual can be 0 everywhere, right? There's no way that this linear term-- no matter what we choose for a1 and a2, there's no way we can make it equal to 50x to the x for all values of x between minus 1 and 1. So that immediately tells you that those two basis functions we used are not rich enough to describe the solution exactly, which is not really a surprise, right?

But now the question is, what are good choices that we can make for a1 and a2 so that we get a decent solution? And again, what are we going to do? With the collocation method we're going to pick a couple of points for x. And we're going to set the residual to 0 at those points. That's going to give us the condition. And then we'll solve for a1 and a2.

So-- yep?

AUDIENCE: [INAUDIBLE]

PROFESSOR: So if the source was 0-- if q were equal to 0, and this were the residual, what would you choose for a1 and a2? You could put them to be 0. So you could make the residual 0 everywhere. What would then-- what would our approximate solution be then?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah, t equal 100. So if you think about the physical problem, if you had 0 heat source and you pin the temperature to be 100 and 100 on either end, that would be the solution. So there's an example of a source where-- I mean, it's kind of a trivial solution but-- of a source where you could get the exact solution.

Yeah?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. So the question is, if we have a q that basically we can't do the integration by hand, what would we do? That's a very good question, and we're going to talk about that, too.

You're going to see some-- in fact you've already seen, probably in the pre-reading, some of the integrals that show up with two. If you can't integrate analytically there's something called Gaussian quadrature. So quadrature is a way to numerically do integrals that are basically too hard to do analytically. So that's going to be in the pre-reading that's due next Monday, and probably [INAUDIBLE] will be covering it in class I would say probably on Monday. So there are ways that basically we can numerically integrate things that we can't do by hand.

PROFESSOR: Yep?.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. So in that case, is there's going to be-- what you'll see is, when we actually get to talking about the finite element method, that there's a bunch of integrals we have to do. By using quadrature you're going to actually end up having to approximate the integrals, but be doing integrals over sort of small elements. But yes, the quadrature is going to introduce a little bit of additional error. Yeah. Yeah.

OK. So we have the expression for the residual. And so now what we're going to do is pick n equal [? two ?] collocation points. So again, just to remind you, collocation method says enforce the PDE at n equals two points. And enforcing the PDE, what does that mean? Enforce the PDE means i.e. set the residual equal to 0 at two-- and by points we mean two values of x-- two points in the domain. And here's our domain. There's x. Remember, we're going from-- we've got [INAUDIBLE] so it goes from minus 1 to 1. So there's 0 in the middle. And the collocation points we're going to choose there and there. That's minus one third and plus on third. And those are the collocation points that spread things out the most, kind of away from putting up the boundary conditions here at minus 1 and 1.

So [INAUDIBLE] that we don't really use the collocation method. It's not a good idea. One of the reasons we're doing it is because it's a good stepping stone to see how the weighted residual works. But that's a good question, because in some cases people do use collocation, and aerodynamics is a good example. And the reason you use the 3/4 quad point is because for a particular kind of left distribution, that integrates you exactly. And in fact, your question is somewhat related to the question about numerical integration. When we see quadrature, you're going to see it's the same-- it's going to be different settings, I don't want to confuse you about quadrature-- when you do numerical integration, we're going to put points in the integration domain. And those points are chosen in a particular way so that we can integrate certain functions exactly.

So it's-- here they've chosen to be spaced out-- there's actually all these rules, they're called quadrature rules, and they're all these different kinds of points, and they actually tend to be named after mathematicians who discovered them-- different patterns of points where it says, space things evenly, or distribute them out. That's actually a pretty interesting area of study. In fact, Alex I would say knows a lot more about these things than I do. Is that true, Alex? What's your favorite set of points?

AUDIENCE: [INAUDIBLE].

PROFESSOR: [INAUDIBLE]?

So depending on the function you're trying to integrate, there are sort of optimal choices of where you might put the point.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah, something about the solution or something about the basis function that you're using to represent the solution.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. So that's a really good--

AUDIENCE: [INAUDIBLE].

PROFESSOR: --that's a really good question.

So if you're trying to approximate things, but you know there's something in the middle that you would really care about-- so now what you're talking about is an adaptive strategy. And again, this is a very current area of research. In fact, a lot of [? professor ?] [INAUDIBLE] research [? facilities ?] I think mention there at the beginning that uses adjoint methods. So that would be bringing in information that tells you where the regions in the domain that are most sensitive to what it is you care about, where might you put more points. People who choose points to do this quadrature, or this numerical integration, sometimes will choose to adapt in these ways. And what you would like is, you would like kind of general rules that work well, but then you would like to be able to adapt depending on your particular problem of interest. And there are definitely a lot of adaptive strategies out there, but it's also an area that a lot of people are working on.

Good question.

So you guys are all ready for the lecture on quadrature. In fact, when we come to talk about Monte Carlo integration-- Monte Carlo methods-- I mean, evaluating means and variances. That's just integration as well. That's all just about putting points in a domain. And what you're going to see is Monte Carlo does it randomly, but in some cases you can do better by putting them in a very spec-- putting the points in a very specific way.

OK. But here we're going to do just [INAUDIBLE] we're in 1D, so, in 1D you can usually get away with pretty much anything.

We're going to just put them at minus a third and a third. So what that is, again, [? sit ?] the residual. And I'm purposely writing out all the functional dependencies. I keep writing r of t tilde comma x because it helps me remember that residual is a function of x. And so what I'm doing here is I'm evaluating the residual at the point x equal to minus one third, that's a point on my domain. So that means substitute x equal minus one third into my expression for the residual, which gives this 50e to the minus one third, and put that equal to 0.

Right? So there's the first condition. And then the second condition is original evaluated plus one third. Again, substitute that in to the expression. So we're just changing the sign here. And then this is going to be 50e to the plus one third. And that's been set equal to 0.

So now you can see we reduced-- we keep sort of reducing the problem. The first thing we did was assume the functions-- these guys-- that we want to approximate the solution in. Then we take-- we've got two degrees of freedom to unknown, so let's use a collocation method. We set the residual to be 0 at two points. We chose these two points. We've enforced these conditions. Now we have two equations, two unknowns. We could solve that-- you could probably solve that analytically, or using mathematical, or whatever you like. And what you find is that a1 and a2-- 26.4 and 8.5.

So those numbers don't mean a whole lot to us, but again, what's the key? The key is that this is our approximation of the solution. So we're saying that by the collocation method, the approximate solution of t tilde is 100 plus 26.4 times something that looks like that-- a quadtratic-- minus plus 8.5 times something that looks like that [? cubed. ?] And when you add those together you get something that is at least an approximate solution to the PDE.

So I want to take a look at what that solution looks like. While I put the projector on, are there questions about the collocation method? If we were actually using collocation method, we would use more than two points typically. My simple example.

So-- so I have a code here that actually implements-- I guess I should plug my laptop in.

OK. So the code here that's going to implement it, there are the two basis functions, c1 is 1 minus x, 1 plus x. c2 is x times that-- the cubic. And the collocation method just gets implemented here. It's kind of hardwired because all the derivations have been done-- sort of done offline.

But let's just run this. Won't run that yet.

OK. So one of the parts that we're looking at here-- so here first, of all, is a plot of the temperature versus x. And the solid blue is the exact line, that's the one that we computed analytically for this problem, we can do that. And a dashed line is what we got with collocation.

OK, so it actually doesn't do too badly, right? We sort of assumed this [INAUDIBLE] of the solution with the solution being 100 plus a quadratic plus a cubic. We figured out just with two degrees of freedom what good choices for a1 and a2 would be. It's actually not too bad. Right?

So there's a plot of the actual compared to the collocation's approximate solution.

Here's a plot of the error. So that's t minus t tilde. We can compute again in this case because we happen to have the exact solution. You can see that error is 0 on either end. And you could see from the previous part that the exact solution-- the approximate solution was always under-predicting in temperature [INAUDIBLE] to be exact. So you see that with the error being negative. OK?

Now for this problem we can look at these because we do actually know what the exact solution is. We're not going to be able to do that in general. But what we could plot is the residual. So again, what is this? This is r of t tilde comma x. And now we're taking the t tilde that we determined. We specified the form of it. We chose a1 and a2. And remember, it's a function of x. So here's the residual plotted out as a function of x. And now you should be able to see, this is where you can make sure you did things correctly. This guy should be 0 where? Here at minus a third, and here at plus a third. The two points at which we asked the residual will be 0, and then everywhere else it ends up being non-zero. Yep.

OK. So that is it for collocation method. Any questions? Good? It's clear?

All right. So now we're going to talk about the weighted residuals method-- the method of weighted residuals. And this is really the method that we want to focus in on because this is going to be the launching point for finite elements.

But again, we were talking about collocation just sort of as a way for you to start thinking about what it means to approximate a solution with a finite number of basis functions, and also to think about there being different ways for you to actually come up with the coefficients.

Yeah, [? Libby? ?]

AUDIENCE: [INAUDIBLE]

PROFESSOR: Maybe start with the error. You have to deal with the fact that error is 0 at the end point.

AUDIENCE: [INAUDIBLE]

PROFESSOR: So let's try to put them side by side.

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. Yeah, that's exactly right.

So if we start with the error, shold the error-- it's warming up-- yeah?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. By construction the error is 0, right? Because we constructed the basis functions to be 0 at either end. So no matter what value we choose for a1 and a2, the approximate solution will be exact at the boundary condition.

What about the residual?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. Say it a bit louder.

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. So it's specifically what do you think is not what it should be at the end point. So I'll put this up [INAUDIBLE].

So derivatives. Yeah, exactly right. So it basically is telling you that-- and you can kind of-- you can see it physically in the shape, but you can also see it mathematically because it has this form. When you take a derivative, as soon as you move away a little bit, things are going to go astray because second derivative is not right. and at the endpoint, the derivatives are not what they should be.

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah, it's not-- would only-- no, come on, [INAUDIBLE] turn it off.

We've only got two degrees of freedom, right? We see the solution is a constant plus a quadratic term plus the cubic term. We've only got two degrees of freedom. So you only get to pick two things, right, to put them down. And we said, let's make a residual be 0 at minus a third and plus a third. I guess we could have chosen to make a residual 0 at the two end points. I don't know what solution-- maybe Alex can quickly figure out in his head what solution that would have been.

That would be kind of a bizarre choice, but I'm not-- I'm not sure is that-- I'm just wondering if that might give a 0 solution everywhere. But you can't you can't kind of have everything. You only got-- we only got the residual at the minus one third and the plus one third collocation point.

There seems to be a lag, a big lag, in the projector-- there we go, get it off.

OK. So any other questions about collocation?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah, so that's a good question. If you look at the residual could you tell whether you're using too low of an approximation? In other words, can you tell by looking at the residual how bad the error might be in the solution? And the answer is, yes. And there's some beautiful theory-- again, it's going to depend on PDE that you're talking about-- this theory that basically says that it relates the norm of the residual-- how big is the residual integrated over the domain-- compared to how big is the error integrated over the domain. And they're related by a constant, which depends on the properties of the PDE you're solving.

And that comes back to what I was talking about earlier, that for a nice PDE like this 1D diffusion, it actually turns out that looking at the residual is a really good way to understand what would be happening with the error. For nastier PDEs, that's not so much.

But what you're sort of asking about is to a lot of the theory of the error analysis in the finite element method-- that people do a lot of very rigorous analysis of error based on being able to compute the residual. And in fact, the residuals also can then start being an indicator for figuring out how to do things like [INAUDIBLE] refinement, and all these things are kind of related. And then the theory with adjoint methods.

So if you're interested in that stuff you have to take [? 16920 ?], which Professor [? Wong ?] will be teaching in the fall, which is the grad class on numerical methods of PDEs. I don't know how much you get into that stuff.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Little bit more than here. But you have to take that class so that then you can take [? 16930 ?] which really would get you into it. Right? Yeah.

OK. That's good. That's great questions.

So now, if you guys are ready, we can talk about the method of weighted residuals, which hopefully won't be too much of a jump from where we are, but it is a different philosophy. So everything is the same up until the point that we define this residual. But where we're going to kind of take a fork in the road is, now we're going to use a different strategy for coming up with the two conditions to figure out what those two degrees of freedom are.

OK? So same idea-- approximating the solution with the basis functions and [? the extension ?]. But now what's different is, how do we come up with the conditions to determine the functions a1 and a2.

And-- so-- method of weighted residuals.

OK. So in order to move forward-- in other words, in order to find these two conditions, we're going to define something else. We're going to define what's called a weighted residual.

OK? So we defined the residual-- it still sits over here-- which is the thing that we get when we substitute the approximate solution into the PDE. Now I'm going to define the weighted residual, and I'm going to call it-- and actually, I was realizing when I was going through that the notation in the notes jumped around a little bit-- so let's call this capital R sub i. It's going to be a function of the approximate solution t tilde, but not a function of x because we're going to integrate things out of x.

So we're going to define it as the integral over the domain. And again, the domain in our simple problem is just x from minus 1 to 1. We're going to take a [? weighting ?] function, which we'll write as wi, it's a function of x. And we're going to take our residual-- you know what, I'm going to give this a little r just to make sure that they're different. And that's going to be integrated over ex. So again, little r sub i is going to be our i'th weighted residual. And what you're going to see is that we're going to need n of these things. So again, we're looking for n conditions to find out n, a non-coefficient, and it's defined to be the integral of a weighting function, which is our wi, times this guy here, which is our residual. And we're integrating over the domain. Here's x going from minus 1 to 1. OK. So residual is a function of x, weighted residual is not a function of x because I've integrated over, and the integration is weighted by some weighting function wi. But the weight of residual is still a function of our approximate solution t tilde.

OK. So now what does the method of weighted residuals do? It says let's choose n different weighting functions, w1, w2, up to wn. Let's define the n corresponding weighted residuals, and let's set each of those weighted residuals equal to 0. Each time we set a weighted residual to 0 we're going to get one condition, right? So we'll do that n times with n different weighting functions, and that will give us the n conditions we need to compute our n a coefficients.

Yes? Seems like kind of a bizarre thing to them. It's OK? You guys are so quiet. Does that mean that you think this is very easy, or kind of weird? We don't know.

So let's write it out a little bit and see. So again, the method of weighted residuals-- we're going to require n weighted residuals to be 0. So that means we're going to have to choose n weighting function. Those are the w1, w2, up to wn. They're all functions of x. And we're going to then get, again, n equations to determine these coefficients, a1, a2, to a n.

OK. So first thing we need to do is choose n weighting function. So this is the first question, is now what do we choose for the wi's? What do we choose here?

So the answer is that there is a variety of things we can choose, but there's a very special choice that results in what's called a Galerkin method. What does the Galerkin method say to do for the weighting functions? You guys read about this.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. Exactly right. Let the weighting function be the same basis function that you're using to approximate the solution. OK? So the same [INAUDIBLE], that we used-- we've erased it now-- but that we used in our in our expansion of t tilde-- choose the same basis function to be the weighting functions that you use to define your weighted residuals.

Turns out don't have to do that. There's a bunch of other methods. If [INAUDIBLE] Galerkin methods, we can choose different weighting functions. But we're only in this class going to consider Galerkin methods where you choose the weighting functions to be the same basis functions that we're using to approximate the solution.

So the Galerkin method is a very special choice that says choose wj to be equal to [? cj. ?] And it turns out that this choice has really good properties for some PDEs. And again, if you take [? 16920 ?] with Professor [? Wong ?] in the fall, you would see how this plays out from an energy perspective.

OK. So this just means the same functions that you use to approximate the solution, they're going to be used to weight the residuals.

OK. So for our example, what does that mean? That means-- coming back to what we were doing before-- that means that the first weighting function would be our c1, which was 1 minus x, 1 plus x. And the second weighting function would be the cubic, which is x times all of that. And then the first weighted residual, 1, which is a function of t tilde, would be the integral from minus 1 to 1 of w1 of x times r of t, xdx.

And the second weighted residual would be the second weighting function multiplied by the residual integrated over x from minus 1 to 1.

OK. I don't want to work through all the math on the board. But basically you can see what's going to happen here, right? We're going to substitute in w1 as 1 minus x, 1 plus x. We have the expression for this guy that we derived before that was minus 2a1 minus 6a2x plus 50 e to the x. So now at that point it's just multiplying things together and integrating. And you could do it all by hand for this example, or you could again throw it into [? Mathematica ?] or something and get it out.

And then what are we going to do? We're going to set our 1 equal to 0, and we're going to set our 2 equal to 0. It's going to give us two conditions, two unknowns. Solve those. And we're going to come out with, in this case, a solution that says a1 is 27.6 and a2 is 8.9.

OK, so I skipped over a lot of messy math there, but I don't think there's anything conceptually difficult. Two conditions now coming from setting a weighted residuals equal to 0, where the weighted residuals correspond to taking the residual, multiplying it by the basis function, the Galerkin, integrating over the domain. Plugging through all the math, solving the equation, gets us this solution which says that by the method of weighted residuals for this choice of the basis function-- the cubic and the quadtratic-- in a Galerkin method the solution we get is-- the approximate solution is 100 plus 27.6 amount of our quadratic basis function, plus 8.9 amount of our cubic basis function.

So the different solution to what we got with collocation, right? We got to it through a different route.

So I'll follow it up, and we can look at that solution and we can compare it to a collocation, but first, questions about method weighted residuals. Does that make sense?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. So that's a good-- that's a good question. The answer is going to be, it's going to depend on the problem. So there's no general guidelines that I can give you.

[AUDIO OUT] you're [INAUDIBLE] exactly right is that, because the-- what came out of the collocation method, and we'll see it in a second-- the residual was-- I forget whether it was positive or negative everywhere, but it always had one sign. And we're going to see that the method of weighted residuals actually kind of balances out. Now that's going to make sense, because what are we doing when we-- when we take the residual, and we weight it, and then we integrated it. I think of this as kind of like-- it's kind of like we're asking the residual to be 0 on average over the domain, but on average weighted by w1. Right? Because if this were gone and I just integrated it and set it equal to 0, it would be like saying [INAUDIBLE] the residual on average 0 over the domain. So it's like taking an average, but weighting it by w1.

So yeah, I'm going to expect this to come out with something that's going to be more evenly distributed, rather than with collocation where had all of one sign. But there's not-- I mean you want to be careful about-- I think you want to be careful because it's going to depend on the problem.

You'll see when I [INAUDIBLE] in a second that it actually turns out our residual will be 0 at a particular point x. Not because we set it that way, but because that's how it comes out. So another [INAUDIBLE] check would be to go back and plug that point in and make sure that the residual-- actually worried about getting the integration wrong. Take that point, x, plug it in, and make sure that indeed the 0 is satisfied.

But I said earlier we don't really use the collocation method, but we do use method of weighted residuals, that it's going to be, because it actually turns out to have really great properties. It wins for this problem, and that's not really a coincidence. And in fact it forms the basis of what we're going to do with the finite element method, which is incredibly powerful.

OK. So I am going to run the same script that I ran before. But now I also have the implementation of the weighted residual method.

So here are-- so this is basically, this is the combination of all those integrals over there-- amounts of the [INAUDIBLE] matrix-- this k here, this matrix, and the coefficients being [INAUDIBLE] times b. So all the integrations amount to that.

And so we're going to run it and plot. So let me-- clear all, close all--

OK. So just like we looked at before, we can look at the temperature as a function of x. Solid lines, just like before, is the exact solution. Dash line is the method of weighted residual solutions.

So remember, we've got the same degrees of freedom as we had for collocation, right? Still any two degrees of freedom. And our same basis functions. The only thing we've changed is the way we choose how much of each basis function to put in. And you can kind of see visually-- I'll put them both up together-- you can see visually it does pretty well. And actually what Kevin was just observing, it's not always under. It actually is under a little bit here, and over a little bit there, and in under a little bit here. It's kind of just the way it came out for this problem.

So there's the temperature compared to the exact solution. We can look at the error as a function of x again because we have the exact solution here. And again we can see it's negative, then it's positive, then it's negative. And we can look at the residual.

And so now, remember with collocation, what did we see? We saw with the collocation method that the residual was 0 at the collocation point, because those were the conditions that we imposed to get the coefficient.

Here it turns out that the residual is 0 minus 0.4 and at plus 0.5. But that just kind of fell out. And again, if you're looking for check to make sure you did the integrations right, you can always take your residual, substitute in whatever these values are, and make it came out to 0. But again, that just fell out. What we actually ask for was that this residual, when weighted by this guy and an integrator over the domain, is 0. And this residual, when weighted by this guy-- the cubic-- an integrator of the domain-- that is 0.

And so then finally-- let's see-- plot the comparisons. So I put them all on the same plot. So there in black is the exact solution. The red is the method of weighted residuals. The blue is collocation.

Turns out method of weighted residuals did better for this problem, but again we want to be careful about making general assumptions. They both have the same number of degrees of freedom. They only differ in the choice of those coefficients.

Here's a plot of the difference errors. And again, I would say the method of weighted individuals is kind of more balanced, again, because we are asking for this weighted residual to be zero. And then lastly, here are the plots of the residuals as a function of x. And now you can see clearly what I was saying. There's the collocation [INAUDIBLE] to 0 at the collation point. The method of weighted residuals happens to be 0 at a couple of other points.

OK? More questions.

I'd say it's really important that you feel very comfortable with the method of weighted residuals. The idea-- and let me just say it again, the idea that we're going to take the PDE-- we're not going to mess around with the derivatives, we're going to instead say, let's approximate the solution and an expansion with a finite number of basis function. We're going to choose the form of the basis functions-- and they were even talking about polynomial basis function.

Now we need to figure out a way to determine the coefficients-- how much of each basis function. And so the way we're going to do that is choose weighting functions, and with Galerkin methods we'll choose those weighting functions to be the same basis function that we're approximating the solution with. We'll weight the residual with the weighting function, integrate it over the domain, define the weighted residuals, set those equal to 0, and get the conditions we need to find the coefficients on our expansion. All those sets.

And what we've done today is define global basis functions, right? So my c1 was a quadratic that varied over the whole domain, and my c2 was a cubic that varied over the whole domain.

Finite element method-- you're all ready for it now-- is just a, I guess kind of a simple, but really pretty huge next step, which is to say, let's not do this on the whole domain, let's divide domain up into little pieces, and let's use this idea. Let's define basis functions-- special basis functions that are polynomials just on the little pieces. And then we're going to have coefficients that go with those.

Yeah, [? Tran? ?]

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. So that's a good-- [INAUDIBLE] In that case, the weighted residual-- we're going to see all that on Wednesday.

So what's going to happen now is that this weighted residual is actually going to have very special structure, because the way we're going to define the basis functions is that they're going to be local on elements-- the finite elements of the finite element method. And so when we do the integration, a whole bunch of it's going to disappear, and we're going to be integrating locally just over the elements. But what you're going to see is that there ends up being a little bit of interaction between one element and its neighbors because of the way the basic functions are going to come out.

So we'll work through all those integrals. I think you read maybe a little bit of it already. How far did you guys get in the reading? You saw the linear, [INAUDIBLE]?

Yeah. Yeah, we're going to work through all of that. And you'll see that's what's incredibly powerful with the finite element method, is this idea of using a polynomial basis and a local element, we're going to get these integrals with so much structure that we can handle-- it's going to sort of all work out in a really neat way.

AUDIENCE: [INAUDIBLE]

PROFESSOR: That's exactly right. We're still integrating a weighted residual over the whole domain, but a whole lot of it's going to go away, and it's going to turn out that we get sort of these special patterns that show up because of the way which is a basis functions. And you'll see all of that. We'll do all of that. I think on Wednesday we'll sort of work through all the steps of the finite element method and have you go along and do it, so actually bring your laptops if you can so you can do some stuff on Matlab.

OK. Yeah, Ben?

AUDIENCE: [INAUDIBLE]

PROFESSOR: You know, that's a good question.

AUDIENCE: [INAUDIBLE]

PROFESSOR: That's a good question. The question is, is the fact that the method of weighted residuals, the resulting residual crosses 0 in time, is that a coincidence, or is it always going to happen?

I think that's a very interesting question, and I'm actually not sure, because I was wondering actually, is there a way to figure out where we could have put the collocation points for this problem. We could have put the collocation points in a particular point and gotten the same answer that we got with the method of weighted residuals. And it's going to have something to do with the 50e to the x that's in the [INAUDIBLE] function. I just think the fact that it crosses twice is not a coincidence. I think it probably has to cross twice, but I don't--

AUDIENCE: [INAUDIBLE]

PROFESSOR: You feel like it's kind of like a fundamental theorem?

AUDIENCE: [INAUDIBLE]

PROFESSOR: Maybe it's a great question for the final. Really good final questions are when the professors don't know the answer What do you think, [? Xixi.? ?]

AUDIENCE: [INAUDIBLE]

PROFESSOR: Yeah. Well the thing is, we have an expression for the residual, which we wrote out, which was-- what's the expression for our residuall-- it's minus 2a1 minus 6a2x plus 50e to the x. So I guess you can figure out how many crossings are possible because of the degrees of freedom that we have.

It actually turns out that even though things seem like coincidences, there are almost no coincidences. In these kinds of problems, it's usually that there's so much structure in the problem that you could have-- but-- yeah.

OK. If you have-- if there are parts of the method of weighted residuals that are a little bi uncomfortable for you, I would strongly suggest you talk to me, or to Alex, [? Xixi, ?] or to [? Bikram ?], before Wednesday, because otherwise things are going to get dramatically worse for you on Wednesday. Make sure that this-- really-- make sure-- that's why I went kind of slowly through this lecture, is because it's really important that you have all these steps kind of clear in your mind.

And please if you have a chance to even just listen to a few minutes, especially those of you who said you like the audio recording-- if the combination of what's on the screen and me talking is good enough then I can keep using the blackboard, but if not please let me know.