Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Description: This lecture covers the sources of error when solving an ODE and introduces accuracy, including forward Euler and midpoint rule. The lecture introduces a 'contest' in which students work in pairs to solve the local order of accuracy.
Instructor: Qiqi Wang
The recording quality of this video is the best available from the source.
Session 2: Numerical Integr...
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR: All right, let's get started. So here, as I said, I assume that everybody [INAUDIBLE] has already done the reading. OK, what I'm going to do today is not [INAUDIBLE] repeat any of the things you have already read.
But I also don't assume you already mastered all the material [INAUDIBLE]. So what I'm going to do is kind of [INAUDIBLE] to give you a view of more or less the same material, maybe a little bit more. But from a slightly different angle. And which means we are going to [INAUDIBLE] but like [INAUDIBLE] may look like this one. The syntax may be different and may denote things with a different symbol.
But, like, any [INAUDIBLE] is the same and you should be able to read the style. You should read what I'm going to be writing. And in addition to that, I'm going to ask you to work out some problems for yourself or actually in teams based on what you read and what I'm going to be discussing today.
So I brought these things. I'm going to be actually [INAUDIBLE]. And I also brought [INAUDIBLE] each of you are going to be working [INAUDIBLE]. OK, and [INAUDIBLE] say which is the most active [INAUDIBLE].
OK, to start this, I'm going to be-- wait, OK. I'm going to be briefly reviewing what you read, which is how to basically write the function. And then in addition to writing down formulas, I'm going to be demonstrating things in MATLAB, OK? Discretize a function-- let's start with a function that, like, most of you very commonly see [INAUDIBLE] ODEs.
When you write down f-- let me change to a different color. OK, if you write down F of T equal to E to the minus lambda T-- OK, can somebody tell me why this function is commonly ODEs? It's a solution to essentially the simplest ODE you can find in the world-- df dt equal to what?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Minus lambda times f of t, right? So if I solve this ordinary differential equation, I'm going to get this function. OK, if I draw the function, if I draw this function as a function of time, at t equal to 0, the function has a value of 1, right? This is f of t, this is t.
And how does the function look like for positive lambda? It's going to decay exponentially. That is what we call exponential decay to the minus lambda t, right? OK, this function is a function. It is a continuous function, which means it is, in some sense, infinite dimensional. You need infinitely many numbers or infinitely much memory to memorize this function in a computer, right?
Because for every t, every single t, you have a function value f of p. But computers have finite memory, although we have a lot of memory, but still finite. So we need to find a way to represent this function using a finite number of bits in a computer. How do we do that?
OK, we do that by first selecting a range. We cannot discretize the function for infinite t. We have to select a range and let's say from 0 to some big t.
Is that enough? No we also need to discretize this interval into small time steps. And the first time step is usually just 0. The second time step is delta t and the 2 delta t, 3 delta t, 4 delta t, et cetera.
And at t is the last time step. All right, so this discretization, when I do this thing in MATLAB-- let me actually start up MATLAB. And when I do it in MATLAB, it is usually how we actually draw the function in MATLAB, right?
I believe a lot of you have done this before in some other classes. Here, I'm just trying to warm you up and get you, if you're not familiar, get you familiar again with how do we do things in computer. I think I started MATLAB.
Right, so OK. So let me set a big T equal to 5. You see this? Is it big enough? Shall I change it to a little bit bigger?
Anybody want it bigger? No? Good? OK.
What [INAUDIBLE] do you want? What one? Good, OK.
And what I want-- how I discretize the time is t equal 0 dt t, all right? This is going to give me a t 1 by 51 double. So if I click on it, it is going to show that the first one is 0, the second one is 0.1, et cetera, et cetera. And the last one should be-- whoops-- is 5, all right?
And now the function value f of t is equal to any set of lambda of equal to 1, OK? Lambda equal to 1. And then my function value is equal to exponential of minus lambda times t. So that way, I computed discretized version of the function f.
How would I show the function? I can go to plot and to do this plot. Like here, I'm just going to use the command line version to log t and f. Yeah, this gives us exactly the function we expect, right? Exponentially decaying function.
And I would actually also prefer to plot this and look at what kind of discretization it is. So if I plot this function with a dash and the o, it is going to display a circle at every discretized point so that we are going to see that this function is actually a discretized function. It is not a continuous.
In MATLAB, the plot is approximated by drawing a linear line, drawing a line, between these circles. But because delta t is small, 1.8, it looks like a small function, all right? OK, let's try another one.
So here-- let's see. How do I insert another? Each option insert [INAUDIBLE] insert. OK, so let's try another function. We did df dt equal to minus lambda u.
How about minus lambda f? How about minus ft squared? Anybody know how to solve this ODE analytically?
Anybody who reviews ODE before this class where you are reading the readings? The original variables? Good, we are going to be dividing f square on each side, right?
So we have df over-- not on this-- f square equal to minus dt, right? We're going to be integrating both sides. And on this side, we get minus f 1 over f plus an arbitrary constant equal to minus t, right?
So we are going to get f of t is going to be equal to 1 over-- we have t-- here is actually plus t, right? Agree? OK, in particular, if I set the same initial condition, which is f of 0 equal to 1, then it determines my constant c and my f of t is equal to 1 over, what?
One over t, right? OK, so let's Test this function. And it is going to be useful later on, right?
So my f2 is going to be 1 over t. OK, I'm going to be-- let me hold on and plot t f2 [INAUDIBLE]. Oh, OK.
That's one, right? I'm going to have serious trouble if I do this. So let me just do this again-- equal to 1 divided by t plus 1.
Yes, thank you for paying attention, OK? And plot t f2. Let me do dash dash and s. So, dash dash means the [INAUDIBLE] line and s means squares.
So we are going to see how different these are going to be. So, one function is the solution of df dt equal to minus f. The other is df dt equal to minus f square, right?
OK, so here is discretization function. But like, this is nothing new, right? We all know this.
What's new is how do we approximate the derivative of these functions. Because if we can approximate the derivative of these function, then we can start to solve these ODEs numerically, right? These two ODEs I just showed you have exact solutions.
But the reason we take this class is because we want to solve ODEs that do not have analytical solutions. If we have a way to approximate the derivative of any function we want, then I'm going to show you that we can solve any differential equation we want. OK, so approximated derivative-- not from analytical function, but from a function discretized like what we just had.
Right, so pretend we have MATLAB. We have this app. We have f2 but we don't know where they're computed from. They are just some kind of discretized function. Now, how do we compute the derivatives?
OK, any ideas? Any ideas? Any ideas?
Any ideas? Any ideas? The linear definition of derivative-- what is the linear definition of derivative? f2 minus f1 over f1-- There is a [INAUDIBLE].
OK, you did, like, partial t's. Anybody wants to [INAUDIBLE] what's your name? [INAUDIBLE] OK, anybody wants to complete this [INAUDIBLE] answer by maybe changing something?
[INAUDIBLE] is f of t plus delta t minus f of t divided by delta t, right? And this is, of course, the definition of exact derivatives. But in the computer, we cannot take delta t and go to zero, right? That would again require infinite memory and infinite computation time.
We have to approximate it. We have to be satisfied with delta t being small enough. OK, so here, we have a function like this.
We have a function like this. Each-- let's look at this one, the one with squares. These two squares is based over delta t in the horizontal side.
On the vertical side, they are spaced by f2 minus f1, right? So if we take the vertical distance, divide by the horizontal distance, that [INAUDIBLE] approximation of this, right? f of t plus delta t minus of of t over delta t, all right?
So that's a good way of approximating the derivative. Now, anybody who can transform this formula into MATLAB? I have two functions, f and f2, in MATLAB. How do you guys transform this formula, which is-- this is like one of the most important skills you are going to learn in this class-- that is, how to incorporate, how to translate, a mathematical formula you can write down on a piece of paper into a code like MATLAB. [INAUDIBLE] see [INAUDIBLE] is that going to keep me a code I can type into MATLAB.
Like, here, I'm writing [INAUDIBLE] type. Please tell me what to type. Yeah, find the function-- the difference between functions between one point and other points. Let me do a [INAUDIBLE].
Yes, the difference and the approximate derivative-- OK. OK, you guys are good. So OK, so I'm going to do something new. I'm going to use if.
I'm going to use if of f. What is that going to give me? So let me just say df equal to diff f, OK?
And let me double click on df to show me what this is. And I'm going to see, compare this with f-- f and df. So what I can see is that df gives me this minus this, right?
The first element of df is the second element of f minus the first element-- sorry, this minus this. Or what I can do is the same thing can be computed by f of 2 to end. Do you know what this means? Exactly-- it gives me all the angles from the second to the end minus f1, 2, and minus 1.
What will this give me? The first to the second last-- so this gets me exactly the same answer, right? And of course, I can just do this divided by et, all right?
And so this df dt. This is an approximation of the derivative. OK, and this is an approximation of the derivative at which point, if I speak to this formula and the discrete time instances instances, but like [INAUDIBLE]. So if look at length of df dt--
AUDIENCE: [INAUDIBLE]
PROFESSOR: Yes, this is an approximation to the derivative from 0 to t minus delta t, right? Because here, I'm taking the difference between t plus delta t, which goes all the way to big t, minus ft, which actually doesn't go all the way to big t, right? So I have the derivative from 0 all the way to big t minus delta t. I don't have the derivative of the last one because at the last one, I do not have f of t plus delta t.
OK, now let me plot-- let me make another figure. I'm going to plot t1 1 to n minus 1 because I don't have the derivative at the very end as df dt. I'm going to plot them using circles. And it's the derivative-- yeah, this is the derivative.
It goes all the way to close minus 1 to 0. And you can see it is roughly an approximation of the slope of the circle line, right? The slope is negative. Therefore, the derivative is negative.
Let's compare this against the real derivative. Now, what is the real derivative of this function? The real derivative of this function if f of t is equal to minus-- e to the minus t, df dt is just equal to negative of e to the minus t, right?
OK, so let's take this bigger two and hold on. And the plot t exponential of minus t negative-- let's plot it using black, [INAUDIBLE] black. OK, we see we had a pretty good approximation of the real derivative.
We can also see that it is not exact, right? So if the black line goes right through the center of the circle, then I have a very good approximation. But, like, [INAUDIBLE]. All right, so we can see we do have some approximation error, which in this case, we call truncation error.
OK, so this, how big the truncation error is, is the real meat of this lecture, OK? Truncation error. And here, we start the Taylor series analysis. So, Professor Wilcox last lecture said you're going to have so much fun with Taylor series.
So I hope you have already started looking at Taylor series and it is important you guys do look at Taylor series. Because of the importance, I use red, OK? Taylor series analysis-- OK, so we want to know how much approximation error do we have by approximating the limit with this finite delta t, all right?
We want to know what is the difference between df dt and our approximation. So this is at some t and this approximation is t plus delta t minus f of t. OK, how do I figure out this difference? How should I figure out this difference?
Then we can compute both because-- yeah, we can compute both. And then we can prove a difference. We can [INAUDIBLE] this line with this line [INAUDIBLE] lambda.
Like, this is probably 0.05 or something like that. Yes, this is a translation error, right? Because this only works when we do have an additional solution, right? And so what-- is there any way to activate how big the trunctation error is in a more general way?
AUDIENCE: Taylor series.
PROFESSOR: Taylor series-- OK, good. Taylor series-- what is Taylor series? Anybody can summarize what Taylor series is in, like, one sentence?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Yeah, an approximation of a function at some point using the values and derivatives of this function at a nearby point, right? OK, so here, what we-- we have three things. If you look at the formula we have, three things. [INAUDIBLE]
The derivative at t, the functionality at t [INAUDIBLE] the functionality, like, we have two points. One point is at t, the other points [INAUDIBLE].
At [INAUDIBLE] t, we have a derivative of [INAUDIBLE]. At another point, we [INAUDIBLE]. We can employ Taylor series in two ways.
One way is more convenient. The other [INAUDIBLE] is a little bit more inconvenient but still works. The most convenient way is [INAUDIBLE] f at t plus delta t using Taylor series. The more inconvenient way is to send both derivatives and the value [INAUDIBLE] using Taylor series [INAUDIBLE]. So let's do the convenient way first. And I'm going to show you and we're going to get something similar using the slightly more convenient way, all right?
So the easy way-- let me use green to denote the easy way. OK, so easy-- f at t plus delta t. So, the start of Taylor series-- the Taylor series I write as a formula is k goes from 0 to-- let me actually-- equal to infinity of f to the k-th derivative at t divided by k factorial delta t to the k-th. This is what Taylor series is.
And in this case, we do not want to keep anything above the first derivative. So I'm just going to be writing that-- I'm just going to be keeping two terms. The first term is k equal to 0. What is f to the 0-th derivative?
It's just f itself, right? This is k equal to 0. What is 0 factorial? One. What is delta t to the 0-th?
one. OK, so that is the first term. Now let's do k equal to 1.
What is the first derivative of k? It's df dt. What is one factorial? It's
one again. And delta t to the one-- that is delta t. Anything above that has a delta t to the at least second order, right? right? At least delta t squared. square.
So I am going to write all these terms a a big O times big O delta t squared. Has anybody seen this big O [INAUDIBLE]? You did, OK. What this means is that this [INAUDIBLE] compares [INAUDIBLE] as long as all these terms have a delta t squared or delta t cube or delta t [INAUDIBLE].
[INAUDIBLE] more of the [INAUDIBLE] square. OK, which is true for all the other terms [INAUDIBLE]. Or maybe green is a little bit too-- doesn't show really well on this. So let me see if I can-- let me change to this.
OK, yeah, that's better. All right, OK. So, once we do that, what does it change?
[INAUDIBLE] action [INAUDIBLE]. Yeah, actually don't know [INAUDIBLE]. OK, once we do this, we can plot this back into the truncation error we call tau.
Tau is going to be df dt at t minus something divided by delta t. And that something is-- first of all, it's a Taylor expansion of ft plus delta t minus f of t, right?
And now it's time to cancel the terms. So I'm going to write this as it is and, first of all, try to cancel the terms here. So delta t is [INAUDIBLE] here. This ft cancels with this ft, right?
So we have a df dt times delta t plus O delta t square, OK? And this delta t cancels with this delta t, which makes this cancel with this. So the only thing we get is, oh, delta t square divides by delta t.
[INAUDIBLE] square, delta t square [INAUDIBLE] all the terms that has delta t square or higher. Then [INAUDIBLE] divided by delta t. What do we get?
STUDENT: [INAUDIBLE]
PROFESSOR: Order delta t, exactly. All right, so what we know is that the truncation error, whatever it is, is order delta t, which means it decreases f delta t. So [INAUDIBLE] to kind of-- to visualize what this means, if I go back to here, so this is [INAUDIBLE] 0.05, right? [INAUDIBLE] 0.05. This for delta t of 0.1.
If I increase delta t to 0.2-- let me exaggerate. If I increase delta t t 1, what do you think this is going to get? Yeah, much bigger of course-- how much bigger?
No, no, the area is O delta t. Let me say, if I can make this [INAUDIBLE] if I may [INAUDIBLE] Now the [INAUDIBLE] 0.05 [INAUDIBLE]. It's 1.
It is going to be o delta t, which means delta t increase by a factor of two. This area increases by a factor of two. Let's just try to do this to see if what I said is true. 0.2-- OK, I'm going to be repeating what I'm going to be doing.
f is equal to-- no, I need to do t plus t equal to this. f is equal to this and the df dt is equal to this. Right, and I'm going to do a hold on and the plot. I'm going to be plotting t from 1 to n minus 1 df dt. And this time, I make it red.
All right, so o means I still want to plot in circles. r means red. So Let's see what I get. Is it clear?
The blue is the derivative I get [INAUDIBLE] delta t 0.1. The red is the derivative [INAUDIBLE] delta t 0.2. The error [INAUDIBLE] the difference between the circle and the black line.
The black line is the exact [INAUDIBLE] derivative, which is only available if we have [INAUDIBLE]. And where [INAUDIBLE] the difference between the circles [INAUDIBLE] derivatives and the black line. This is real derivative.
It grew by a factor of two. OK, so this is using the easy way, Taylor series analysis. I'm also going to show you briefly the hard way, which is kind of pointless here.
But as you are going to see later on when we get to more complex schemes, we have to start using the more difficult way of doing Taylor series. I'm going to rewrite what is the [INAUDIBLE] error we're interested in here-- f of t plus delta t minus f of t divided by delta t, right? This is at t.
OK, Taylor series-- we're going to do it again, but we're going to represent f of t as a Taylor series of k goes from 0 to infinity f to the k-th t plus delta t divided by k factorial. Here should be, what? So I'm extending f of t with a Taylor series that is based on t plus delta t.
But there is something to the k power. And what is this something? OK, I'm just going to write a box here.
f of y is equal to summation of k equal to 0. Taylor series k is power fx k factorial what to the k-th power. Let me explain to you y at x. Y minus x-- exactly, right?
Previously, we have x being t, y being t plus delta t, OK? So y minus x is delta t. How about in this case?
Negative delta t, exactly. OK, does this make sense? All right, yeah. So this is going to be very useful, like for making sure you know every different view, every different manipulation of the Taylor series. It's going to be useful.
I'm going to show you another manipulation of the Taylor series. That is, if I take a derivative to both sides of the Taylor series, take derivative with respect to y, what I'm going to get-- if I take derivatives on the left hand side, what I'm going to get?
df d-- yeah, let me just say, yeah, df dy is equal to summation k equal to 0 to infinity. What is this term, taking derivative y? It's a constant. There is no derivative. So this term doesn't depend on y. Only this can depends on y, right? So what I'm going to get is k, after k-th power [INAUDIBLE], write x divided by k factorial.
I'm going to have y minus x to the k minus y power times another k here. So this is what happens when you type in [INAUDIBLE] derivative as a Taylor series. And now, if you expand this term at t plus delta t, you can write the same conclusion as we previously arrived, which is our scheme has a truncation error of order delta t. All right, let's take a mini break of like five minutes and then come back to our lecture.
All right, so [INAUDIBLE] to you a small entry into what we are going to be doing with Taylor series. It's not going to be obvious in the beginning, but like, this is going to be getting better and better method to exercise. OK, [INAUDIBLE].
OK, let's get back to [INAUDIBLE] again. I'm sure I lost all of you when I talk about Taylor series of t [INAUDIBLE] plus delta t. But let's not get [INAUDIBLE] onto this. It just requires you to go back and look at Taylor series with a little bit more familiarity.
OK, so here are-- I'm going to say a little bit on why we want to approximate the derivative and how is that going to give us a way of solve [INAUDIBLE]. OK, so remember, we are approximating the derivative df dt at time equal to t at f of t plus delta t minus f of t, right, over delta t. And that's not an idea I proposed. It's proposed by-- sorry, what was your name again?
STUDENT: Ariya.
PROFESSOR: Ariya, OK. OK, so yeah, if we stick to this scheme, we can actually start to integrate ODEs. OK, let's call this 0 equal to t0. So that's actually how each time step has a name-- so, t1 equal to delta t, p2 equal to 2 delta t, et cetera.
And the function values-- f of t at t equal to 0. I'm going to call it f0. The function value at f1, I'm going to call it t1 equal to f1. The function value at t2, I'm going to call it f2. Now let's say I have an ODE and I only have the solution at t0, t1, t2.
I want to know what is the solution at the next time stamp. What is the value of f3? So that is what happens when we solve ODEs, right? We start with the initial condition. We start with f0 that is given. The test case will compute what f1 si.
And now, once I computed f1, the task is to compute what f2 is. Once I compute f2, now the task is to compute what f3. How do I compute f3?
Yeah, how do I compute f3 using Taylor data points. So here's what is given. I have the ODE. I have df df equal to a function of f. Let's say-- I'm going to say minus lambda times f. So here's my single ODE I need to integrate.
And I can approximate-- I have an approximation to the derivative. And I also have f0, f1, f2. How do I compute f3?
So this is given. This is given. This is given.
This is given. This is given. How do I put them together to compute [INAUDIBLE]?
STUDENT: [INAUDIBLE]
PROFESSOR: Right, we have our function value at f2. We also have an estimate of the derivative at f2, which actually involved the unknown. So let me write down-- let me write down-- this is important. Let me write down what is the derivative estimate at f2-- at t2, sorry.
It is equal to f of t plus delta t, which is t3, right? This is t3 minus f at t2 divided by delta t. This actually involves an unknown in the derivative. And this is df dt by the differential equation is equal to minus lambda times f at, what? Is t2. Now, through this, by plotting both the approximation of the derivative and-- by basically putting both the approximation of the derivative and the function together, we converted the differential equation into a what equation?
STUDENT: [INAUDIBLE]
PROFESSOR: Yeah, into a difference equation, into a [INAUDIBLE] equation. We can just compute. So let me color this. This is unknown, this is known, and this is known.
We can just move all the unknowns to one side. So f of t3 is my f3 is equal to all the known on the other side. f of t2 is f2 minus delta t times lambda times f of t to [INAUDIBLE] 2.
Right, by doing this, I compute F3 out of f2. Now, what do I do when I-- now I have f3. What do I do when I want to compute f4?
Same thing, right? It's kind of recursive because my f4 is now equal to f3 minus delta t lambda f3, right? And I just could go on. The only thing I need is to plug in the derivative approximation here into the differential equation.
Let me do it again using another different scheme. [INAUDIBLE] me do this [INAUDIBLE]. OK, different scheme.
OK, another way to approximate the function-- I mean, I have been calling the function f, but like, because we started by discretizing a function. But, like, in [INAUDIBLE] ODEs, it is more convention to call the solution u. So from now on, I'm just going to call my function as u of t instead of f of t.
This is just a-- but, like, everything else is the same. [INAUDIBLE] with this, we'll just call it u of t. OK, u of t, I have du dt equal to minus lambda times u of t.
I'm going to devise another scheme of approximating the derivative. I have du dt at a certain time t equal to u at t plus delta t minus u at t minus delta t divided by 2 delta t. OK, so let's do two things.
One is, why is this a [INAUDIBLE] approximation to the derivative? Two-- now, if this is a valid approximation of the derivative, how do I make this a [INAUDIBLE] of [INAUDIBLE] ODE? So, anybody have an idea to how to answer either of these [INAUDIBLE]? Yes?
STUDENT: [INAUDIBLE]
PROFESSOR: It's the definition of the derivative, but the definition of the derivative is also [INAUDIBLE], yes?
STUDENT: [INAUDIBLE]
PROFESSOR: Good. After the [INAUDIBLE] problems, it can be solved using Taylor series [INAUDIBLE]. OK, it's good.
So, answer to the first question-- y is a good approximation. OK, to answer that, again we use Taylor series. u at t plus delta t-- as usual, we are going to be expanding it using u and t plus du dt at t times delta t plus o delta t square, right? This is the same thing as we did of the other scheme.
The other scheme is [INAUDIBLE], right? Remember your reading. And ut minus delta t-- how do I expand that? Yes.
STUDENT: [INAUDIBLE]
Good. It's the same thing except for the distance between this t minus delta t and t is minus delta t, right? So in here, the distance between this variable and this variable is delta t. Here, the distance between this variable and this variable is minus delta t.
Therefore, all the values and derivative is the same except for here, I put minus delta t. And o, it should be minus delta t to the q. But does it matter? No, in the big O notation, the constant coefficients, [INAUDIBLE] the terms doesn't matter, right? So minus O delta t square is the same as o delta t square, is the same as five times o delta t square. It's the same as 1 million times over delta t square. As long as the terms have delta t square in front of it, no matter if it's multiplied by 1 million or 1 million or 1 trillion, it doesn't matter.
It's o delta t squared. If it's multiplied by minus 1, it's still o delta t squared, right? OK, so now, when I plug in both approximations into this formula, t plus delta t minus u t minus delta t over 2 delta t, is equal to-- these two terms cancel out. These two terms-- they actually have different signs.
So they become [INAUDIBLE] 2 times delta t. Du dt plus-- I'm just going to write this down. What is this plus? What is the difference between these 2 o delta t's? Do they cancel out? No?
AUDIENCE: [INAUDIBLE]
PROFESSOR: It's still o delta t square. Why?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Exactly, because they don't know what the coefficients are. Maybe one of these [INAUDIBLE] has 1 million in front of it. The other o del t has a 0.001 in front of it.
So no matter the summation of them or the difference between them, they are still o delta t. So this is going to be-- these two cancel out. du dt, this is at t, plus-- again, this o delta t squared divided by delta t, it becomes o delta t, right? OK, so this is still a good approximation, because the difference between them is o delta t. And it is actually stronger than o delta t because in these two Taylor series expansions, you can expend more terms.
And you're going to find out that coefficients before [INAUDIBLE] these o delta t squares before the delta t squares still are going to cancel out. So you're going to get o delta t cubed out of these Taylor series [INAUDIBLE]. And over here, you can actually prove that-- it's in the reading. You can prove that this is not only o delta t but also o delta t square.
It's a confusing [INAUDIBLE] conflict itself. We're saying that this is both o delta t and and o delta t square. Do I contradict myself? No? Hm?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Yeah, how does it go? How can this term be both o delta t and o delta t squared? Again [INAUDIBLE] I want-- somebody else, yeah.
AUDIENCE: Because o delta t is the [INAUDIBLE] delta t squared and [INAUDIBLE]
PROFESSOR: Yeah, because o delta t actually includes o delta t squared. o delta t means it can contain delta t terms, delta t squared terms, and delta t [INAUDIBLE]. But any or all of these terms can have zero as their coefficient. If I have an o delta t term and it happens that the coefficient before the delta t term is zero, then it is automatically going to be o delta t squared, right?
So I can say that o 1 is a superset of o delta t, which is a superset of o delta t square, which is a superset of o delta t cube, et cetera. Any questions? Questions?
All right. OK, and so now, the second question is how to make a scheme out of it. We already have an approximation. That is du dt at t is equal to u of t plus delta t minus u of t minus delta t divided by t delta t. We have a differential equation-- du dt equal to minus lambda u. Again, the only thing you need to do is to plug this into this.
So we have u of t plus delta t minus u of t minus delta t divided by 2 delta t is equal to minus lambda u at t. And when you look at these terms, when you are at time t, when the solution at time t is already known but anything beyond t is unknown, then this is known because this is the before t. This is known because this at t. The only unknown is this.
So again, we are going to rearrange the terms and say u t plus delta t is equal to u t minus delta t minus 2 delta t times lambda times ut. If t is going to be exactly on a time step, what I'm going to say is that u of i plus 1 is equal to ui minus 1 minus 2 delta t lambda times ui. OK, this notation is assuming I have discretized u into a uniform grid of space in delta t.
AUDIENCE: [INAUDIBLE]
PROFESSOR: Yeah.
AUDIENCE: [INAUDIBLE]
PROFESSOR: The question is, am I using a scheme that has a mathematical definition?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Oh, OK. So what does a scheme mean? A scheme is a numerical method, a method that you can implement into the computer to solve a differential equation. It just means if I have an initial condition, a scheme must be able to tell me where is the solution at the next time step and then tell me again, what is the solution at the next time step, et cetera, et cetera, right? A scheme is basically, you can think of it as an algorithm.
AUDIENCE: [INAUDIBLE]
PROFESSOR: Right.
AUDIENCE: [INAUDIBLE]
PROFESSOR: We're making what?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Exactly, exactly, yeah. Good question. I was just so familiar with what scheme is, I didn't explain it. So it's a good catch.
If you find something like this, please to raise a question. Because if you have a question, your classmates probably also have the same question. OK, so here, we basically derived two schemes.
By the way, this scheme is the midpoint scheme. We have tow rite two schemes-- the [INAUDIBLE] scheme, which is the scheme that we have-- yeah, the scheme we have here. We can run it in general as f of i plus 1 equal to f of i minus delta t lambda f i. So this is the Forward Euler.
This is called a forward Euler scheme in your reading. And we have also derived the midpoint scheme both from-- they are different only by that they're different in how to approximate the derivative in time. Once you have an approximation of the derivative in time, you plug into the differential equation. You can [INAUDIBLE], OK?
Now what I want to talk about next is what is the local order of accuracy. And I'm going to say that the local order of accuracy is how accurate the scheme approximates the time derivative. It is related to what tau is.
So for example, in forward Euler, how is o delta t, right? This is what we derived right before we take the break. So tau is-- tau in forward Euler is the df dt at t minus-- let me call it u because it's a small-- it's more consistent to the notes-- minus u at t plus delta t minus ut divided by delta t. This is o delta t for forward Euler.
And the tau is the same derivative minus a different approximation for the derivative, ut minus delta t divided by 2 delta t. This is actually o delta t square for the midpoint and through Taylor series analysis. OK, the local order of accuracy is the exponent on top of delta t of this truncation error.
So forward Euler is first order accurate because it's o delta t. Midpoint is second order accurate because it's o delta t square. So the local order of accuracy is 1 for forward Euler. The local one of accuracy is 2 for midpoint.
Usually, the higher the order is, the better scheme is. Why? Because you can make the scheme more accurate by only increasing or decreasing delta t a little bit.
We just need an analysis of forward Euler. If I make delta t half as small, how much smaller the truncation error is going to be? Half.
Now, if I have a midpoint, if I decrease the-- if I decrease the delta t by half, how much more accurate is the midpoint going to be? A quarter-- exactly. So-- yeah, so this is how we usually kind of visualize the order of accuracy. OK, so let's say the [INAUDIBLE] is delta t.
OK, so let's say this is forward Euler. So let's say this is first order accurate. This is delta t and delta t, let's say, is 1 here, 1 to 1 here, 1 to o1 here, 1 to oo1 here. So let's say when you have 1 I get a pretty big error [INAUDIBLE].
When I decrease delta t to 0.1, what do you think the error is going to become? [INAUDIBLE], right. So I should be right here, right? I should be right here because it's o delta t. Now, if I decrease it to 0.01, [INAUDIBLE] should decrease by another 10 and by another 10.
So if I link it, it will be a line with a slope of 1 in a [INAUDIBLE] All right? Now-- so this is forward Euler. This is first order accurate.
How about a second order accurate scheme? So again, [INAUDIBLE] delta t is 1, 0.1, 0.01, 0.001 here. And again, if I am [INAUDIBLE] large error at delta t equal to 1, what do you think is the error going to be at delta t equal to 0.1? [INAUDIBLE]
AUDIENCE: [INAUDIBLE]
PROFESSOR: Why should it be here? It's delta t square [INAUDIBLE] smaller. Delta t square is 100 times smaller.
So I should be here when delta t decreased by a factor of 10. I should be here when delta t is decreased by another factor of 10. If I link it, it will be a slope like this and I'm kind of out of touch when I'm 0.001. So you can see that second order scheme is something much better than the first order scheme.
Essentially, I have the luxury of making delta t very small-- OK, so going back to MATLAB, if I can say df dt equal to-- now I cannot no longer do this, right? I have to do f of 3 to n [INAUDIBLE] 3 to n OK, let me clear the workspace so that you can see better. [INAUDIBLE] window [INAUDIBLE].
OK, so df dt-- let me say midpoint is equal to f of 3 to n minus f of 1 to n minus 2. What does this mean? I'm picking from the third value to the last value-- minus the first value to the third last value. OK.
AUDIENCE: [INAUDIBLE]
PROFESSOR: I'm taking the different space by two. OK, so that is like-- there is how I view f at t plus delta t minus f of t minus delta t. So I should divide this by how much?
2 times delta t-- exactly. And here, I think I'm actually doing a really big delta t. It doesn't matter. [INAUDIBLE] OK, so when I plot it I should be plotting it against the t going from 2 to n minus 1, right? Because this is now my t.
2 to n minus 1 is my t. 3 to n is my t plus delta t. 1 to n minus 2 is t minus delta t. So this is my t. I'm going to [INAUDIBLE] this against df dt midpoint. I'm going to be square and what color do you want?
AUDIENCE: Pink.
PROFESSOR: Pink? Magenta? OK, [INAUDIBLE] square, OK.
AUDIENCE: [INAUDIBLE]
PROFESSOR: Yes, so [INAUDIBLE] even though [INAUDIBLE] delta t [INAUDIBLE] 0.2 [INAUDIBLE] kind of goes right through the exact derivative. OK, so this is how a second order scheme is better than a first order scheme. And how to assess that? Taylor analysis, right?
OK, maybe I will just show you how did I get the certain order. So if I further extend this, [INAUDIBLE] what I get is the d square u dt times delta t squared over 2, right? [INAUDIBLE] squared over 2 and then plus the cube.
Here, what I get is plus d square u dt times minus delta t square, which is the same as delta t square, over two plus this to the cube. And you can see when I subtract the u of t plus delta t by u [INAUDIBLE] delta t, the [INAUDIBLE] order term cancel. The first order term, because they are the same, so they add together.
And the second order term also cancel. The third order term again is going to have different signs and subtracting them actually makes them bigger. So it is order two.
So here, I can modify this to delta t cube because the square term actually cancel each other. And here is going to be square. All right, and we see that by actually observing the increased accuracy.
OK, so let's-- we have a little bit of time. Let's actually start the contest. And I would actually not end it today.
I would have you start it today and maybe we'll come back and report your findings in the beginning of the next lecture. All right, so the concept is like this. I want the best X scheme.
How do I [INAUDIBLE] my best? The highest local order of accuracy-- OK, what does that mean? I want to o delta t to as high power as possible, right?
OK, and the X-- what does X mean? X means one of these-- one of these four. And I'd like you to form a eteam with one person or two other persons and commit to either one, two, or three or four and figure out how do you design a scheme to make it as accurate as possible.
AUDIENCE: [INAUDIBLE]
PROFESSOR: OK, so I'm going to give you the formula now. Best implicit one step scheme means I want u at [INAUDIBLE] so u as i plus 1. Or let me actually use u [INAUDIBLE] t plus delta t equal to something times du dt at t plus delta t plus something times u at t plus something times du dt at t.
All right, this is the best implicit one step scheme and your task is to figure out what teh questions are. OK, best explicit two step scheme-- it means this. It means you add t plus delta t has to equal to-- because it is explicit, it does not allow a derivative [INAUDIBLE] order t.
So you have to do a question mark of ut plus a question mark at du dt at t. And now, because it's two step, two step means I allow one more step to be used in determining the solution t plus delta t, which means I can use u at t minus delta t and du dt at t minus delta t. All right, so this is the degrees of freedom you have in your best explicit two step scheme. And what is the difference between explicit two step scheme and an implicit two step scheme?
Yes, the answer is you can use du dt at t plus delta t. So I'm going to circle the term that makes a scheme implicit-- right, so implicit. So with the implicit scheme, you are allowed to use u plus delta t equal to-- the increased term is a question mark times du dt at t plus delta t. If I don't have this term, it will be explicit.
All the other terms are the same-- question mark times ut times question mark times du dt at t plus question mark u at t minus delta t plus question mark times du dt at t minus delta t. So let's just choose between the three-- the first three options. OK, and time is over, but I'd like you to find a team mate. Commit to one of these axes and come up with an answer. And we'll star to discuss this in the next lecture.
