3.6 Introduction to Design Optimization | 3.6 Introduction to Design Optimization | Unit 3: Probabilistic Methods and Optimization | Computational Methods in Aerospace Engineering | Aeronautics and Astronautics

3.6.4 Finite Difference Method Applied to 1D Convection

Generally the objective function and constraints are complex, nonlinear functions of the design variables which are not known analytically. We therefore cannot compute analytic derivatives for the required gradients and Hessian information. However, it is still possible to estimate the gradient and Hessian using other methods. The simplest approach is to use finite differences.

A one-sided finite difference requires \(n+1\) function evaluations. The gradient is estimated as

\[\frac{\partial J}{\partial x_ j} \approx \frac{J(x_1,x_2,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)-J(x_1,x_2,\ldots ,x_ j,x_{j+1},\ldots ,x_ n)}{\Delta x_ j}\]

(3.75)

This approximation is first-order accurate: reducing \(\Delta x_ j\) by a factor of 2 reduces the error in the gradient by a factor of 2.

A more accurate estimate can be computed using a central difference, which requires \(2n\) function evaluations:

\[\frac{\partial J}{\partial x_ j} \approx \frac{J(x_1,x_2,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)-J(x_1,x_2,\ldots ,x_ j-\Delta x_ j,x_{j+1},\ldots ,x_ n)}{2\Delta x_ j}\]

(3.76)

This approximation is second-order accurate: reducing \(\Delta x_ j\) by a factor of 2 reduces the error in the gradient by a factor of 4.

The Hessian matrix can also be estimated using finite differences. A second-order central approximation of the \((i,j)\) entry of the Hessian matrix is given by:

\[\frac{\partial ^2 J}{\partial x_ i \partial x_ j} \approx \frac{J(x_1,x_2,\ldots ,x_ i+\Delta x_ i,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j} \\ + \frac{J(x_1,x_2,\ldots ,x_ i-\Delta x_ i,\ldots ,x_ j-\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j} \\ - \frac{J(x_1,x_2,\ldots ,x_ i-\Delta x_ i,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j} \\ - \frac{J(x_1,x_2,\ldots ,x_ i+\Delta x_ i,\ldots ,x_ j-\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j}\]

(3.77)