Numerical differentiation · scipython.com

P10.2.1Numerical differentiation

The simplest (and least accurate) way to calculate the first derivative of a function is to simply use the definition:

f'(x) = \lim_{h\rightarrow 0} \frac{f(x+h) - f(x)}{h},

fixing $$h$$ at some small value, our approximation is

f'(x) \approx \frac{f(x+h) - f(x)}{h},

Using the function $$f(x) = e^x$$ , which value of $$h$$ (to the nearest power of 10) gives the most accurate approximation to $$f'(1) = e$$ ?

Show solution

First define a function to return the derivative of $$f(x)$$ using a provided value for $$h$$ :

In [x]: def calc_deriv(x, f, h):
   ...:     return (f(x+h) - f(x)) / h

and use it to approximate the derivative of $$f(x)=e^x$$ at $$x=1$$ for $$h$$ between $10^{-17}$ and $$0.1$$ on a logarithmic scale:

In [x]: h = np.logspace(-17,-1,17)
In [x]: derivs = calc_deriv(1, np.exp, h)

We're interested in the differences between these approximations and the true answer, $$f'(1) = e$$ . On a log scale,

In [x]: errors = np.abs(derivs - np.e)
In [x]: plt.loglog(h, errors, base=10)
In [x]: plt.xlabel("$h$")
In [x]: plt.ylabel("error")
In [x]: plt.show()

Finding the optimum displacement in a simplistic numerical calculation of a derivative.

The optimum, $$h$$ , balancing truncation and cancellation errors, is about $10^{-8}$ .