The Taylor series expansion is a widely used method for approximating a complicated function by a polynomial. Taylor’s theorem and its remainder can be expressed in several different forms depending the assumptions one is willing to make.
This page discusses Taylor series results for scalar-valued functions. See also Taylor series for vector-valued functions.
Single variable
Basic form
The most basic statement of Taylor’s theorem is as follows:
Theorem (Taylor): Suppose \(n\) is a positive integer and \(f:\real \to \real\) is \(n\) times differentiable at a point \(x_0\). Then
\[f(x) = \sum_{k=0}^n \frac{f^{(k)}(x_0)}{k!}(x-x_0)^k + R_n(x,x_0),\]where the remainder \(R_n\) satisfies
\[R_n(x,x_0) = o(\abs{x-x_0}^n) \text{ as } x \to x_0.\]This little o form of the remainder is sometimes called the “Peano form” of the remainder.
Lagrange form
Theorem (Taylor): If \(f^{(n+1)}\) exists over an open interval containing \((x, x_0)\), then there exists \(\bar{x} \in (x, x_0)\):
\[R_n(x,x_0) = \frac{f^{(n+1)}(\bar{x})}{(n+1)!}(x-x_0)^{n+1}.\]This is also known as the mean-value form, as the mean value theorem is the central idea in proving the result.
Notes:
-
Depending on where you look, the necessary conditions for the above theorem are sometimes presented in a slightly more stringent form where \(f^{(n+1)}\) is required to be continuous, not merely exist. This is not necessary, but makes the proof substantially easier. The conditions can also be loosened slightly since \(f^{(n+1)}\) need not exist at the boundary points \(x\) and \(x_0\), so you sometimes see conditions that look like “If \(f^{(n+1)}\) exists on the open interval and \(f^{(n)}\) is continuous on the closed interval between \(x\) and \(x_0\), then…”. I prefer the above form for the sake of simplicity. The same conditions are present in the mean value theorem. For more discussion of this topic, see here.
-
Comparing the Basic and Lagrange forms for a second-order expansion,
\[\begin{align*} \textrm{(Basic)} && \quad f(x) &= f(x_0) + f'(x_0)(x-x_0) + \tfrac{1}{2}f''(x_0)(x-x_0)^2 + o(|x-x_0|^2) \\ \textrm{(Lagrange)} && \quad f(x) &= f(x_0) + f'(x_0)(x-x_0) + \tfrac{1}{2}f''(\bar{x})(x-x_0)^2 \end{align*}\]We can see that in the second case, we have a simpler expression, but to obtain it, we require \(f''\) to exist along the entire interval from \(x\) to \(x_0\), not just at the point \(x_0\).
Bounds on the remainder
A related concept is that if we can bound the derivative over the interval, then we can bound the remainder.
Theorem (Lagrange error bound): If \(f^{(n+1)}\) is continuous over an open interval containing \((x, x_0)\) and there exists \(M\) such that \(\abs{f^{(n+1)}(a)} \le M\) for all \(a \in (x, x_0)\), then
\[R_n(x,x_0) \le \frac{M}{(n+1)!}\abs{x-x_0}^{n+1}.\]Taylor’s theorem can therefore be expressed in both little o and big O forms (keeping in mind that the big O form requires additional assumptions concerning differentiability):
\[\begin{align*} \textrm{(Basic)} && \quad f(x) &= f(x_0) + f'(x_0)(x-x_0) + \tfrac{1}{2}f''(x_0)(x-x_0)^2 + o(|x-x_0|^2) \\ \textrm{(Big O)} && \quad f(x) &= f(x_0) + f'(x_0)(x-x_0) + \tfrac{1}{2}f''(x_0)(x-x_0)^2 + O(|x-x_0|^3) \\ \end{align*}\]Other forms
The above forms (basic, Lagrange, and big O) are the most common forms of Taylor’s theorem, although the remainder term can be expressed in several other ways, including the integral form, Cauchy form, and Roche-Schlömilch form. For example, the integral form is given below.
Theorem (Integral form): If \(f^{(n+1)}\) is continuous over an open interval containing \((x, x_0)\), then
\[R_n(x,x_0) = \int_{x_0}^x \frac{f^{(n+1)}(t)}{(n+1)!}(x-t)^n dt.\]Multivariable
For the multivariate case, the same concepts apply as they did above, so I’ll provide the theorems without much commentary. Also, I’ll give first- and second-order expansions explicitly rather than abstract formulas involving \(f^{(n)}\), since the form of \(f^{(n)}\) changes depending on \(n\) (scalar, vector, matrix, etc.).
Basic form
Theorem: Suppose \(f:\real^d \to \real\) is differentiable at a point \(\x_0\). Then
\[f(\x) = f(\x_0) + \nabla f(\x_0) \Tr (\x-\x_0) + o(\norm{\x-\x_0})\]Theorem: Suppose \(f:\real^d \to \real\) is twice differentiable at a point \(\x_0\). Then
\[\begin{align*} f(\x) &= f(\x_0) + \nabla f(\x_0) \Tr (\x-\x_0) + \\ &\quad \tfrac{1}{2} (\x-\x_0) \Tr \nabla^2 f(\x_0) (\x-\x_0) + o(\norm{\x-\x_0}^2) \end{align*}\]Lagrange form
For the theorems below, “\(\bar{\x}\) on the line segment connecting \(\x\) and \(\x_0\)” means that there exists \(w \in [0,1]\) such that \(\bar{\x} = w\x + (1-w)(\x_0)\); I sometimes abbreviate this as \(\bar{\x} \in LS(\x, \x_0)\) in proofs. The theorems also use neighborhood notation.
Theorem: Suppose \(f:\real^d \to \real\) is differentiable on \(N_r(\x_0)\). Then for any \(\x \in N_r(\x_0)\), there exists \(\bar{\x}\) on the line segment connecting \(\x\) and \(\x_0\) such that
\[\begin{align*} f(\x) = f(\x_0) + \nabla f(\bar{\x}) \Tr (\x-\x_0) \end{align*}\]Theorem: Suppose \(f:\real^d \to \real\) is twice differentiable on \(N_r(\x_0)\). Then for any \(\x \in N_r(\x_0)\), there exists \(\bar{\x}\) on the line segment connecting \(\x\) and \(\x_0\) such that
\[\begin{align*} f(\x) &= f(\x_0) + \nabla f(\x_0) \Tr (\x-\x_0) + \tfrac{1}{2} (\x-\x_0) \Tr \nabla^2 f(\bar{\x}) (\x-\x_0) \end{align*}\]Third order
Lastly, I’ll provide a form that goes out to third order. One could keep going, of course, but higher order terms for multivariate functions become rather cumbersome once they can no longer be represented with vectors and matrices.
Theorem: Suppose \(f:\real^d \to \real\) is three times differentiable on \(N_r(\x_0)\). Then for any \(\x \in N_r(\x_0)\), there exists \(\bar{\x}\) on the line segment connecting \(\x\) and \(\x_0\) such that
\[\as{ f(\x) &= f(\x_0) + \sum_{j=1}^d \frac{\partial f(\x_0)}{\partial x_j}(x_j-x_{0j}) \\ &\qquad + \frac{1}{2} \sum_{j=1}^d \sum_{k=1}^d \frac{\partial^2 f(\x_0)}{\partial x_j \partial x_k} (x_j-x_{0j}) (x_k-x_{0k}) \\ &\qquad + \frac{1}{6} \sum_{j=1}^d \sum_{k=1}^d \sum_{\ell=1}^d \frac{\partial^3 f(\bar{\x})}{\partial x_j \partial x_k \partial x_{\ell}} (x_j-x_{0j}) (x_k-x_{0k}) (x_{\ell}-x_{0\ell}), }\]where \(\partial f(\x_0)/\partial x_j\) is shorthand for \(\partial f(\x)/\partial x_j\) evaluated at \(\x_0\).