Numerical ordinary differential equations is the part of numerical analysis which studies the numerical solution of ordinary differential equations (ODEs). This field is also known under the name numerical integration, but some people reserve this term for the computation of integrals. Numerical analysis is the study of algorithms for the problems of continuous mathematics (as distinguished from discrete mathematics). ... In mathematics, a differential equation is an equation in which the derivatives of a function appear as variables. ... In numerical analysis, the term numerical integration is used to describe a broad family of algorithms for calculating the numerical value of a definite integral, and by extension, the term is also sometimes used to describe numerical algorithms for solving differential equations. ... This article deals with the concept of an integral in calculus. ...

Many differential equations cannot be solved analytically, in which case we have to satisfy ourselves with an approximation to the solution. The algorithms studied here can be used to compute such an approximation. An alternative method is to use techniques from calculus to obtain a series expansion of the solution. Flowcharts are often used to represent algorithms. ... For other uses of the term calculus see calculus (disambiguation) Calculus is a central branch of mathematics, developed from algebra and geometry, and built on two major complementary ideas. ... In mathematics, a series is a sum of a sequence of terms. ...

Ordinary differential equations occur in many scientific disciplines, for instance in mechanics, chemistry, ecology, and economics. In addition, some methods in numerical partial differential equations convert the partial differential equation into an ordinary differential equation, which must then be solved. Mechanics (Latin mechanicus, from the Greek mechanikos, one skilled in machines) is a variety of specialised sciences pertaining to the functions and routine operations of machines, machine-like devices or objects. ... -1... Ecology can mean either: the natural environment, or an analysis or study using the principles and methods of ecological science. ... Economics is the social science studying production and consumption through measurable variables. ... In mathematics, and in particular analysis, a partial differential equation (PDE) is an equation involving partial derivatives of an unknown function. ...

The problem

We want to approximate the solution of the differential equation

where f is a function that maps [t₀,∞) × R^d to R^d, and the initial condition y₀ ∈ R^d is a given vector.

The above formulation is called an initial value problem (IVP). The Picard-Lindelöf theorem states that there is a unique solution, if f is Lipschitz continuous. In contrast, boundary value problems (BVPs) specify (components of) the solution y at more than one points. Different methods need to be used to solve BVPs, for example the shooting method, multiple shooting or global methods like finite differences or collocation. In mathematics, an initial value problem is a statement of a differential equation together with specified value of the unknown function at a given point in the domain of the solution. ... In mathematics, the Picard-Lindelöf theorem on existence and uniqueness of solutions of differential equations ( Picard 1890, Lindelöf 1894) states that an initial value problem has exactly one solution if f is Lipschitz continuous in , continuous in as long as stays bounded. ... In mathematics, a function f : M → N between metric spaces M and N is called Lipschitz continuous (or is said to satisfy a Lipschitz condition) if there exists a constant K > 0 such that d(f(x), f(y)) ≤ K d(x, y) for all x and y in M... In mathematics, a boundary value problem consists of a differential equation to be satisfied at all points in the interior of an interval or a region and a set of boundary conditions specifying the values of the solution or some of its derivatives everywhere on the boundary of the interval... In numerical analysis, the shooting method is a method for solving a boundary value problem by reducing it to the solution of an initial value problem. ... There are two subfields of mathematics that concern themselves with finite differences. ... This article is in need of attention. ...

Note that we restrict ourselves to first-order differential equations (meaning that only the first derivative of y appears in the equation, and no higher derivatives). However, a higher-order equation can easily be converted to a first-order equation by introducing extra variables. For example, the second-order equation y'' = −y can be rewritten as two first-order equations: y' = z and z' = −y.

Methods

Two elementary methods are discussed to give the reader a feeling for the subject. After that, pointers are provided to other methods (which are generally more accurate and efficient). The methods mentioned here are analysed in the next section.

The Euler method

Starting with the differential equation (1), we replace the derivative y' by the finite difference approximation There are two subfields of mathematics that concern themselves with finite differences. ...

which yields the following formula

This formula is usually applied in the following way. We choose a step size h, and we construct the sequence t₀, t₁ = t₀ +  h, t₂ = t₀ + 2 h, ... We denote by y_n a numerical estimate of the exact solution y(t_n). Motivated by (3), we compute these estimates by the following recursive scheme In mathematics and computer science, recursion is a particular way of specifying (or constructing) a class of objects (or an object from a certain class) with the help of a reference to other objects of the class: a recursive definition defines objects in terms of the already defined objects of...

y_{n + 1} = y_n + hf(t_n,y_n).

This is the Euler method, named after Leonhard Euler who described this method in 1768. Leonhard Euler aged 49 (oil painting by Emanuel Handmann, 1756) Leonhard Euler (April 15, 1707 - September 18, 1783) (pronounced oiler) was a Swiss mathematician and physicist. ... Events January 9 - Philip Astley stages the first modern circus (London) May 10 - John Wilkes is imprisoned for writing an article for the North Briton severely criticizing King George III. This action provokes rioting in London Secretary of State for colonies appointed in Britain Massachusetts Assembly dissolved for refusing to...

The backward Euler method

If, instead of (2), we use the approximation

we get the backward Euler method:

y_{n + 1} = y_n + hf(t_{n + 1},y_{n + 1}).

The backward Euler method is an implicit method, meaning than we have to solve an equation to find y_n+1. One often uses functional iteration or (some modification of) the Newton-Raphson method to achieve this. Of course, it costs time to solve this equation; this cost must be taken into consideration when one selects the method to use. In numerical analysis, Newtons method (or the Newton-Raphson method) is an efficient algorithm for finding approximations to the zeros (or roots) of a real-valued function. ...

Generalisations

The Euler method is often not accurate enough. In more precise terms, it only has order one (the concept of order is explained below). This caused mathematicians to look for higher-order methods.

One possibility is to use not only the previously computed value y_n to determine y_n+1, but to make the solution depend on more past values. This yields a so-called multistep method. Almost all practical multistep methods fall within the family of linear multistep methods, which have the form

Another possibility is to use more points in the interval [t_n,t_n+1]. This leads to the family of Runge-Kutta methods, named after Carle Runge and Martin Kutta. One of their fourth-order methods is especially popular. In numerical analysis, the Runge-Kutta methods are a family of techniques for the approximation of solutions of ordinary differential equations. ... Carle David Tolmé Runge (August 30, 1856 – January 3, 1927) was a German mathematician, physicist, and spectroscopist. ... Martin Wilhelm Kutta (November 3, 1867 - December 25, 1944) was a German mathematician. ...

Both ideas can also be combined. The resulting methods are called general linear methods.

Advanced features

A good implementation of one of these methods for solving an ODE entails more than the time-stepping formula.

It is often inefficient to use the same step size all the time, so variable step-size methods have been developed. Usually, the step size is chosen such that the (local) error per step is below some tolerance level. This means that the methods must also compute an error indicator, an estimate of the local error.

An extension of this idea is to choose dynamically between different methods of different orders (this is called a variable order method). Extrapolation methods are often used to construct various methods of different orders.

Other desirable features include:

• dense output: cheap numerical approximations for the whole integration interval, and not only at the points t₀, t₁, t₂, ...

• event location: finding the times where, say, a particular function vanishes.

• support for parallel computing.

Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain faster results. ...

Alternative methods

Many methods do not fall within the framework discussed here. Some classes of alternative methods are:

• multiderivative methods, which use not only the function f but also its derivatives. This class includes Hermite-Obreschkoff methods and Fehlberg methods.

• methods for second order ODEs. We said that all higher-order ODEs can be transformed to first-order ODEs of the form (1). While this is certainly true, it may not be the best way to proceed. In particular, Nyström methods work directly with second-order equations.

• geometric integration methods are especially designed for special classes of ODEs (e.g. Hamiltonian equations, reversible equations). They take care that the numerical solution respects the underlying structure or geometry of these classes.

Hamiltonian mechanics is a re-formulation of classical mechanics that was invented in 1833 by William Rowan Hamilton. ...

Analysis

Numerical analysis is not only the design of numerical methods, but also their analysis. Three central concepts in this analysis are convergence (whether the method approximates the solution), order (how well it approximates the solution), and stability (whether errors are damped out). Numerical analysis is the study of algorithms for the problems of continuous mathematics (as distinguished from discrete mathematics). ... In the mathematical subfield of numerical analysis, numerical stability is a property of numerical algorithms. ...

Convergence

A numerical method is said to be convergent if the numerical solution approaches the exact solution as the step size h goes to 0. More precisely, we require that for every ODE (1) with a Lipschitz function f and every t^* > 0, In mathematics, a function f : M → N between metric spaces M and N is called Lipschitz continuous (or is said to satisfy a Lipschitz condition) if there exists a constant K > 0 such that d(f(x), f(y)) ≤ K d(x, y) for all x and y in M...

All the methods mentioned above are convergent. In fact, convergence is a condition sine qua non for any numerical scheme. Sine qua non or conditio sine qua non is a Latin legal term for without which it could not be (but for). It refers to an indispensable action, condition or thing. ...

Order

Suppose the numerical method is

The method is said to have order p if

The quantity on the left-hand side is called the local error of the method. The (forward) Euler method and the backward Euler method introduced above both have order 1. Most methods being used in practise attain higher order.

The local error is the error committed in a single step. A related concept is the global error, the error sustained in all the steps one needs to reach a fixed time t. Explicitly, the global error at time t is y_N - y(t) where N = (t-t₀)/h. The global error of a pth order one-step method (that is, a method of the form (4) with k = 1) is O(h^p); in particular, such a method is convergent. This statement is not necessarily true for multi-step methods.

Stability and stiffness

Loosely speaking, a numerical method is called stable if unwanted components in the numerical solution die out over time. Many different aspects of stability have been discussed in the literature. We will only treat one of them. In the mathematical subfield of numerical analysis, numerical stability is a property of numerical algorithms. ...

A method is A-stable if the numerical results y_n approach zero as n → 0 for all values of the step size h when this method is applied to the equation y' = λy for all λ ∈ C with Re λ < 0. Note that for this equation, the exact solution also goes to zero. The (forward) Euler method is not A-stable, but the backward Euler method is A-stable.

For some differential equations, it does not matter much whether the method is stable. However, for other equations, stable methods perform far better; these equations are said to be stiff (it is hard to formulate a more precise definition). Stiffness is often caused by the presence of different time scales in the underlying problem. Stiff problems are ubiquitous in chemical kinetics, control theory, solid mechanics, weather prediction, biology, and electronics. This article is in need of attention. ... In engineering and mathematics, control theory deals with the behaviour of dynamical systems over time. ... Solid mechanics (also known as the theory of elasticity) is a branch of physics, which governs the response of solid material to applied stress (e. ... Composite satellite image showing the progress of a hurricane weather system approaching the east coast of America Weather comprises all the various phenomena that occur in the atmosphere of a planet. ... Biology is the science of life (from the Greek words bios = life and logos = reasoned account). ... Electronics is the study and use of electrical devices that operate by controlling the flow of electrons or other electrically charged particles in devices such as thermionic valves and semiconductors. ...

History

Below is a concise timeline of some important developments in this field. Alternative meanings: Timeline is a 1999 science fiction novel by Michael Crichton Timeline is a 2003 film based on the novel. ...

• 1768 - Leonhard Euler publishes his method.
• 1824 - Augustin Louis Cauchy proves convergence of the Euler method. In this proof, Cauchy uses the implicit Euler method.
• 1855 - First mention of the multistep methods of John Couch Adams in a letter written by F. Bashforth.
• 1895 - Carle Runge publishes the first Runge-Kutta method.
• 1905 - Martin Kutta describes the popular fourth-order Runge-Kutta method.
• 1910 - Lewis Fry Richardson announces his extrapolation method.
• 1952 - Charles F. Curtiss and Joseph Oakland Hirschfelder coin the term stiff equations.

Events January 9 - Philip Astley stages the first modern circus (London) May 10 - John Wilkes is imprisoned for writing an article for the North Briton severely criticizing King George III. This action provokes rioting in London Secretary of State for colonies appointed in Britain Massachusetts Assembly dissolved for refusing to... Leonhard Euler aged 49 (oil painting by Emanuel Handmann, 1756) Leonhard Euler (April 15, 1707 - September 18, 1783) (pronounced oiler) was a Swiss mathematician and physicist. ... 1824 was a leap year starting on Thursday (see link for calendar). ... Augustin Louis Cauchy Augustin Louis Cauchy ( August 21, 1789 – May 23, 1857) was a French mathematician. ... 1855 was a common year starting on Monday (see link for calendar). ... For other people named John Adams, see John Adams (disambiguation). ... 1895 was a common year starting on Tuesday (see link for calendar). ... Carle David Tolmé Runge (August 30, 1856 – January 3, 1927) was a German mathematician, physicist, and spectroscopist. ... In numerical analysis, the Runge-Kutta methods are a family of techniques for the approximation of solutions of ordinary differential equations. ... 1905 was a common year starting on Sunday (see link for calendar). ... Martin Wilhelm Kutta (November 3, 1867 - December 25, 1944) was a German mathematician. ... In numerical analysis, the Runge-Kutta methods are a family of techniques for the approximation of solutions of ordinary differential equations. ... 1910 was a common year starting on Saturday (see link for calendar). ... Lewis Fry Richardson (October 11, 1881 - September 30, 1953) was a mathematician, physicist and psychologist. ... 1952 - Wikipedia /**/ @import /skins/monobook/IE50Fixes. ...

See also

• Midpoint method

References

• Ernst Hairer, Syvert Paul Nørsett and Gerhard Wanner, Solving ordinary differential equations I: Nonstiff problems, second edition, Springer Verlag, Berlin, 1993. ISBN 3-540-56670-8.
• Ernst Hairer and Gerhard Wanner, Solving ordinary differential equations II: Stiff and differential-algebraic problems, second edition, Springer Verlag, Berlin, 1996. ISBN 3-540-60452-9. This two-volume monograph systematically covers all aspects of the field.
• Arieh Iserles, A First Course in the Numerical Analysis of Differential Equations, Cambridge University Press, 1996. ISBN 0-521-55376-8 (hardback), ISBN 0-521-55655-4 (paperback). Textbook, targeting advanced undergraduate and postgraduate students in mathematics, which also discusses numerical partial differential equations.
• John Denholm Lambert, Numerical Methods for Ordinary Differential Systems, John Wiley & Sons, Chichester, 1991. ISBN 0-471-92990-5. Textbook, slightly more demanding than the book by Iserles.