Optimal control theory, a generalization of the calculus of variations, is a mathematical optimization method for deriving control policies. The method is largely due to the work of Lev Pontryagin and his collaborators, summarized in English in Prontryagin (1962). Calculus of variations is a field of mathematics that deals with functions of functions, as opposed to ordinary calculus which deals with functions of numbers. ... In mathematics, the term optimization, or mathematical programming, refers to the study of problems in which one seeks to minimize or maximize a real function by systematically choosing the values of real or integer variables from within an allowed set. ... Lev Semenovich Pontryagin (Russian: Лев Семёнович Понтрягин) (3 September 1908- 3 May 1988) was a Soviet/Russian mathematician. ...

General method

Optimal control deals with the problem of finding a control law for a given system such that a certain optimality criterion is achieved. A control problem includes a cost functional that is a function of state and control variables. An optimal control is a set of differential equations describing the paths of the control variables that minimize the cost functional. The optimal control can be derived using Pontryagin's maximum principle (a necessary condition), or by solving the Hamilton-Jacobi-Bellman equation (a sufficient condition). Optimization is a branch of mathematics which is concerned with finding maxima and minima of real-valued functions. ... It has been suggested that this article or section be merged with Hamiltonian (control theory). ... In logic, the words necessary and sufficient describe relations that hold between propositions or states of affairs, if one is conditional on the other. ... The Hamilton-Jacobi-Bellman (HJB) equation is a partial differential equation which is central to optimal control theory. ... In logic, the words necessary and sufficient describe relations that hold between propositions or states of affairs, if one is conditional on the other. ...

We begin with a simple example. Consider a car traveling on a straight line through a hilly road. The question is, how should the driver press the accelerator pedal in order to minimize the total traveling time? Clearly in this example, the term control law refers specifically to the way in which the driver presses the accelerator and shifts the gears. The "system" consists of both the car and the road, and the optimality criterion is the minimization of the total traveling time. Control problems usually include ancillary constraints. For example the amount of available fuel might be limited, the accelerator pedal cannot be pushed through the floor of the car, speed limits, etc. A constraint is a limitation of possibilities. ...

A proper cost functional is a mathematical expression giving the traveling time as a function of the speed, geometrical considerations, and initial conditions of the system. It is often the case that the constraints are interchangeable with the cost functional.

Another optimal control problem is to find the way to drive the car so as to minimize its fuel consumption, given that it must complete a given course in a time not exceeding some amount. Yet another control problem is to minimize the total monetary cost of completing the trip, given assumed monetary prices for time and fuel.

A more abstract framework goes as follows. Given a dynamical system with time-varying input u(t), time-varying output y(t) and time-varying state x(t), define a cost functional to be minimized. The cost functional is the sum of the path costs, which usually take the form of an integral over time, and the terminal costs, which is a function only of the terminal (i.e., final) state, x(T). Thus, this cost functional typically takes the form The Lorenz attractor is an example of a non-linear dynamical system. ...

where T is the terminal time of the system. It is common, but not required, to have the initial (i.e., starting) time of the system be 0 as shown. The minimization of a functional of this nature is related to the minimization of action in Lagrangian mechanics, in which case L(x,u,t) is called the Lagrangian. In physics, the action is an integral quantity that is used to determine the evolution of a physical system between two defined states using the calculus of variations. ... Lagrangian mechanics is a re-formulation of classical mechanics introduced by Joseph Louis Lagrange in 1788. ... A Lagrangian of a dynamical system, named after Joseph Louis Lagrange, is a function of the dynamical variables and concisely describes the equations of motion of the system. ...

Linear quadratic control

It is very common, when designing proper control systems, to model reality as a linear system, such as A linear system is a model of a system based on some kind of linear operator. ...

One common cost functional used together with this system description is

where the matrices Q and R are positive-semidefinite and positive-definite, respectively. Note that this cost functional is thought in terms of penalizing the control energy (measured as a quadratic form) and the time it takes the system to reach zero-state. In mathematics, a matrix (plural matrices) is a rectangular table of numbers or, more generally, a table consisting of abstract quantities that can be added and multiplied. ...

This functional could seem rather useless since it assumes that the operator is driving the system to zero-state, and hence driving the output of the system to zero. This is indeed correct. However the problem of driving the output to the desired level can be solved after the zero output one is. In fact, it can be proved that this secondary problem can be solved in a very straightforward manner. The optimal control problem defined with the previous functional is usually called the state regulator problem and its solution the linear quadratic regulator (LQR) which is no more than a feedback matrix gain of the form The theory of optimal control is concerned with operating a dynamic system at minimum cost. ...

where K is a properly dimensioned matrix and solution of the continuous time dynamic Riccati equation. This problem was elegantly solved by Rudolf Kalman (1960). In mathematics, a Riccati equation is any ordinary differential equation that has the form It is named after Count Jacopo Francesco Riccati (1676-1754). ... Rudolf Emil Kálmán Rudolf Emil Kálmán (born May 19, 1930 in Budapest, Hungary) is an American-Hungarian mathematical system theorist, who is an electrical engineer by training. ...

Discrete time control

The examples thus far have shown continuous time systems and control solutions. In fact, as optimal control solutions are now often implemented digitally, contemporary control theory is now primarily concerned with discrete time systems and solutions. Continuous time occurs when time is sampled continuously. ... A digital system is one that uses discrete values (often electrical voltages), especially those representable as binary numbers, or non-numeric symbols such as letters or icons, for input, processing, transmission, storage, or display, rather than a continuous spectrum of values (ie, as in an analog system). ... Discrete time is non-continuous time. ...

See also

• dynamic programming
• Bellman equation
• Linear-quadratic regulator
• trajectory optimization

In mathematics and computer science, dynamic programming is a method of solving problems exhibiting the properties of overlapping subproblems and optimal substructure (described below) that takes much less time than naive methods. ... Bellman equations occur in dynamic programming. ... The theory of optimal control is concerned with operating a dynamic system at minimum cost. ... Trajectory optimization is the process by which engineers design a trajectory that minimizes or maximizes some measure of performance. ...

Reference books

• Rudolf Kalman, 1960 .
• L. S. Pontryagin, 1962. The Mathematical Theory of Optimal Processes.
• Bryson, A. E., 1969. Applied Optimal Control: Optimization, Estimation, & Control.
• Kirk, D. E., 2004. Optimal Control Theory: An Introduction.
• Lebedev, L. P., and Cloud, M. J., 2003. The Calculus of Variations and Functional Analysis with Optimal Control and Applications in Mechanics. World Scientific. Especially chpt. 2.
• Lewis, F. L., and Syrmos, V. L., 19nn. Optimal Control, 2nd ed. John Wiley & Sons.
• Stengel, R. F., 1994. Optimal Control and Estimation. Dover.
• Sontag, Eduardo D. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second Edition. Springer. (ISBN 0-387-984895) (available free online)

Rudolf Emil Kálmán Rudolf Emil Kálmán (born May 19, 1930 in Budapest, Hungary) is an American-Hungarian mathematical system theorist, who is an electrical engineer by training. ... Lev Semenovich Pontryagin (Russian: Лев Семёнович Понтрягин) (3 September 1908- 3 May 1988) was a Soviet/Russian mathematician. ...

Journals

• Optimal Control Applications and Methods. John Wiley & Sons, Inc.

External links

• Elmer G. Wiens: Optimal Control - Applications of Optimal Control Theory Using the Pontryagin Maximum Principle with interactive models.