This is a substantially expanded (by about 30%) and improved edition of Vol. 1 of the best-selling dynamic programming book by Bertsekas. (A relatively minor revision of Vol.\ 2 is planned for the second half of 2001.) DP is a central algorithmic method for optimal control, sequential decision making under uncertainty, and combinatorial optimization. The treatment focuses on basic unifying themes and conceptual foundations. It illustrates the power of the method with many examples and applications from engineering, operations research, and economics.

Among its special features, the book:

(a) provides a unifying framework for sequential decision making

(b) develops the theory of deterministic optimal control including the Pontryagin Minimum Principle

(c) describes neuro-dynamic programming techniques for practical application of DP to complex problems that involve the dual curse of large dimension and lack of an accurate mathematical model

(d) provides a comprehensive treatment of infinite horizon problems in the second volume, and an introductory treatment in the first volume

(e) contains many exercises, with solutions of the most theoretical ones posted on the book's www page

Highlights of the revision: (a) Much new material on suboptimal control, including neuro-dynamic programming and rollout algorithms, and their applications in combinatorial optimization and stochastic optimal control. (b) A section on estimation and control of systems with a non-probabilistic (set membership) description of uncertainty. (c) A section on infinite horizon continuous-time (semi-Markov) decision problems. (d) A new appendix dealing with the minimax and expected utility approaches for formulating decision problems under uncertainty.