The previous volume Advances in Unmanned Marine Vehicles brought together eighteen chapters describing research and developments in unmanned marine vehicles (UMVs). It was observed that almost without exception research groups worldwide were developing and working on real UMVs which means that they are able to test, evaluate and re-evaluate their designs in relatively quick succession, thereby rapidly reporting new approaches, techniques, designs and successes. This rapid design-evaluation cycle is the prime mover for progress, not only for consolidating designs but also leading to new design ideas and innovation. Since its publication in 2006, Advances in Unmanned Marine Vehicles has proven to be a useful and popular source of reference. However, the rapid design-evaluation cycle means further advances have been made which need to be reported. Thus, the seventeen chapters contained in this volume cover further advances in autonomous underwater vehicles, remotely operated vehicles, semi-submersibles, unmanned surface vessels whilst operating autonomously and/or in co-operation with other types of UMV. This book will be of interest to undergraduates, postgraduates, researchers and industrialists who are involved in the design and development of UMVs.

This book gives an exposition of recently developed approximate dynamic programming (ADP) techniques for decision and control in human engineered systems. ADP is a reinforcement machine learning technique that is motivated by learning mechanisms in biological and animal systems. It is connected from a theoretical point of view with both adaptive control and optimal control methods. The book shows how ADP can be used to design a family of adaptive optimal control algorithms that converge in real-time to optimal control solutions by measuring data along the system trajectories. Generally, in the current literature adaptive controllers and optimal controllers are two distinct methods for the design of automatic control systems. Traditional adaptive controllers learn online in real time how to control systems, but do not yield optimal performance. On the other hand, traditional optimal controllers must be designed offline using full knowledge of the systems dynamics. It is also shown how to use ADP methods to solve multi-player differential games online. Differential games have been shown to be important in H-infinity robust control for disturbance rejection, and in coordinating activities among multiple agents in networked teams. The focus of this book is on continuous-time systems, whose dynamical models can be derived directly from physical principles based on Hamiltonian or Lagrangian dynamics.