In this chapter, we discuss the least squares/equation error techniques for parameter estimation, which are used for aiding the parameter estimation of dynamic systems (including algebraic systems), in general, and the aerodynamic derivatives of aerospace vehicles from the flight data, in particular. In the first few sections, some basic concepts and techniques of the least squares approach are discussed with a view to elucidating the more involved methods and procedures in the later chapters. Since our approach is model-based, we need to define a mathematical model of the dynamic (or static) system.

In the area of signal processing, we come across analogue and digital filtering concepts and methods. The real-life systems give rise to signals, which are invariably contaminated with the so-called random noise. A usual characteristic of such a random noise that affects the signal is Gaussian (normally distributed) noise with zero mean and some finite variance. This variance measures the power of the noise and it is often compared to the power of the signal that is influenced by the random noise. This leads to a measure called signal to noise ratio (SNR). The aim is then to maximise SNR by filtering out the noise from the signal/data of the dynamical system. There are mainly two approaches: model free and model based. In the model free approach, no mathematical model (equations) is presumed to be fitted or used to estimate the signal from the signal plus noise. These techniques rely upon the concept of the correlation of various signals, like input-output signals and so on. In the present chapter, we use the model based approach and especially the approach based on the state-space model of a dynamical system. Therefore, our major goal is to get the best estimate or prediction of the signal, which is buried, in the random noise. Estimation (of a signal) is a general term. One can make three distinctions in context of an estimate of a signal: filtered, predicted or smoothed estimate. We assume that the data is available up to the time 't'. Then, obtaining the estimate of a signal at the time 't' is called filtering. If we obtain an estimate, say at 't + 1', it is called prediction and if we obtain an estimate at 't-1' by using data up to 't', it is called a smoothed estimate. In this chapter, we mainly study the problem of filtering and prediction using Kalman filtering methods.

The time-series methods have gained considerable acceptance in system identification literature in view of their inherent simplicity and flexibility. These techniques provide external descriptions of systems under study and lead to parsimonious, minimum parameterisation representation of the process. The accurate determination of the dynamic order of the time-series models is a necessary first step in system identification. Many statistical tests are available in the literature which can be used to find the model order for any given process. Selection of a reliable and efficient test criterion has been generally elusive, since most criteria are sensitive to statistical properties of the process. These properties are often unknown. Validation of most of the available criteria has generally been via simulated data. However, these order determination techniques have to be used with practical systems with unknown structures and finite data. It is therefore necessary to validate any model order criterion using a wide variety of data sets from differing dynamic systems. The aspects of time-series/transfer function modelling are included here from the perspective of them being special cases of specialised representations of the general parameter estimation problems. The coefficients of time-series models are the parameters, which can be estimated by using the basic least squares, and maximum likelihood methods discussed in Chapters 2 and 3. In addition, some of the model selection criteria are used in EBM procedure for parameter estimation discussed in Chapter 7, and hence the emphasis on model selection criteria in the present chapter.

There are many real life situations where accurate identification of nonlinear terms (parameters) in the model of a dynamic system is required. In principle as well as in practice, the parameter estimation methods can be applied to nonlinear problems. We recall here that the estimation before modelling approach uses two steps in the estimation procedure and the extended Kalman filter can be used for joint state/parameter estimation. As such, the Kalman filter cannot determine the deficiency or discrepancy in the model of the system used in the filter, since it pre-supposes availability of an accurate state-space model. Assume a situation where we are given the measurements from a nonlinear dynamic system and we want to determine the state estimates. In this case, we use the extended Kalman filter and we need to have the knowledge of the nonlinear function f and h. Any discrepancy in the model will cause model errors that will tend to create a mismatch of the estimated states with the true state of the system. In the Kalman filter, this is usually handled or circumvented by including the process noise term Q. This artifice would normally work well, but it still could have some problems: i) deviation from the Gaussian assumption might degrade the performance of the algorithm; and ii) the filtering algorithm is dependent on the covariance matrix P of the state estimation error, since this is used for computation of Kalman gain K. Since the process noise is added to this directly, as GQG^T term, one would have some doubt on the accuracy of this approach. In fact, the inclusion of the 'process noise' term in the filter does not improve the model, since the model could be deficient, although the trick can get a good match of the states. Estimates would be more dependent on the current measurements. This approach will work if the measurements are dense in time, i.e., high frequency of measurements, and are accurate.

Research in the area of artificial neural networks has advanced at a rapid pace in recent times. The artificial neural network possesses a good ability to learn adaptively. The decision process in an artificial neural network is based on certain nonlinear operations. Such nonlinearities are useful: i) in improving the convergence speed (of the algorithm); ii) to provide more general nonlinear mapping between input-output signals; and iii) to reduce the effect of outliers in the measurements. One of the most successful artificial neural networks is the so-called feed forward neural network. In this chapter first the description of the feed forward neural network and its training algorithms is given. Next, parameter estimation using this approach is discussed. The presentation of training algorithms is such that it facilitates MATLAB implementation. Subsequently, recurrent neural networks are described. Several schemes based on recurrent neural networks are presented for parameter estimation of dynamical systems. Subsequently, the genetic algorithm is described and its application to parameter estimation considered.

A good estimator should possess certain properties in terms of errors in parameter estimation and/or errors in the predicted measurements or responses of the mathematical model thus determined. Since the measured data used in the estimation process are noisy, the parameter estimates can be considered to have some random nature. In fact, the estimates that we would have are the mean of the probability distribution, and hence the estimation error would have some associated covariance matrices. Thus, due to the stochastic nature of the errors, one would want the probability of the estimate being equal to the true value to be 1. We expect an estimator to be unbiased, efficient and consistent - not all of which might be achievable. In this appendix, we collect several properties of signals, matrices, estimators and estimates that would be useful in judging the properties and 'goodness of fit' of the parameter/state estimates and interpreting the results. Many of these definitions, properties and other useful aspects are used or indicated in the various chapters of the book and are compiled in this appendix.