This study presents a unified resilient model-free reinforcement learning (RL) based distributed control protocol for leader-follower multi-agent systems. Although RL has been successfully used to learn optimal control protocols for multi-agent systems, the effects of adversarial inputs are ignored. It is shown in this study, however, that their adverse effects can propagate across the network and impact the learning outcome of other intact agents. To alleviate this problem, a unified RL-based distributed control frameworks is developed for both homogeneous and heterogeneous multi-agent systems to prevent corrupted sensory data from propagating across the network. To this end, only the leader communicates its actual sensory information and other agents estimate the leader' state using a distributed observer and communicate this estimation to their neighbours to achieve consensus on the leader state. The observer cannot be physically affected by any adversarial input. To further improve resiliency, distributed control protocols are designed to attenuate the effect of the adversarial inputs on the compromised agent itself. An off-policy RL algorithm is developed to learn the solutions of the game algebraic Riccati equations arising from solving the control problem. No knowledge of the agent's dynamics is required and it is shown that the proposed RL-based control protocol is resilient against adversarial inputs.

A two-degree-of-freedom consensus scheme is investigated for fractional-order multi-agent systems with time delay. The two-degree-of-freedom protocol including the individual controllers and the coupling controllers can realise the global tasks and improve consensus performance. Based on such a protocol, a closed-loop multi-input multi-output framework in frequency domain is established. Then the multi-agent system is decoupled into several subsystems with respect to the eigenvalues of the Laplacian matrix. For each subsystem, the robust stabilising region of the individual controller satisfying index is obtained on the basis of the improved D-partition method. The controller parameters chosen in the region can provide robustness for the individual agent. The parameters of the coupling controller that ensure fast consensus are determined by first deriving the stabilising region and then searching for the fastest consensus speed in the obtained region. The two-degree-of-freedom structure can trade off the consensus performance and disturbance rejection and is applicable to arbitrary-order multi-agent systems with time delay including the fractional-order system and the integer-order system. The two-degree-of-freedom consensus controller satisfying other performance criteria can also be designed based on the resultant stabilising region. The simulation results verify the effectiveness of the proposed method.

In dual-rate rational systems, some output data are missing (unmeasurable) to make the traditional recursive least squares (RLS) parameter estimation algorithms invalid. In order to overcome this difficulty, this study develops a bias compensation RLS algorithm for estimating the missing outputs and then the model parameters. The algorithm based on auxiliary model and particle filter has four steps: (i) to establish an auxiliary model to estimate unmeasurable outputs, (ii) to compensate bias induced by correlated noise, (iii) to add a filter to improve estimation accuracy of the unmeasurable outputs and (iv) to obtain an unbiased parameter estimation. Three examples are selected for simulation demonstrations to give further guarantees on the usefulness of the proposed algorithms. The comparative studies show that the bias compensation RLS is more effective for such systems with dual-rate input and output data.

This study develops a numerical method to find optimal ergodic (long-run average) dividend strategies in a regime-switching model. The surplus process is modelled by a regime-switching process subject to liability constraints. The regime-switching process is modelled by a finite-time continuous-time Markov chain. Using the dynamic programming principle, the optimal long-term average dividend payment is a solution to the coupled system of Hamilton–Jacobi–Bellman equations. Under suitable conditions, the optimal value of the long-term average dividend payment can be determined by using an invariant measure. However, due to the regime switching, getting the invariant measure is very difficult. The objective is to design a numerical algorithm to approximate the optimal ergodic dividend payment strategy. By using the Markov chain approximation techniques, the authors construct a discrete-time controlled Markov chain for the approximation, and prove the convergence of the approximating sequences. A numerical example is presented to demonstrate the applicability of the algorithm.

This study aims at exploring numerical stability properties of software sensors used in chemical and other engineering. These are utilised for evaluation of variables and/or parameters of plants, which are not measurable by technical devices. Software sensors are often grounded in the extended Kalman filtering (EKF) technique. A conventional continuous-discrete stochastic system consists of an It-type stochastic differential equation representing the plant's dynamics and a discrete-time equation linking the model's state to measurements. Here, the authors focus on the numerical stability of EKF-type methods, which are applicable to ill-conditioned stiff stochastic models arisen in applied science and engineering. They explore filters' accuracies when the inverse matrices are replaced with the Moore-Penrose pseudo-inverse ones in their measurement updates. This investigation is fulfilled within the authors' ill-conditioned stochastic Oregonator scenario and evidences that the pseudo-inversion indeed resolves many performance problems in some non-square-root methods when the stochastic system is sufficiently ill-conditioned. However, it fails to improve the accuracy in the mildly ill-conditioned case. Eventually, only the square-root nested implicit Runge-Kutta-based filters are found out to be accurate and robust in their examination and, hence, to be the methods of choice.

The Laplacian controllability of combined graphs are studied. It is shown that an antiregular graph, namely, a connected simple graph with exactly two vertices having the same degree, can be connected to another antiregular graph by a newly added edge, and becomes a Laplacian controllable graph under one controller. The methods to identify the vertices connected by the new edge and to assign the control vectors that render the resulting graph Laplacian controllable are presented. Numerical examples are given to illustrate the authors work.

This study is concerned with the mixed control problem under an open-loop information pattern with input delay. Compared with the delay-free case, one of the main difficulties is the non-causality caused by the time delay in the control input. The mixed control problem is reformulated as a leader–follower game and the Stackelberg strategy based on the maximum principle is introduced to deal with this. By introducing two co-states to capture the future information and one new state to capture the past effects, the problem can be addressed. The controllers are designed by solving the symmetric and decoupled Riccati equations. Necessary and sufficient conditions that guarantee the existence and uniqueness of the solution are proposed in this study.

For multisensor descriptor systems with uncertain noise variances, the weighted measurement fusion (WMF) reduced-order robust estimators are presented. The multisensor descriptor systems with known upper bounds of uncertain noise variances are defined as the worst-case conservative multisensor descriptor systems. For the worst-case conservative multisensor descriptor systems, the WMF reduced-order conservative Kalman estimators, including filter, predictor and smoother, are obtained, by applying the WMF method and the singular value decomposition method. However, the measurements of the conservative multisensor descriptor systems are unavailable. Replacing these unavailable conservative measurements by the actual and available measurements yields the actual WMF reduced-order robust estimators. Their robustness is proven by the dynamic error variance system method, i.e. the actual estimation error variances are not greater than those of conservative systems. Two simulation examples of four-sensor descriptor systems show the effectiveness of the presented algorithms.

This study investigates the distributed consensus control problem of multi-agent systems impacted by the combined uncertainty of measurement noises and communication delays under directed fixed topology. Each agent can only obtain the state information of its own and neighbours, which is delayed and interfered by measurement noises. The intensity of the noise is considered as a function related to the relative state of the agent. Under general networks with directed fixed topology, a distributed consensus protocol is proposed to obtain the pth moment linear -consensus of the multi-agent system. The sufficient condition for the consensus of the multi-agent system is derived under the arbitrary upper bound of the time-delay, in which both relative state-dependent measurement noises and time-varying time delays are taken into account. The effectiveness of the proposed consensus protocol is shown by some simulations.

In this study, based on tensor decomposition and matrix product, the authors investigate two model-order reduction (MOR) methods for the quadratic-bilinear (QB) systems which are equivalently transformed from the non-linear input–output systems. Since the quadratic term coefficient of the QB system can be considered as the matricisation of a tensor, they propose two computationally efficient ways to obtain the reduced system by using tensor calculus. First, the Tucker decomposition of tensors is used to deal with the quadratic term coefficient of the QB system. The transformational matrix is constructed by applying the analysis of matrix product. Then, they get the reduced QB system which can match the first several expansion coefficients of the original output. Besides, they propose another MOR method based on the CANDECOMP/PARAFAC decomposition. These two methods can avoid large computational complexity in the process of computing the reduced system. Moreover, the error estimation and stability of the authors' MOR methods are discussed. The efficiency of their MOR methods is illustrated by two numerical examples and they show their competitiveness when compared to the proper orthogonal decomposition method.

Average dwell time (ADT) of switched systems is dependent on two parameters: the decay coefficient of the subsystems and the increase coefficient of the Lyapunov functions of different subsystems, namely, and , respectively. In the literature, the two parameters are prescribed because popular computation techniques cannot deal with bilinear conditions easily. A new method for the computation of ADT is presented in this study, where a suggested optimisation algorithm is proposed to obtain a loose ADT condition. The advantages of the method lie in twofold: (i) the parameters and do not need to be given, and (ii) the presented ADT conditions are tractable to solve. The method is also extended to mode-dependent ADT (MDADT), ADT in asynchronous control, and ADT and MDADT of positive switched systems.

How to find the minimum number of control agents needed to guarantee the strategy consensus of networked evolutionary games (NEGs) is studied. First, the minimal-agent consensus control (MACC) problem is formulated, and given a set of control agents, a necessary and sufficient condition is presented for the consensus of NEGs based on the algebraic form. Second, using the necessary and sufficient condition, an algorithm is established to solve the MACC problem. Third, the consensus of NEGs with switching topology is considered, and a necessary and sufficient condition is presented by introducing the concept of switching reachability. Finally, three illustrative examples are given to show the effectiveness of the obtained results.

In this brief, a novel leader-following appointed-time consensus control protocol is investigated for second-order multi-agent systems on a directed graph from the prescribed performance perspective. Through developing a new state-independent performance function, the practically fixed-time stability or expected appointed-time tracking accuracy of the leader-following network systems can be achieved, wherein, the state-independent setting time can be arbitrarily specified a priori. Compared with the existing works, the primary advantage of the proposed control protocol is that no fractional power state feedback is used and the transient behaviours including the appointed-time convergence rate, overshoot and undershoot can be synthesised simultaneously. Moreover, the excellent disturbance suppression capacity can be achieved just through adjusting a few parameters involved in the newly developed performance controller. Finally, two groups of illustrative examples are employed to validate the effectiveness of the proposed control protocol.

In this study, a new algorithm is proposed for distributed asynchronous consensus-based blind calibration of sensor networks with noisy communications and measurements. The algorithm consists of one autonomous recursion of the instrumental variable type for gain correction and one additional recursion of gradient type for offset correction based on the corrected gains. It is proved using asynchronous stochastic approximation arguments that the algorithm achieves asymptotic consensus with regard to both the corrected sensor gains and offsets in the mean square sense and with probability one. The algorithm is more flexible than the existing similar algorithms for blind macro-calibration and provides a superior convergence rate, especially when used in networks with one fixed reference node. Simulation results confirm the main theoretical statements.