Adaptive Optimization in Stochastic Systems via Fiducial Approach

In this paper, the problem of determining the optimal control law for discrete-time stochastic linear systems with respect to a quadratic performance criterion is considered. It is assumed that the system is subject to additive system noise and that the state variables are measured with additive measurement noise, without specifying the specific characteristics of random variables. It is shown that the problem of stochastic optimal control can be reduced to two independent problems, one of equivalent deterministic optimal control and the other of stochastic estimation of underlying uncertainties. This holds even if the system noise, the measurement noise and/or the initial state of the system are non-Gaussian, mutually and time-wise dependent. The aim of the present paper is to show how the invariant embedding technique and fiducial approach may be used to solve the problem of adaptive cautious controlling a discrete-time stochastic linear system in which the state transition matrix and the control driven matrix are unknown. This is the case when the certainty equivalence principle does not yield the admissible adaptive control laws for the present problem. The proposed approach does not require the arbitrary selection of priors as in the Bayesian approach. It makes it possible to simplify the problem of adaptive optimization of stochastic systems and, if the system noise and/or the measurement noise are Gaussian, to carry out the algorithm in closed form. The examples are given to illustate the suggested methodology.