Analysis, synthesis, and learning of network-inspired stochastic finite-valued systems

Abstract

This thesis is concerned with the analysis, synthesis, and learning problems for several classes of network-inspired stochastic finite-valued systems, including finite-field networks (FFNs), Boolean networks (BNs), logical dynamic systems (LDSs), and Markov chains (MCs). They are effective models to describe agents with limited capacities for storing, processing, and transmitting information.

In terms of analysis, leader-follower consensus criteria are explored over the finite fields from the transition graph and characteristic polynomial viewpoints. The former approach discovers that the transition graph of leader-follower consensus FFNs is either a spanning in-tree topped at zero-state or several spanning in-trees with the same structure topped at steady states. The latter approach merely requires polynomial computational complexity with respect to the number of network nodes. Besides, the concept and criteria of observability categorization are first formulated for Boolean control networks (BCNs) based on the graph-theoretic approach and algebraic approach. The observability of BCNs regarding each distinct state pair is categorized into four disjoint classes: distinguishable, transient, primitive, and imprimitive ones.

In terms of synthesis, an investigation of quotient/bisimulation-based scale reduction is conducted for stochastic FFNs and continuous-time (controlled) MCs. The reduced network preserves the essential properties of the original network but has a relatively smaller scale. To save the controller usage, an optimal triggering controller is constructed to stabilize LDSs via the Fibonacci-heap-typed data structure, and its design is proved to be equivalent to finding the minimum-weight arborescence in the transition graph. Moreover, a distributed pinning control scheme is proposed to stabilize Markovian jump BCNs based on the network structure information, which dramatically reduces memory storage. Furthermore, a sampled-data control strategy is designed to stabilize continuous-time probabilistic logical control networks (PLCNs) by setting the minimum average dwell time or restraining the dwell time of each system mode.

In terms of learning, the stabilizability, controllability, and observability of mixed-valued PLCNs are explored via reinforcement learning techniques. First, the bounds of the transition probability are formulated to derive stabilizability, controllability, and observability criteria. Then, the correspondence between stabilizability/controllability/observability probability and the optimal state-value function is proved, which converts the probability calculation to a dynamic programming problem. Finally, model-free approximate estimation algorithms are presented to calculate the probabilities and the optimal control inputs, which relieves the curse of dimensionality and offers a lower time complexity.

In summary, the obtained results in this thesis pave the way for the network-based research of stochastic finite-valued systems. The investigation also provides some approaches to alleviate high computational complexity and large memory storage of the existing results by utilizing bisimulation techniques, exploring network structure information, leveraging data structure storage, and resorting to graph-theoretical and reinforcement learning algorithms. Future research can focus on learning-based controller design for network-based finite-valued systems under more complicated topology variations.