Continuous-Time Markov Networks: From Merging Rules to Synchronizing Control

Abstract

This paper characterizes the state space representation and control Lyapunov theory for continuous-time Markov networks (CT-MNs), which constitute a family of Markov chains endowed with network structures, serving as the theoretical basis for the investigations of several swarm behaviors. First, the notions of coupling Markov property and conditional independent property are introduced, enabling a CT-MN to be merged as a decentralized continuous-time Markov chain (CT-MC). Then, explicit formulas for the transition rate matrix of the augmented CT-MCs are derived. Consequently, key problems in CT-MNs can be reformulated as set-stability or set-stabilization problems for the augmented CT-MCs, with feedback controllers designed via linear programming subject to Lyapunov-based stability constraints. Furthermore, master slave synchronization and output regulation of CT-MNs are addressed. Finally, numerical examples validate the theoretical results and demonstrate their effectiveness in practical applications.