On centralized and decentralized decision-making problems with partial information
Date
2023
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
The advent of cyber-physical systems has revolutionized numerous applications, including connected and automated vehicles, medicine and healthcare, the Internet of Things, social media platforms, and robotic swarms. These systems require new approaches that can utilize the improved computational capabilities of the cyber core to optimally control their physical components, while accounting for various forms of incomplete information and uncertain disturbances during real-world implementation. This dissertation primarily focuses on two areas of research: (1) centralized worst-case control and learning with partial observations, and (2) decentralized control of a team of cooperative agents. Additionally, the dissertation presents a mechanism design approach to effectively coordinate the actions of competing agents, specifically in a misinformation filtering problem involving competing social media platforms and a democratic government. ☐ The first contribution of this dissertation is to introduce a general non-stochastic theory of approximate information states, addressing the computational challenges of worst-case control and facilitating worst-case reinforcement learning in partially observed systems. An important feature of the proposed framework is that approximate information states can be constructed using output data in control problems and can be learned from output data in reinforcement learning problems. Then, these states facilitate the efficient computation of approximate control strategies while only conceding a bounded loss in worst-case performance. Thus, our proposed framework provides a principled approach for approximately optimal worst-case control and worst-case reinforcement learning in systems with partially observed states. ☐ The second contribution of this dissertation is towards the theory of decentralized decision-making for teams with one-directional information sharing, in both stochastic and non-stochastic formulations. A general information structure, called nested accessible information, is introduced for teams of two agents with one-directional communication. This information structure is analyzed in the stochastic setting to derive structural results and a dynamic programming decomposition that computes optimal control strategies. Then, this information structure is extended to multiple agents residing within nested subsystems, which is analyzed in the non-stochastic setting. As before, structural results and a dynamic programming decomposition are presented for control strategies that optimize the worst-case performance of the team. The effectiveness of the results is illustrated using a numerical example in the non-stochastic setting. ☐ The third contribution of this dissertation is in the application of a mechanism design approach to a specific problem concerning how a democratic government can incentivize social media platforms to filter misinformation. This problem is modeled by drawing upon theoretical and empirical conclusions from political science and sociology. Then, a mechanism is proposed that takes a fixed budget of the government, creates incentives for social media platforms and distributes these incentives to encourage an optimal level of filtering. Key properties of the proposed mechanism, such as budget balance, voluntary participation of all agents, and implementation of a globally optimal level of filtering given a budget, are derived. ☐ Collectively, this dissertation contributes towards an improved understanding of decision-making in cyber-physical systems with partial information. The theoretical results presented here have potential applications in various cyber-physical systems subject to uncontrolled disturbances, adversarial attacks, and information asymmetry.
Description
Keywords
Decentralized control, Mechanism design, Partial observations, Reinforcement learning, Robust control