Journal's Facebook pageA Method for Solving Reconfiguration Blueprints Based on Multi-Agent Reinforcement Learning
- School of Computer Science and Engineering, Xi’an Technological
University, Xi’an 710016, China
chengjing@xatu.edu.cn, tanwen@st.xatu.edu.cn - Xi’an Institute of Aeronautical Computing Technology (XICT),
Xi’an 710065, China
guangzhe lv@163.com - Aviation Industry First Aircraft Design and Research Institute,
Xi’an 710089, China
guodonglee@163.com - School of Software, Northwestern Polytechnical University,
Xi’an 710129, China
wentaoz1223@gmail.com - Leihua Electronic Technology Research Institute of Aviation
Industry Corporation of China (AVIC), Wuxi, China
2020264414@mail.nwpu.edu.cn
Abstract
Integrated modular avionics systems primarily achieve system fault tolerance by reconfiguring the system configuration blueprints. In the design of manual reconfiguration, the quality of reconfiguration blueprints is influenced by various unstable factors, leading to a certain degree of uncertainty. The effectiveness of reconfiguration blueprints depends on various factors, including load balancing, the impact of reconfiguration, and the time required for the process. Solving high-quality reconfiguration configuration blueprints can be regarded as a type of multi-objective optimization problem. Traditional algorithms have limitations in solving multi-objective optimization problems. Multi-Agent Reinforcement Learning (MARL) is an important branch in the field of machine learning. It enables the accomplishment of more complex tasks in dynamic real-world scenarios through interaction and decision-making. Combining Multi-Agent Reinforcement Learning algorithms with reconfiguration techniques and utilizing MARL methods to generate blueprints can optimize the quality of blueprints in multiple ways. In this paper, an Improved Value-Decomposition Networks (VDN) based on the average sequential cumulative reward is proposed. By refining the characteristics of the integrated modular avionics system, mathematical models are developed for both the system and the reconfiguration blueprint. The Improved VDN algorithm demonstrates superior convergence characteristics and optimization effects compared with traditional reinforcement learning algorithms such as Q-learning, Deep Q-learning Network (DQN), and VDN. This superiority has been confirmed through experiments involving single and continuous faults.
Key words
Integrated modular avionics system, Multi-Agent Reinforcement Learning, reconfiguration blueprint, multi-objective optimization problem
Digital Object Identifier (DOI)
https://doi.org/10.2298/CSIS231129035C
Publication information
Volume 21, Issue 4 (September 2024)
Year of Publication: 2024
ISSN: 2406-1018 (Online)
Publisher: ComSIS Consortium
Full text
Available in PDF
Portable Document Format
How to cite
Cheng, J., Tan, W., Lv, G., Li, G., Zhang, W., Liu, Z.: A Method for Solving Reconfiguration Blueprints Based on Multi-Agent Reinforcement Learning. Computer Science and Information Systems, Vol. 21, No. 4, 1335–1357. (2024), https://doi.org/10.2298/CSIS231129035C
