Mean Field LQG Social Optimization: A Reinforcement Learning Approach
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918191243984896 |
|---|---|
| author | Xu, Zhenhui Wang, Bing-Chang Shen, Tielong |
| author_facet | Xu, Zhenhui Wang, Bing-Chang Shen, Tielong |
| contents | This paper presents a novel model-free method to solve linear quadratic Gaussian mean field social control problems in the presence of multiplicative noise. The objective is to achieve a social optimum by solving two algebraic Riccati equations (AREs) and determining a mean field (MF) state, both without requiring prior knowledge of individual system dynamics for all agents. In the proposed approach, we first employ integral reinforcement learning techniques to develop two model-free iterative equations that converge to solutions for the stochastic ARE and the induced indefinite ARE respectively. Then, the MF state is approximated, either through the Monte Carlo method with the obtained gain matrices or through the system identification with the measured data. Notably, a unified state and input samples collected from a single agent are used in both iterations and identification procedure, making the method more computationally efficient and scalable. Finally, a numerical example is given to demonstrate the effectiveness of the proposed algorithm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15119 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mean Field LQG Social Optimization: A Reinforcement Learning Approach Xu, Zhenhui Wang, Bing-Chang Shen, Tielong Optimization and Control This paper presents a novel model-free method to solve linear quadratic Gaussian mean field social control problems in the presence of multiplicative noise. The objective is to achieve a social optimum by solving two algebraic Riccati equations (AREs) and determining a mean field (MF) state, both without requiring prior knowledge of individual system dynamics for all agents. In the proposed approach, we first employ integral reinforcement learning techniques to develop two model-free iterative equations that converge to solutions for the stochastic ARE and the induced indefinite ARE respectively. Then, the MF state is approximated, either through the Monte Carlo method with the obtained gain matrices or through the system identification with the measured data. Notably, a unified state and input samples collected from a single agent are used in both iterations and identification procedure, making the method more computationally efficient and scalable. Finally, a numerical example is given to demonstrate the effectiveness of the proposed algorithm. |
| title | Mean Field LQG Social Optimization: A Reinforcement Learning Approach |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2410.15119 |