by , , , , , ,
Abstract:
Many applications, e.g., in shared mobility, require coordinating a large number of agents. Mean-field reinforcement learning addresses the resulting scalability challenge by optimizing the policy of a representative agent interacting with the infinite population of identical agents instead of considering individual pairwise interactions. In this paper, we address an important generalization where there exist global constraints on the distribution of agents (e.g., requiring capacity constraints or minimum coverage requirements to be met). We propose Safe-M3-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions. As a key ingredient, it uses epistemic uncertainty in the transition model within a log-barrier approach to ensure pessimistic constraints satisfaction with high probability. Beyond the synthetic swarm motion benchmark, we showcase Safe-M3-UCRL on the vehicle repositioning problem faced by many shared mobility operators and evaluate its performance through simulations built on vehicle trajectory data from a service provider in Shenzhen. Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.
Reference:
Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning M. Jusup, B. Pásztor, T. Janik, K. Zhang, F. Corman, A. Krause, I. BogunovicIn Proc. International Conference on Autonomous Agents and Multiagent Systems (AAMAS), International Foundation for Autonomous Agents and Multiagent Systems, 2024
Bibtex Entry:
@inproceedings{jusup2024safe,
	address = {Richland, SC},
	author = {Jusup, Matej and P{\'a}sztor, Barna and Janik, Tadeusz and Zhang, Kenan and Corman, Francesco and Krause, Andreas and Bogunovic, Ilija},
	booktitle = {Proc. International Conference on Autonomous Agents and Multiagent Systems (AAMAS)},
	isbn = {9798400704864},
	location = {Auckland, New Zealand,},
	month = {May},
	numpages = {10},
	pdf = {https://arxiv.org/pdf/2306.17052.pdf},
	publisher = {International Foundation for Autonomous Agents and Multiagent Systems},
	title = {Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning},
	year = {2024}}