Tackling Credit Assignment Using Memory and Multilevel Optimization for Multiagent Reinforcement Learning
Candidate for Doctor of Philosophy in Robotics
Major Professor: Dr. Kagan Tumer

There is growing commercial interest in the use of multiagent systems in real-world applications. Some examples include inventory management in warehouses, smart homes, extraterrestrial exploration, search and rescue, air-traffic management and autonomous transportation systems. However, multiagent coordination is an extremely challenging problem. Information relevant for coordination is often distributed across the team members and fragmented amongst each agent's observation histories (past states). We leverage memory as a tool in enabling better credit assignment by facilitating associations between rewards and actions separated across time. We achieve this by introducing novel neural network architectures that can reliably retain and propagate information over an extended period of time. We then utilize this capability towards tackling multiagent settings where a team of agents has to rapidly adapt its joint behavior based on a singular observation of one of the agents. Further, designing general mechanisms for generating agent-specific reward functions that incentivizes an agent to collaborate with each other is extremely difficult. We introduce a multilevel optimization framework that hybridizes the strength of an evolutionary algorithm with fast policy gradient methods towards addressing this issue. We apply this framework towards addressing challenging multiagent coordination tasks where what’s good for an individual agent is not always aligned with what’s good for the team.