Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot


Most evaluation suites for multi-agent reinforcement learning do not assess generalization to novel situations as their primary objective (unlike supervised-learning benchmarks). The subject of this talk, Melting Pot, is an evaluation suite that fills this gap, and is scalable because it uses reinforcement learning to reduce the human labor required to create novel test scenarios. This works because one agent's behavior constitutes (part of) another agent's environment. Melting Pot currently contains 85 unique test scenarios covering a broad range of topics such as social dilemmas, reciprocity, resource sharing, and task partitioning. Melting Pot is open source and free to use for anyone.


Joel Z. Leibo is a research scientist at DeepMind. He obtained his PhD in 2013 from MIT where he worked on the computational neuroscience of face recognition. Nowadays, Joel's research is aimed at the following questions:
How can we get deep reinforcement learning agents to perform complex cognitive behaviors like cooperating with one another in groups?
How should we evaluate the performance of deep reinforcement learning agents?
How can we model processes like cumulative culture that gave rise to unique aspects of human intelligence?

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from Youtube
Consent to display content from Vimeo
Google Maps
Consent to display content from Google