Agents Among Us: A Large-Scale Study of LLMs in Multi-Agent Environments

LLMs behaviors and foundational models have been widely studied in isolation, but their performance in multi-agent environments remains underexplored. This project, "Agents Among Us," aims to fill this gap by conducting a large-scale study of LLMs in multi-agent settings. We developed a custom simulation platform that allows us to create diverse environments where multiple agents, powered by different LLMs, can interact, collaborate, and compete.

Demonstration Video

Key Results

• System identifies failure states of agents in social deduction, generative reasoning (50.4% F1) fails to match supervised detection (85% F1) of malicious agents (imposters).
• Game introduces noise disrupting voting capabilities of agents
• Model scale proved to yield minimal improvements in crew voting accuracy
• Smaller models perform only marginally better than random chance voting

Team

• Kevin Kurian - Ph.D Student
• Aryan Patel - Undergraduate Student
• Prof. Christan Grant - Advisor

Code and Data

• Agents Among Us Repo
• Website Demonstration To Be Released Soon