LLM-Based Multi-Agent Systems for Automated Research: A Systematic Review of Architectures, Capabilities, and Evaluation Practices

Abstract

Large language model (LLM) agents are becoming more prevalent with teams organized to suggest hypotheses, to search the literature, to conduct experiments and to write papers with very little human input. The capabilities these multi-agent systems are capable of is widely distributed throughout machine-learning venues, domains journals, and technical reports. This review presents a synthesis of 50 sources published from 2023 to 2025 that address four questions: What is architecture? What are the research-lifecycle aspects covered? What is evaluated? What is failure? Sources were located using keyword search and citation chain, filtered with explicit criteria and coded according to the following categories: contribution type, architectural pattern, research stage, level of validation. There were six architectural patterns identified, the most prevalent of which was role-specialised pipelines. Literature synthesis and idea generation are the most well supported stages, with fewer systems covering experimentation, analysis, and writing, and these systems are mainly validated on machine-learning problems. There are few systems that report laboratory identification of agent-generated proposals, primarily those in chemistry and biology. There is a consistent finding in benchmarks that agents are able to accomplish only a small fraction of the tasks that are typically research-engineering tasks, and failure analyses indicate that a significant number of failures are due to specification, coordination, and verification issues, not just to the inability to do the tasks themselves. We propose a reference architecture that incorporates and implements all of the above requirements for credible automated research, namely: verification-centred design, explicit provenance, calibrated human oversight, and standardised reporting.