Software agents have emerged as promising tools for addressing complex software engineering tasks. However, existing works oversimplify software development workflows by following the waterfall model. Thus, we propose AgileCoder, a multi-agent system that integrates Agile Methodology (AM) into the framework. This system assigns specific AM roles such as Product Manager, Developer, and Tester to different agents, who then collaboratively develop software based on user inputs. AgileCoder enhances development efficiency by organizing work into sprints, focusing on incrementally developing software through sprints. Additionally, we introduce Dynamic Code Graph Generator, a module that creates a Code Dependency Graph dynamically as updates are made to the codebase. This allows agents to better comprehend the codebase, leading to more precise code generation and modifications throughout the software development process. AgileCoder surpasses existing benchmarks, like ChatDev and MetaGPT, establishing a new standard and showcasing the capabilities of multi-agent systems in advanced software engineering environments.
AgileCoder integrates Agile Methodology into a multi-agent system framework, enabling a collaborative environment where software agents assume specific Agile roles such as Product Manager, Scrum Master, Developer, Senior Developer, and Tester. These agents work together to develop software efficiently and iteratively, simulating a dynamic and adaptive software development team.
In AgileCoder, each agent is not just a participant but a stakeholder in the software development process, engaging in sprints that mimic real-world Agile practices. This setup enhances the adaptability and effectiveness of development workflows, aligning closely with modern software engineering demands. The system is designed to handle the complexities of real-world software projects, supporting incremental development and continuous integration seamlessly.
Key Features of AgileCoder:
We will evaluate the performance of AgileCoder on two types of datasets to assess its effectiveness in generating code for different scenarios:
Results are as below:
Category | Model | Dataset Performance | ||
---|---|---|---|---|
HumanEval | MBPP | |||
LLMs (prompting) | CodeGeeX-13B | 18.9 | 26.9 | |
PaLM Coder-540B | 43.9 | 32.3 | ||
DeepSeeker-33B-Inst | 79.3 | 70.0 | ||
GPT-3.5 Turbo | 60.3 | 52.2 | ||
Claude 3 Haiku | 75.9 | 80.4 | ||
GPT 4 | 80.1 | 80.1 | ||
LLMs-based Agents | with GPT-3.5 Turbo | ChatDev | 61.79 | 74.80 |
MetaGPT | 62.80 | 74.73 | ||
AgileCoder | 70.53 | 80.92 | ||
with Claude 3 Haiku | ChatDev | 76.83 | 70.96 | |
AgileCoder | 79.27 | 84.31 | ||
with GPT 4 | MetaGPT | 85.9 | 87.7 | |
AgileCoder | 90.85 | - |
Metric | ChatDev | MetaGPT | AgileCoder |
---|---|---|---|
Executability | 32.79 | 7.73 | 57.79 |
Entire Running Time (s) | 120 | 48 | 444 |
Avg. Time/Sprint (s) | - | - | 306 |
#Sprints | - | - | 1.64 |
Token Usage | 7440 | 3029 | 36818 |
Expenses (USD) | 0.12 | 0.02 | 0.44 |
#Errors | 6 | 32 | 0 |
@article{nguyen2024agilecoder,
title={AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology},
author={Minh Huynh Nguyen and Thang Phan Chau and Phong X. Nguyen and Nghi D. Q. Bui},
journal={arXiv preprint arXiv:2406.11912},
year={2024}
}