Agent-based Systems for Software Development
Learning Objectives
- You know of agent-based systems for software development.
- You know that agents can be used to create software engineering teams.
Agents are programs that act on behalf of a user or another program. In the context of software applications, agents include chatbots, personal assistants, and other programs — as an example, an agent could be a program that helps you with your calendar, or a program that helps you with your email.
In the context of automated software engineering, an agent could be a program that retrieves GitHub issues, generates code to fix issues, and then creates pull requests that contain the issues. Generating the code to fix the issues could involve testing the code and iteratively making adjustments, seeking for changes that fix the issue. For this to work, the agent would require access to functionality for reading GitHub issues and retrieving code from GitHub, modifying files, running tests, and creating (and pushing) pull requests.
In essence, an automated software engineering agent needs an interface that allows interacting with the computer system.
As a concrete example of such agent, we look briefly to the SWE-agent system from the team behind SWE-bench. The SWE-agent system has an agent-computer interface that allows performing actions on a computer system based on the suggestions of a large language model -based agent (Fig. 1).
Building such a system requires deciding the available commands and the action/command -mappings for the agent-computer interface, a sandboxed computer system, and the concrete prompts that the agent uses to decide on the next actions to concretely provide the actions. The source code of SWE-agent is available on GitHub — the system can also be tried out through a web user interface on your own computer.
For additional details on the SWE-agent system, see the article “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering”.
Beyond working on GitHub issues, the concept of an agent can also be used to create software engineering teams. Here, the idea would be to have multiple types of agents, each having a specific role in the software engineering team. This approach was highlighted in an article called “Communicative Agents for Software Development”, where the authors demonstrated the possibility of building software with a team of agents consisting of a a variety of roles, such as “chief product owner”, “chief technology officer”, “programmer”, “reviewer”, and “tester”.
By creating a chat chain that allowed decomposing the development process into small tasks that could be collaborated on by two (or more) agents, the article authors demonstrated the possibility of creating working software based on an initial idea. In their experiments, when creating a simple game, 86% of the resulting software systems could be executed without errors. The code for the system is available on GitHub at https://github.com/OpenBMB/ChatDev.
While building agent-based systems are out of scope of this course, if you’re interested, the Hugging Face Transformers library provides tools for interacting with large language models. The project also has functionality for building agents that could be used to interact with large language models and other systems.