AgentSquare
VerifiedAutomated search for optimal modular LLM agent designs.
What is AgentSquare?
AgentSquare is an open-source implementation that treats LLM agents as compositions of distinct modules. It enables systematic search over design choices to identify high-performing combinations for specific tasks and environments.
The approach works by defining standardized input-output interfaces for each module type, then running automated experiments that mix and evaluate different options. Users can launch searches, run evaluations on supported benchmarks, or integrate their own tasks through provided workflow examples.
It targets AI researchers and developers who want to experiment with agent architectures, contribute new modules, or benchmark designs across environments like household simulations and web interactions.
Capabilities
What you can build with AgentSquare
ALFWorld task solving
Combine modules such as dependency planning, chain-of-thought reasoning, and memory retrieval to complete household instruction-following tasks.
Benchmark evaluation
Run standardized tests on WebShop, M3ToolEval, and ScienceWorld by selecting module sets and base models to measure performance.
Module contribution
Develop and submit new agent components that follow the defined interfaces, allowing them to be included in future automated searches.
Install AgentSquare
git clone https://github.com/tsinghua-fib-lab/AgentSquare.git && pip install -r requirements.txtgit clone https://github.com/tsinghua-fib-lab/AgentSquare.git
conda create -n agentsquare python=3.9.12
conda activate agentsquare
cd AgentSquare
pip install -r requirements.txt- 1Export your OpenAI API key as an environment variable.
- 2Clone the repository and create a conda environment with Python 3.9.12.
- 3Activate the environment and install dependencies from requirements.txt.
- 4Navigate to a task directory such as ALFWorld and execute the provided run script with chosen modules.
- 5For custom searches, move to the search folder and launch agent_search.py.
Works with
AgentSquare: pros & cons
Pros
- +Modular structure allows flexible recombination of agent components.
- +Includes automated search functionality to reduce manual tuning.
- +Provides ready support for multiple established agent benchmarks.
- +Encourages community contributions through clear interface guidelines.
Cons
- –Setup requires separate environment configuration for each benchmark.
- –Depends on external LLM APIs and may incur usage costs during search.
- –New tasks need custom integration following the workflow template.
Frequently asked questions
ALFWorld, WebShop, M3ToolEval, and ScienceWorld are included with example scripts.
User reviews
Verified reviews from the community shape this listing's rating.
Loading reviews…
Similar agents
Other agent frameworks options worth comparing.
