AI Agents 5 - Planning, Tool Use and Multi-Agent Collaboration

Video Description

This lecture, part of CSE 491 and 895 at Michigan State University, focuses on three pillars of agentic design: planning, tool use, and multi-agent collaboration. The guiding theme is how to implement agentic design patterns, which are reusable structures that help solve recurring problems in the design of intelligent systems. The first section introduces planning. Here, an agent takes a complex goal and breaks it down into smaller, manageable steps. The agent does not just execute instructions but generates possible plans, evaluates them, and then selects the best course of action. For example, if you ask an agent to organize a team offsite, you provide the objective and constraints, while the agent determines the path to get there—considering budgets, dates, and logistics. The slides illustrate this process with a ReAct-style planner, showing how code can decompose a goal into step-by-step execution. The second section turns to tool use. This is where agents interact with external systems such as APIs, databases, and software libraries. The core idea is that the language model at the heart of the agent decides when to call these tools, executes them, and processes the results. One example demonstrates how an agent can query Wikipedia for historical data and then use a Python calculator to compute facts like the lifespan of Alan Turing. The slides include code built with LangChain, showing how to define tools, chain prompts, and connect everything into a working system. The third section explores multi-agent collaboration. Instead of relying on a single agent, the task is distributed among a collection of specialized agents that work together. A research query, for instance, might be divided into information retrieval handled by one agent, statistical processing performed by another, and synthesis of the results by a third. The efficacy of such systems depends on the quality of inter-agent communication. The slides illustrate different collaboration structures—networked, hierarchical, supervised, and custom—and provide a Python example where a researcher agent and an analyst agent coordinate to generate insights. The lecture also highlights recent research on OpenAgent. This system demonstrates how agents can autonomously integrate GitHub repositories as tools. The process involves three phases: searching for relevant repositories by evaluating README files, setting up the environment by cloning repos and installing dependencies, and finally applying the tools to produce outputs relevant to the user’s query. OpenAgent was tested on the OpenAct benchmark, which includes 339 domain-specific tasks across fields like finance, chemistry, and security analysis. The results show that OpenAgent improves performance across most domains, significantly raising pass rates compared to baseline methods. The key message is that intelligent agent systems require more than just a powerful language model. They need structured planning, the ability to extend themselves through tool use, and collaboration between specialized agents. These principles are not only theoretical but also validated in recent research, demonstrating their potential to solve complex real-world problems.