Guide to Building Multi Agent Systems Based on CrewAI

1. Introduction

In the field of Artificial Intelligence, Multi Agent system is becoming a key technology for solving complex problems and realizing efficient collaboration, and CrewAI, as a powerful Multi Agent collaboration tool, provides a convenient way for developers to build intelligent collaboration systems. In this article, we will introduce how to build a Multi Agent system based on CrewAI.

2. CrewAI core concepts in detail

2.1 Agent

2.1.1 Definition and Function of Agent

Agents are autonomous units in CrewAI with the ability to perform tasks, make decisions, and communicate with other agents. They act as members of a team, each taking on a specific role, such as researcher, writer, or customer support, and together they contribute to the realization of the team's goals.

2.1.2 Proxy properties

• Role : Clarify the agent's functional position within the team and determine the types of tasks it is best suited to perform. For example, agents in the researcher role specialize in gathering and analyzing information, while agents in the writer role focus on creating content.
• Goal : The decision-making process that guides an agent is an individual goal that the agent strives to achieve. For example, the goal of a data analyst agent might be to extract actionable insights to support business decisions.
• Backstory : Enhance the dynamics of interaction and collaboration by providing rich contextual information about the agent's role and goals. In the case of a data analyst, for example, the backstory could be "You are a data analyst at a large company. You are responsible for analyzing data and providing insights to the business. You are currently working on a project to analyze the performance of our marketing campaigns."
• LLM (optional) : Indicates the language model for running the agent. If not specified, the default is to run the agent from the OPENAI_MODEL_NAME Get the model name in the environment variable, otherwise it defaults to "gpt-4". Developers can choose the appropriate language model to meet the processing requirements of different tasks.
• Tools (optional) : A set of capabilities or feature sets available to an agent, usually instances of custom classes compatible with the execution environment, with an empty list of default values. These tools can extend the capabilities of an agent to enable it to perform operations such as web searching, data analysis, and so on.
• LLM for call functions (optional) : Specifies the language model that handles this agent tool call, overriding the team function call LLM if passed, defaults to None. By flexibly configuring this property, developers can more precisely control the agent's tool invocation behavior.
• Maximum number of iterations (optional) : The maximum number of iterations the agent can perform before it is forced to give the best answer, the default value is 25. Setting this parameter appropriately will help control the depth and efficiency of the task execution.
• Maximum number of requests (optional) : The maximum number of requests the proxy can execute per minute, used to avoid rate limiting, the default value is None. The developer can make settings according to the actual situation to ensure stable operation of the system.
• Maximum execution time (optional) : The maximum length of time the agent can perform a task, the default value is None, indicates that there is no maximum execution time limit. This attribute can be used to control the execution period of a task to prevent it from taking up resources for a long time.
• Detailed mode (optional) : Set it to True Configurable internal logger provides detailed execution logs for debugging and monitoring, default value is False. Turning on Detailed Mode helps to quickly locate problems during development and optimization.
• Permission to delegate (optional) : Agents can delegate tasks or issues to each other, ensuring that tasks are handled by the most appropriate agent, with a default value of True. This feature promotes flexible collaboration within the team and improves the accuracy and efficiency of task processing.
• Step callback (optional) : Functions called after each step of the agent, which can be used to log operations or perform other operations, will override the team's step_callback. Developers can use this callback function to implement custom monitoring and processing logic.
• Cache (optional) : Indicates whether the proxy uses the tool-use cache, with a default value of True. The caching mechanism helps to increase the efficiency of task execution and reduce duplicate computations.

2.1.3 Creating Agents

Agents can interact with each other using CrewAI's built-in delegation and communication mechanisms. This allows for dynamic task management and problem solving within teams.

To create a proxy, it is common to initialize it with the required properties Agent instance of the class. The following is an example of a concept with all the attributes:

from crewai import Agent
agent = Agent(
role="Data Analyst",
goal="Extract actionable insights",
backstory="""You are a data analyst at a large company.
You are responsible for analyzing data and providing insights to the business.
You are currently working on a project to analyze the performance of our marketing campaigns."""" ,
tools=[my_tool1, my_tool2], # Optional, defaults to empty list
llm=my_llm, # Optional
function_calling_llm=my_llm, # optional
max_iter=15, # optional
max_rpm=None, # optional
verbose=True, # optional
allow_delegation=True, # optional
step_callback=my_intermediate_step_callback, # optional
cache=True # optional
)

2.2 Tasks

2.2.1 Task definition and collaboration

In the CrewAI framework, a task is a specific job that an agent needs to accomplish, containing various details required for execution, such as a clear description, the agent responsible for execution, and the required tools. Tasks can be collaborative, and through attribute management and process orchestration, multiple agents can work together to improve the efficiency of teamwork.

2.2.2 Attributes of tasks

• Description : A clear and concise statement of the content of the task so that the agent knows exactly what needs to be done. For example, "Find and summarize the latest and most relevant news about artificial intelligence".
• Agent : Agents responsible for task execution can be specified directly or assigned by the Crew's processes based on roles, availability, and other factors. This ensures that tasks find the right executor.
• Expected Output : A detailed description of the task once it has been completed so that the agent is clear about the goal and desired outcome of the task. For example, "Summarize the bulleted list of the top 5 most important AI news items".
• Tools (optional) : Functions or capabilities used by agents to perform tasks can enhance task performance and agent interaction. By choosing the right tools, agents can accomplish tasks more efficiently.
• Asynchronous execution (optional) : The task will be executed asynchronously when set, allowing other tasks to continue to advance without waiting for completion, for tasks that take a long time or have little impact on the execution of subsequent tasks.
• Context (optional) : Specify the tasks whose output is to be used as the context of this task, clarify the dependencies between tasks, and achieve effective delivery and utilization of task output.
• Configuration (optional) : Additional configuration details of the agent used to perform the task provide the flexibility to further customize the task execution.
• Output JSON (optional) : To output JSON objects, OpenAI API is required and only one output format can be set. This property facilitates data interaction and integration with other systems.
• Output Pydantic (optional) : To output Pydantic model objects, OpenAI API is also required, and only one output format can be set. Developers can choose the appropriate output format according to their actual needs.
• Output file (optional) : Saves the task output to a file, if compared with the Output JSON maybe Output Pydantic Used together, you can specify how the output is saved. This facilitates data persistence and subsequent analysis.
• Callback (optional) : A Python callable function that is executed after a task is completed and can be used to trigger actions or notifications based on the results of the task, such as sending an email.
• Manual input (optional) : Indicates whether the task requires eventual manual feedback, useful for tasks that require manual supervision to ensure the accuracy and reliability of the task results.

2.2.3 Using tools to create tasks

After understanding the basic properties of a task, we further explore how to create a task using the tool. Below is an example:

import os
os.environ["OPENAI_API_KEY"] = "Your Key"
os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key
from crewai import Agent, Task, Crew
from crewai_tools import SerperDevTool
research_agent = Agent(
role="Researcher",
goal="Find and summarize the latest AI news",
backstory="""You are a researcher at a large company.
You are responsible for analyzing data and providing insights for the business.""" ,
verbose=True
)
search_tool = SerperDevTool()
task = Task(
description="Find and summarize the latest AI news",
expected_output="Summarize a bulleted list of the top 5 most important AI news stories", agent=research_agent
agent=research_agent,
tools=[search_tool]
)
crew = Crew(
agents=[research_agent],
tasks=[task],
verbose=2
)
result = crew.kickoff()
print(result)

2.2.4 Creating tasks

Creating a task involves defining its scope, the responsible agent, and any additional attributes for flexibility:

from crewai import Task
task = Task(
description="Find and summarize the latest and most relevant news about Artificial Intelligence.
agent=sales_agent
)

Task assignments can be assigned directly to the assignment by specifying a act on behalf of sb. in a responsible positionOr let lamination The CrewAI process is determined based on roles, availability, etc.

2.2.5 Task dependencies

In CrewAI, the output of one task is automatically passed on to the next, which can be useful when there is a task that depends on the output of another task, but is not immediately executed after it. This is accomplished through the task's (textual) context Attribute Completion:

research_ai_task = Task(
description="Find and summarize the most recent AI news",
expected_output="Summarize a bulleted list of the top 5 most important AI news items",
async_execution=True, agent=research_agent
agent=research_agent,
tools=[search_tool]
)
research_ops_task = Task(
description="Find and summarize the latest news on AI operations",
expected_output="Summarize a bulleted list of the top 5 most important AI operations news",
async_execution=True,
agent=research_agent,
tools=[search_tool]
)
write_blog_task = Task(
description="Write a full blog post on the importance of AI and its latest news",
expected_output="4-paragraph long full blog post",
agent=writer_agent,
context=[research_ai_task, research_ops_task]
)

2.2.6 Asynchronous execution

Tasks are executed asynchronously, meaning that the Crew does not wait for them to complete before moving on to the next task. This is useful for tasks that take a long time to complete or are not critical to the execution of the next task. You can use the (textual) context attribute is defined in a future task, it should wait for the output of the asynchronous task to complete.

list_ideas = Task(
description="Explore 5 interesting ideas for articles about artificial intelligence." ,
expected_output="A bulleted list of 5 article ideas." ,
agent=researcher,
async_execution=True # will be executed in an asynchronous manner
)
list_important_history = Task(
description="Research the history of artificial intelligence and tell me the 5 most important events." ,
expected_output="A bulleted list containing the 5 important events." ,
agent=researcher,
async_execution=True # will be executed asynchronously
)
write_article = Task(
description="Write an article about artificial intelligence, its history and interesting ideas." ,
expected_output="A 4-paragraph article about artificial intelligence." ,
agent=writer,
context=[list_ideas, list_important_history] # will wait for the output of the two tasks to complete
)

2.2.7 Callback mechanisms

Executes a callback function upon task completion, allowing actions or notifications to be triggered based on task results.

def callback_function(output: TaskOutput).
# Perform some action after task completion
# e.g. send an e-mail to a manager
print(f"""
Task completion!
Task: {output.description}
Output: {output.raw_output}
""")
research_task = Task(
description="Find and summarize the latest AI news",
expected_output="Summarize a bulleted list of the top 5 most important AI news items",
agent=research_agent,
tools=[search_tool],
callback=callback_function
)

2.2.8 Accessing task-specific output

Once a group of Crews has run, you can run them by using the task object's output attribute accesses the output of a specific task:

task1 = Task(
description="Find and summarize the most recent AI news",
expected_output="Summarize bulleted list of top 5 most important AI news",
agent=research_agent,
tools=[search_tool]
)
crew = Crew(
agents=[research_agent],
tasks=[task1, task2, task3],
task=[task1, task2, task3], verbose=2
)
result = crew.kickoff()
print(f"""
Task completion!
Task: {task1.output.description}
Output: {task1.output.raw_output}
""")

2.2.9 Tool override mechanisms

Specifying tools in a task allows for dynamic adjustment of agent capabilities, highlighting the flexibility of CrewAI.

2.2.10 Error handling and validation mechanisms

Validation mechanisms exist to ensure the robustness and reliability of task attributes when creating and executing tasks. These validations include but are not limited to:

• Ensure that only one output type is set per task to maintain clear output expectations.
• Prevention of manual distribution id attribute to maintain the integrity of the unique identifier system.

These validations help maintain consistency and reliability of task execution within the CrewAI framework.

2.3 Tools

2.3.1 Role and types of tools

The CrewAI tool empowers agents with a variety of capabilities, including web search, data analysis, content generation, and task delegation, enabling agents to perform a wide range of complex operations that leapfrog from simple searches to complex interactions and effective teamwork.

2.3.2 Key features of the tool

• Utility : Designed specifically for the task, it covers the functional requirements in several areas, such as web search, data analysis, content generation, etc., to meet the needs of agents working in different scenarios.
• Integration : The ability to seamlessly integrate into workflows and work closely with agents and tasks to enhance the overall capabilities of agents and enable efficient collaboration.
• Customizability : Provides the flexibility for developers to either develop custom tools to meet specific needs, or to leverage existing tools and personalize the configuration to the actual needs of the agent.
• Error Handling : A powerful error handling mechanism ensures that the tool handles exceptions gracefully when they are encountered during operation, ensuring the smooth running of the system.
• Caching Mechanism : Features intelligent caching to optimize performance and reduce redundant operations. Developers can also use the cache_function attribute provides finer control over the caching mechanism, further improving efficiency.

2.3.3 Using the CrewAI Tool

To enhance the capabilities of the agent with the CrewAI tool, you need to install the additional toolkit first:

pip install 'crewai[tools]'

Below is a use case:

import os
from crewai import Agent, Task, Crew
# import CrewAI tools
from crewai_tools import (
DirectoryReadTool, FileReadTool, CrewAI_tools
FileReadTool,
SerperDevTool, WebsiteSearchTool
WebsiteSearchTool
)
# Setting the API Key
os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key
os.environ["OPENAI_API_KEY"] = "Your Key"
# instantiation tool
docs_tool = DirectoryReadTool(directory='. /blog-posts')
file_tool = FileReadTool()
search_tool = SerperDevTool()
web_rag_tool = WebsiteSearchTool()
# Creating an Agent
researcher = Agent(
role="Market Research Analyst",
goal="Provide up-to-date market analysis on the AI industry",
backstory="An expert analyst with a keen eye on market trends." ,
tools=[search_tool, web_rag_tool],
verbose=True
)
writer = Agent(
role="Content Writer", , goal="Write engaging blog posts about the AI industry", verbose=True )
goal="Writes engaging blog posts about the AI industry",
backstory="A skilled writer with a passion for technology." ,
tools=[docs_tool, file_tool],
verbose=True
)
# Define the task
research = Task(
description="Research and provide a summary of the latest trends in the AI industry." ,
expected_output="A summary of the top three hot developments in the AI industry and provide a unique perspective on their importance." ,
agent=researcher
)
write = Task(
description="Write an engaging blog post based on the research analyst's summary. Draw inspiration from the latest blog posts in the catalog." ,
expected_output="A four-paragraph blog post laid out in markdown format that is engaging, informative, and easy to understand, avoiding complex terminology." ,
agent=writer,
output_file='blog-posts/new_post.md' # The final blog post will be saved here
)
# Assembling the Crew
crew = Crew(
agents=[researcher, writer],
tasks=[research, write],
verbose=2
)
# Execute the task
crew.kickoff()

2.3.4 Creating your own tools

The toolkit needs to be installed first:

pip install 'crewai[tools]'

• subclassification BaseTool : by inheriting BaseTool class, developers can create custom tools. You need to define the name and description of the tool and implement the _run method to define the specific functional logic of the tool. Example:

from crewai_tools import BaseTool
class MyCustomTool(BaseTool).
name: str = "Name of my tool"
description: str = "Clear description of what this tool is used for, your agent will need this information to use it."
def _run(self, argument: str) -> str.
# Implementation here
return "The result of customizing the tool"

• utilization tool decorator : Use tool Decorators allow for more concise creation of custom tools. Simply define the function, add the @tool decorator, and just provide the tool name and description. Example:

from crewai_tools import tool
@tool("Name of my tool")
def my_tool(question: str) -> str.
"""Clearly describes what this tool is used for, and your agent will need this information to use it.""""
# The function logic is here
return "The result of your customized tool"

• Customized Caching Mechanisms : The tool has the option of realizing cache_function to fine-tune cache behavior. Determining when to cache results based on specific conditions provides fine-grained control over caching logic. Example:

from crewai_tools import tool
@tool
def multiplication_tool(first_number: int, second_number: int) -> str.
"""Useful when you need to multiply two numbers together.""""
return first_number * second_number
def cache_func(args, result).
# In this case, the result is cached only if the result is a multiple of 2
cache = result % 2 == 0
return cache
multiplication_tool.cache_function = cache_func

2.3.5 Using the LangChain Tool

CrewAI integrates seamlessly with LangChain's toolkit, allowing developers to take advantage of built-in tools provided by LangChain such as GoogleSerperAPIWrapper etc., extend the capabilities of the agent by integrating it into the CrewAI system through simple configuration. Example:

from crewai import Agent
from langchain.agents import Tool
from langchain.utilities import GoogleSerperAPIWrapper
# Setting the API key
os.environ["SERPER_API_KEY"] = "Your Key"
search = GoogleSerperAPIWrapper()
# Create and assign the search tool to the agent
serper_tool = Tool(
name="Intermediate Answers",
func=search.run,
description="Tool for search-based queries"
)
agent = Agent(
role="Research Analyst",
goal="Provide up-to-date analysis of the market",
backstory="An expert analyst with a keen eye for market trends." ,
tools=[serper_tool]
)

2.4 Process

2.4.1 Process realization approach

• Sequential : Tasks are executed sequentially in a predefined order, and the output of one task is used as the context for the next task, ensuring orderly and coherent task execution.
• Hierarchical : Organize tasks in management levels by specifying the management language model (manager_llm) to enable the process. The manager agent is responsible for overseeing task execution, including planning, delegating and validating tasks, which are assigned based on the agent's capabilities, not pre-assigned.
• Consensus process (planned) : Aims to enable collaborative decision making between agents on task execution and introduce a democratic approach to task management, which has not yet been implemented in the codebase but reflects CrewAI's pursuit of continuous development and innovation.

2.4.2 The role of process in teamwork

Processes enable individual agents to function as a cohesive whole, streamlining collaborative efforts to achieve common goals in an efficient and coordinated manner. Through sound process design, teams are better able to cope with complex tasks and increase overall productivity.

2.4.3 Assigning Processes to Manager

When creating a Manager, the developer can specify the process type to set the execution policy. For hierarchical processes, you must define for the Manager Agent the manager_llm. Example:

from crewai import Crew, Process
from langchain_openai import ChatOpenAI
# Sequential Process
crew = Crew(
agents=my_agents,
tasks=my_tasks,
process=Process.sequential
)
# hierarchical process to ensure availability manager_llm
crew = Crew(
agents=my_agents,
tasks=my_tasks,
process=Process.hierarchical,
manager_llm=ChatOpenAI(model="gpt-4")
)

2.4.4 Additional task characteristics

• Asynchronous Execution (Asynchronous Execution) : Allows tasks to be executed asynchronously, enabling parallel processing and improving overall team productivity. Developers can flexibly choose synchronous or asynchronous execution methods according to task characteristics.
• Human Input Review (HIR) : Provides optional manual review of task outputs to ensure quality and accuracy before finalizing task results, introducing an additional layer of oversight to guarantee the reliability of task execution.
• Output Customization : The task supports a variety of output formats, such as JSON (output_jsonPydantic model (output_pydantic) and file output (output_file) to meet different needs and facilitate data processing and utilization.

2.5 Crews

2.5.1 Team Attributes

• Tasks : A list of tasks assigned to the team that clarifies what the team needs to accomplish.
• Agents : A list of agents belonging to the team, defining the composition of team members, each with their unique roles and capabilities.
• Process (optional) : The type of process the team follows, such as a sequential or hierarchical process, determines the order and manner in which tasks are performed.
• Level of detail (optional) : The level of detail of logging during execution, making it easy for developers to obtain sufficient information for debugging and monitoring.
• Manager LLM (optional, required for tiered process) : A language model used by manager agents in a hierarchical process to manage the task execution process.
• Function call LLM (optional) : If passed on, the team will use this LLM to make feature calls to the tool for all agents, which can be flexibly configured by the developer as required.
• Configuration (optional) : Optional configuration settings for teams to Json maybe Dict[str, Any] format is provided for further customization of team behavior.
• Maximum RPM (optional) : Maximum number of requests that can be executed per minute during team execution to avoid rate limiting, can override individual agent's max_rpm Setting.
• Language (optional) : The language used by the team, which is English by default, can be adjusted according to actual needs.
• Language files (optional) : Language file paths for teams to facilitate multi-language support.
• Memory (optional) : Used to store executive memories (short-term, long-term, physical memories) to enhance the team's ability to execute and learn.
• Cache (optional) : Specify whether or not to use caching to store the results of tool execution to improve process efficiency.
• Embedder (optional) : The embedder configuration used by the team, primarily for in-memory functions, affects how data is embedded and retrieved.
• Full output (optional) : Decide whether the team returns a complete output containing all task outputs or only the final output, to satisfy different result acquisition needs.
• Step callback (optional) : A function called after each step of each agent to log operations or perform other operations that does not override agent-specific step_callbackThe
• Task callbacks (optional) : A function called after each task is completed to monitor or perform other actions after the task.
• Shared team (optional) : Whether or not complete team information and implementation is shared with the CrewAI team to improve the library and allow training of models.
• Output log file (optional) : Whether or not to create a file containing the complete team's output and execution, you can specify the file path and name.

2.5.2 Example of creating a team

Below is an example of a team being assembled, showing how to combine agents with complementary roles and tools, assign tasks and select processes:

from crewai import Crew, Agent, Task, Process
from langchain_community.tools import DuckDuckGoSearchRun
# Define agents with specific roles and tools
researcher = Agent(
role="Senior Research Analyst",
goal="Discover innovative AI technologies",
tools=[DuckDuckGoSearchRun()]
)
writer = Agent(
role="Content Writer", goal="Write about innovative AI technologies", tools=[DuckDuckGoSearchRun(]) ) writer = Agent(
goal="Write engaging articles about AI discoveries", verbose=True), tools=[DuckDuckGoSearchRun(] ) )
verbose=True
)
# Create a task for the agent
research_task = Task(
description="Identify breakthrough AI technologies",
agent=researcher
)
write_article_task = Task(
description="Write articles about the latest AI technologies", agent=writer )
agent=writer
)
# Assembling a team using a sequential process
my_crew = Crew(
agents=[researcher, writer],
tasks=[researcher_task, write_article_task],
process=Process.sequential,
full_output=True,
verbose=True,
)

2.5.3 Team execution process

• Sequential : Tasks are executed sequentially, and work flows linearly, which is simple and intuitive, and is suitable for scenarios where there is a clear sequence between tasks.
• Hierarchical : The Manager Agent coordinates the team, delegates tasks and validates results before proceeding. This process requires manager_llmIt is suitable for the management and assignment of complex tasks, ensuring efficient task execution and quality control.
• Launch Team : Use kickoff() method initiates a team workflow, starts executing the task according to the defined process, and gets the results of the task execution. Example:

# Start the team's task execution
result = my_crew.kickoff()
print(result)

2.6 Memory

2.6.1 Memory system components

• Short-term Memory (Short-term Memory) : Temporarily storing recent interactions and outcomes allows the agent to quickly recall and utilize information that is closely related to the current context, thereby maintaining coherence in a conversation or task sequence and making more contextually relevant decisions and responses.
• Long-term Memory (Long-term Memory) : It is like a treasure trove of knowledge, retaining valuable insights and learning from past executions. As time passes, the agent is able to learn from the experience and gradually build and improve its own knowledge system, which in turn improves its future decision-making and problem-solving capabilities to better cope with complex and changing task scenarios.
• Entity Memory : Focuses on capturing and organizing information about various types of entities encountered in the course of task execution, covering people, places, concepts, and so on. This helps the agent to deeply understand the intrinsic connections between complex information, process and integrate relevant knowledge more efficiently through precise mapping of entity relationships, and provide a more comprehensive and in-depth perspective on problem solving.
• Context Memory : Works to maintain contextual information during interactions, ensuring that the agent is able to consistently respond with coherence and relevance throughout a series of tasks or a continuous dialog. Even during long interactions, it can accurately understand the task context and intent, avoiding information disconnection or misinterpretation, and thus provide more accurate and reasonable output.

2.6.2 How Memory Systems Empower Agents

• Increased contextual awareness The agent is able to grasp the context during the evolution of a conversation or a task through the synergistic effect of short-term memory and contextual memory. Whether it is information correlation in a multi-round dialog or logical continuation in a task sequence, the agent can generate more coherent, consistent and contextualized responses based on the stored contextual information, which greatly enhances the fluency and logic of the interaction experience.
• Experience building and learning acceleration : Long-term memory provides a platform for agents to grow and evolve. By storing and reviewing past actions and results, agents are able to draw lessons from them, discover patterns and patterns, and then continuously optimize their decision-making strategies and problem-solving methods. This process of experience accumulation enables the agent to make smarter and more efficient decisions when facing similar problems, significantly improving work efficiency and quality.
• Entity understanding and information processing optimization : Entity memory empowers agents to recognize and remember key entities, enabling them to quickly focus on core points and clarify the interrelationships between information when processing complex information. This not only helps the agent to understand the task requirements more accurately, but also enables it to quickly filter out the valuable content and improve the speed and accuracy of information processing in the face of massive information, so as to cope with complex tasks and diversified problem scenarios more effectively.

2.6.3 Implementing Memory in Teams

When configuring a team, developers have the flexibility to enable and customize each memory component based on the team's goals and the nature of the task. By default, the memory system is turned off and can be customized by setting the team configuration in the memory=True, you can activate the Memories feature to infuse your team with powerful memory capabilities. Memories use OpenAI Embeddings by default, but developers can also tweak the embedder parameter to other models such as Google AI, Azure OpenAI, GPT4ALL, Vertex AI, or Cohere to meet the needs of different scenarios. Example:

from crewai import Crew, Agent, Task, Process
# Using a crew with memory, using the default OpenAI Embeddings
my_crew = Crew(
agents=[...] ,
tasks=[...] ,
process=Process.sequential,
process=Process.sequential, memory=True, verbose=True
verbose=True
)
# Switch to Google AI Embedding using a team with a memory function
my_crew = Crew(
agents=[...] ,
tasks=[...] ,
process=Process.sequential,
process=Process.sequential, memory=True,
verbose=True,
embedder={
"provider": "google",
"config": {
"model": "models/embedding-001",
"task_type": "retrieval_document",
"title": "Embeddings for Embedchain"
}
}
)

2.6.4 Benefits of Using the CrewAI Memory System

• Adaptive learning and continuous optimization : With the passage of time and the continuous execution of tasks, the team is able to gradually adapt to new information and task requirements, and continuously improve the methods and strategies for handling tasks. The memory system enables agents to learn from past experiences and automatically adjust their behavioral patterns so that they can be more comfortable in facing new situations, and the overall efficiency of the team will continue to improve with the accumulation of experience.
• Personalized Experience Enhancement : The memory function enables the agent to record and identify user preferences, historical interactions, and other information, so as to provide users with more personalized services and experiences. Whether it is in content recommendation, question answering or interaction, the agent is able to respond accurately according to the user's personalized characteristics, meet the user's specific needs, and enhance the stickiness and satisfaction between the user and the system.
• Problem solving skills enhancement : Rich memory repositories provide agents with a strong knowledge backbone, enabling them to fully draw on past learning and contextual information when solving problems. By quickly retrieving and utilizing relevant experience, agents are able to analyze problems more comprehensively, discover potential solutions, and make smarter, more accurate decisions, thus effectively improving their ability to solve complex problems and providing strong support for successful team operations.

3. Practical Steps for Building a Multi Agent System Based on CrewAI

3.1 Preparations

3.1.1 Environment configuration

Ensure that your system meets the installation requirements for CrewAI and related dependencies. Depending on the operating system, you may need to install specific packages or configure environment variables. For example, on some systems you may need to install the Python environment, related libraries, and configure API keys.

3.1.2 Installation of CrewAI and Ollama (optional)

Follow the instructions in the official documentation to install CrewAI and Ollama (if you need to use Ollama) in turn. You may need to be aware of version compatibility, dependencies, and other issues during the installation process to ensure that the installation completes successfully. For example, use the pip command to install CrewAI and its toolkit:pip install crewai[tools]The Ollama installation guide should be followed accordingly.

3.1.3 Setting up the model

In the CrewAI configuration file, set the Language Model (LLM) to the desired model, such as Llama3, etc. This step ensures that the agent is able to use the appropriate language model for reasoning and decision making when performing tasks.

3.2 Creating Agents

3.2.1 Defining agent roles and goals

Define the roles and goals of each agent according to the task requirements. For example, to create an agent responsible for information collection, its role can be defined as "information collector" with the goal of "collecting relevant information in a specific domain"; and then, to create an agent focusing on data analysis with the role of "data analyst" with the goal of "deeply analyzing the collected data and extracting valuable insights".

3.2.2 Configuring agent properties

According to the agent's role and task characteristics, it is reasonable to configure the agent's attributes, such as choosing the appropriate language model, adding necessary tools, setting the maximum number of iterations, and caching strategy. Taking the information collector agent as an example, it can be configured with a web search tool, set a larger maximum number of requests to improve the efficiency of information acquisition, and turn on caching to reduce repetitive searches; whereas the data analyst agent may need to connect to a specific database tool, and adjust the maximum number of iterations and other attributes according to the complexity of the analysis task.

3.2.3 Develop agent backstory (optional but recommended)

Writing a detailed backstory for the agent, although it is an optional step, can provide a richer context for the agent's behavior and decision-making, enhance the anthropomorphic features of the agent, and help improve the interactivity and interpretability of the system. For example, to write a backstory for an information collector: "You are a professional information collector who specializes in quickly sifting through the vast amount of information on the web to find valuable content. You have been engaged in data collection for a long time, know a variety of information sources well, and are skilled in a variety of search techniques and tools. This assignment is to help the team obtain the latest information about [specific area], providing strong support for subsequent analysis and decision-making."

3.3 Defining tasks

3.3.1 Clarification of task descriptions

Provide a clear and precise description of each task to ensure that agents clearly understand the specific requirements of the task. For example, "Analyze user feedback about [product name] on social media over the past month and extract key comments and suggestions," "Predict trends in [industry name] over the next three months based on market data," etc.

3.3.2 Assignment of task agents

Reasonably assign tasks to the appropriate agents based on their roles and capabilities. You can directly specify the agent responsible for performing the task, or you can utilize CrewAI's process mechanism to allow the system to automatically assign tasks based on the agent's availability, expertise, and other factors. For example, assign information collection tasks to information collector agents and data analysis tasks to data analyst agents.

3.3.3 Configuring Task Properties

According to the task requirements, set other attributes of the task, such as the expected output format (JSON, Pydantic model or file, etc.), whether asynchronous execution is required, whether manual input review is required, and set the task context. For example, for tasks with high real-time requirements, they can be set to asynchronous execution to improve the overall response speed of the system; for important decision-making tasks, manual input review can be turned on to ensure the accuracy of the results.

3.4 Integration tools

3.4.1 Selection of appropriate tools

Choose the right tool for the task from the rich library of tools provided by CrewAI, such as for web searching WebsiteSearchTool, for file reading FileReadTooland for data analysis CSVSearchTool etc. Customized tools can also be created as needed to meet the unique needs of specific tasks.

3.4.2 Integrating tools with agents and tasks

Assign the selected tools to the appropriate agents so that the agents can invoke them when performing tasks. When defining a task, explicitly specify the tools to be used for the task to ensure that the tools are tightly integrated and work in concert with the task. For example, in an information gathering task, equip the information collector agent with the WebsiteSearchTool cap (a poem) DirectorySearchToolso that it can retrieve information from web pages and local directories.

3.5 Team building

3.5.1 Combined agents

Combine agents with different roles and competencies to form a collaborative team. Ensure that team members have complementary roles and can work together to accomplish complex tasks. For example, a team could include agents with different roles such as information gatherers, data analysts, content creators, etc., each specializing in their respective roles and working together to achieve project goals.

3.5.2 Define team processes

Depending on the characteristics and requirements of the tasks, choose an appropriate team process, such as a sequential process or a hierarchical process. Sequential processes are suitable for scenarios where there is a clear order of precedence between tasks, and agents perform tasks sequentially in a predetermined order; hierarchical processes are suitable for complex tasks that require management and coordination, and tasks are assigned and supervised through managerial agents.

3.5.3 Configuring Team Properties

According to the needs of the team, set various attributes of the team, such as the level of logging detail, maximum request rate, language settings, memory and cache configuration, and whether to share team information. Reasonable configuration of team attributes can optimize the team's operational efficiency and meet the needs of different scenarios.

3.6 Execution of tasks and monitoring

3.6.1 Initiate team tasking

utilization kickoff() method initiates the team's task execution process, and the system will drive the agent to start working according to the defined process and task assignment. During the task execution process, the progress of the task can be monitored in real time through logging and console output.

3.6.2 Monitoring of the task execution process

Utilizing the logging function and related monitoring tools provided by CrewAI, we keep a close eye on the execution status of the tasks, the operation steps of the agents, the output results and other information. Through detailed monitoring, problems that may occur during task execution, such as agent execution errors, task timeout, excessive resource utilization, etc., can be discovered in time.

3.6.3 Processing of task execution results

After the task execution is completed, the output of the task is obtained and processed. Based on the expected output format of the task, the results are parsed and analyzed to extract valuable information for subsequent decision making or further processing. At the same time, based on the results of the task execution and the problems found during the monitoring process, necessary adjustments and optimizations are made to the agent, task, or team configurations to improve the performance and accuracy of the system.

4. Case study: application of CrewAI in a real project

4.1 Intelligent Customer Service System

4.1.1 System Architecture and Agent Roles

In the intelligent customer service system, several agents are constructed to simulate the workflow of human customer service. These include user question receiving agent, question categorization agent, information query agent, answer generation agent and answer review agent. The user problem receiving agent is responsible for receiving the user's consulting problem and passing it to the problem classification agent; the problem classification agent assigns it to the corresponding information query agent according to the type of the problem; the information query agent utilizes a variety of tools (e.g., knowledge base query tool, database search tool, etc.) to find relevant information; the answer generating agent generates a preliminary answer based on the queried information; and the answer auditing agent audits and optimizes the generated answer to ensure that the answer is not a problem. The answer generation agent generates preliminary answers based on the queried information; the answer review agent reviews and optimizes the generated answers to ensure the accuracy and quality of the answers.

4.1.2 Task flow and collaboration

When a user raises a question, the user question receiving agent records the question and passes it to the question categorization agent. The problem classification agent classifies the problem into different types such as common problems, technical problems, after-sale problems, etc. through keyword analysis, semantic understanding and other techniques. Then, based on the classification result, the problem is assigned to the corresponding information query agent. The information query agent looks for relevant information in the knowledge base, database or external resources according to the type of problem. For example, for common problems, answers are retrieved directly from the knowledge base; for technical problems, it may be necessary to query technical documents or interact with technical expert systems. Based on the queried information, the answer generation agent uses natural language generation techniques to transform the information into clear and understandable answers. Finally, the answer review agent performs grammatical checking, logical review and politeness assessment of the answer, and further optimization and modification if necessary. Throughout the process, each agent realizes efficient collaboration through CrewAI's communication mechanism and task scheduling function, providing users with fast and accurate answers.

4.2 Intelligent Content Creation Assistant

4.2.1 Agent functions and division of labor

Intelligent content creation assistant consists of multiple agents working together, including theme planning agent, material collection agent, content generation agent and content touch-up agent. The theme planning agent is responsible for determining the creation theme based on user needs or market trends; the material collection agent collects relevant materials, such as articles, pictures, data, etc., using web search tools and database query tools; the content generation agent creates the first draft based on the theme and materials using natural language generation algorithms; and the content touch-up agent carries out grammatical optimization, style adjustment, and logical combing of the first draft, so as to better meet the requirements and expectations of the users. The content touch-up agent will optimize the grammar, adjust the style and comb the logic of the first draft to make it more in line with the requirements and expectations of users.

4.2.2 Creative processes and synergistic mechanisms

When a user puts forward a request for creation (e.g., writing an article about [a specific topic]), the topic planning agent first analyzes and plans the topic, and determines the general framework and key content of the article. Then, the material collection agent searches for relevant articles, research reports, cases, and other materials on the Internet according to the theme, and organizes and passes them to the content generation agent. The content generation agent generates the first draft of the article based on the topic framework and the collected materials, using the deep learning model. For example, it can generate paragraph content and organize the article structure based on the viewpoints, data, etc. in the material. Finally, the content touch-up agent carries out a comprehensive touch-up of the first draft, checking for grammatical errors, optimizing vocabulary selection, and adjusting sentence structure to make the article more fluent, smooth, and attractive. Throughout the creation process, the agents share information and feedback in real time through CrewAI's communication and collaboration mechanisms to ensure the smooth progress of the creation work.

4.3 Intelligent investment decision-making systems

4.3.1 System Composition and Agent Responsibilities

Intelligent investment decision-making system consists of market data collection agent, data analysis agent, risk assessment agent, investment strategy recommendation agent and portfolio optimization agent. The market data collection agent is responsible for collecting real-time market data from various financial data sources (e.g. stock exchanges, financial news websites, financial databases, etc.), including stock prices, turnover, macroeconomic indicators, etc.; the data analysis agent cleans, organizes, and analyzes the collected data and extracts valuable information, such as market trends, industry dynamics, and the financial status of the company; the risk assessment agent evaluates the risk level of the investment target using a risk assessment model based on the results of data analysis; the investment strategy recommendation agent evaluates the risk level of the investment target using a risk assessment model; and the investment strategy recommendation agent evaluates the risk level of the investment target based on the data analysis results. Based on the data analysis results, the risk assessment agent uses the risk assessment model to assess the risk level of the investment target; the investment strategy recommendation agent recommends suitable investment strategies, such as long-term investment, short-term speculation, diversified investment, etc., based on the market conditions and investors' risk preferences; the portfolio optimization agent optimizes the investment portfolio based on the recommended investment strategies and determines the optimal asset allocation ratio.

4.3.2 Decision-making processes and collaboration models

Market data collection agents continuously collect up-to-date market data and pass it on to data analysis agents. The data analysis agent performs in-depth analysis of the data, such as predicting stock price movements through technical analysis methods and assessing the intrinsic value of companies through fundamental analysis. The risk assessment agent evaluates the risk level of different investment targets based on the results of data analysis, combined with historical data and risk models. The investment strategy recommendation agent recommends personalized investment strategies for investors by taking into account market trends, risk assessment results and investors' risk preferences. For example, for investors with higher risk appetite, it may recommend a higher proportion of stock investment; for conservative investors, it may recommend a more robust bond and fund investment portfolio. Finally, the portfolio optimization agent applies a mathematical optimization algorithm based on the recommended investment strategies to determine the optimal portfolio allocation to maximize returns and minimize risks. During the entire decision-making process, each agent works closely together to achieve efficient data flow and scientific decision-making through CrewAI's process orchestration and communication mechanism.