This guide describes how to utilize Claude s advanced conversational capabilities handle customer inquiries in real-time, provide 24/7 support, reduce wait times, and manage high volumes of support requests with accurate responses and proactive interactions.
Before building with Claude
Decide whether to use Claude for support chat
Here are some key indicators that you should use an LLM like Claude to automate some of your customer support processes:
Large number of repetitive queries
Claude specializes in efficiently handling large numbers of similar problems, freeing human agents to tackle more complex problems.
The need for rapid information integration
Claude is able to quickly retrieve, process, and integrate information from a vast knowledge base, whereas a human agent may need time to conduct research or consult multiple sources.
24/7 availability requirements
Claude is able to provide 24/7 support without tiring, whereas providing continuous coverage for human agents can be both expensive and challenging.
Rapid expansion during peak periods
Claude can handle sudden increases in query volume without having to hire and train additional staff.
Consistent brand voice
You can instruct Claude to consistently represent your brand's tone and values, while human agents may have different communication styles.
Some factors to consider when choosing Claude over other LLMs:
- You prioritize natural and nuanced conversations: Claude's fine-grained language comprehension allows for more natural, context-aware conversations that are closer to human communication than chats with other LLMs.
- You often receive complex and open-ended questions: Claude can handle a wide range of topics and queries without generating boilerplate responses or having to write extensive permutations of user-speak.
- You need scalable multilingual support: Claude's multilingual capabilities make it possible to converse in more than 200 languages without having to set up a separate chatbot for each supported language or go through an extensive translation process.
Defining the ideal chat interaction
Outline the ideal customer interaction to define how and when you expect your customers to interact with Claude. This overview will help determine the technical requirements for your solution.
Below is a sample chat interaction for auto insurance customer support:
- our customers: Launch Support Chat Experience
- Claude: Warmly welcome customers and start a conversation
- our customers: Ask about insurance for their new electric car
- Claude: Provide information about electric vehicle insurance
- our customers: Ask questions related to the special needs of electric vehicle insurance
- Claude: Provide accurate and useful answers with links to relevant sources
- our customers: Ask off-topic questions that have nothing to do with insurance or automobiles
- Claude: Clarify that they do not discuss irrelevant topics and direct customers back to auto insurance related content
- our customers: Expressed interest in insurance quotes
- Claude: Ask a series of questions to determine an appropriate quote and make adjustments based on the customer's answers
- Claude: Send a request to the Quote Generation API tool with the necessary information gathered from the customer
- Claude: receive response information from the API tool, synthesize it into a natural answer and provide a quote to the customer
- our customers: Ask follow-up questions
- Claude: Answer follow-up questions as needed
- Claude: Guide the customer to the next step in the insurance process and end the conversation
As you write actual examples for your own use cases, you may find it helpful to write out the actual words used in the dialog so that you can also get a feel for Claude's ideal tone, length of response, and level of detail required.
Breaking down interactions into discrete tasks
Customer support chat consists of a number of different tasks, including answering questions, retrieving information, and handling request actions, all brought together in one complete customer interaction. Before you start building, break down the ideal customer interaction into each task that Claude needs to perform. This ensures that you prompt and evaluate Claude for each task and provides a good understanding of the scope of the interaction to consider when writing test cases.
Clients sometimes find it helpful to visualize this process as an interaction flowchart showing the possible branching points of the conversation based on user requests.
The following are key tasks related to the above insurance interactions:
- Welcome and general guidance
- Warmly welcome customers and start a conversation
- Provide basic information about the company and interactions
- Product Information
- Providing information about EV insurance This needs to be done to ensure that Claude has the necessary information in its context, and may mean that it needs to be RAG IntegrationThe
- Answer questions related to the special needs of electric vehicle insurance
- Answer follow-up questions about quotes or insurance details
- Provide links to relevant sources where appropriate
- Dialogue management
- Staying Topic-Focused (Auto Insurance)
- Steering off-topic questions back to relevant topics
- Quote Generation
- Ask appropriate questions to determine qualification of offers
- Adjusting questions based on customer responses
- Submit the collected information to the Quote Generation API
- Provide customers with generated quotes
Establishing success criteria
Working with your support team, theDefine clear success criteriaand writing with quantifiable benchmarks and targetsDetailed assessmentThe
The following are the criteria and benchmarks for evaluating Claude's success in performing the defined tasks:
Query comprehension accuracy
This metric assesses the accuracy of Claude's understanding of customer inquiries across a variety of topics. By reviewing a sample of conversations, Claude's understanding of customer intent, critical next steps, successful solutions, etc. is assessed. The goal is to achieve an understanding accuracy of 95% or better.
Relevance
Evaluate whether Claude's responses are effective in solving a customer's specific problem. Evaluate a set of conversations and rate the relevance of each response (large-scale evaluation using a scoring system based on a large language model). Target relevance score of 90% or higher.
Response accuracy
Evaluate the accuracy of the company and product information provided to Claude based on the contextual information provided. The goal is to achieve an accuracy rate of 100% for this lead information.
Relevance provided by citation
Tracks the frequency and relevance of links or resources provided. The goal is to provide relevant resources in 80% interactions that require additional information.
Theme adherence
Evaluate whether Claude stays on topic during the conversation, such as in our auto insurance example implementation, and evaluate Claude's responses to auto insurance or customer-specific questions. The goal is that 95%'s responses are directly related to auto insurance or customer-specific queries.
Effectiveness of content generation
Evaluate Claude's success in when to generate message content and the relevance of its content. For example, in our implementation, we will evaluate Claude's understanding of when to generate quotes and their accuracy. The goal is 100% accuracy, as this is critical information for successful customer interactions.
Upgrade efficiency
This measures Claude's ability to recognize queries that require human intervention and escalate them when appropriate. Tracks the percentage of conversations that are escalated correctly and those that should have been escalated but were not. The goal is 95% or better escalation accuracy.
The following are criteria and benchmarks for assessing the business impact of using Claude for support:
emotional maintenance
Assess Claude's ability to maintain or improve customer sentiment throughout the conversation. Use a sentiment analysis tool to measure the emotional state at the beginning and end of each conversation. The goal is for sentiment to be maintained or improved during interactions in 90%.
diversion rate
Percentage of customer inquiries that are successfully handled by a chatbot without human intervention. Typically targets 70%-80% triage rate, depending on the complexity of the inquiry.
Customer Satisfaction Rating
Measures customer satisfaction with interactions with chatbots. Usually done through a post-interaction survey. The goal is a customer satisfaction score of 4 out of 5 or higher.
Average processing time
The average time it takes a chatbot to resolve an inquiry. This time varies greatly depending on the complexity of the question, but the overall goal is to make the average handling time shorter than human customer service.
How to implement Claude as a customer service agent
Selecting the right Claude model
The choice of model depends on the trade-off between cost, accuracy and response time.
For customer support chat.claude-3-5-sonnet-20240620 Ideal for balancing intelligence, latency, and cost. However, if multiple prompts are included in a dialog (including RAGs, tool usage, and/or long contextual prompts), theclaude-3-haiku-20240307 It may be better to optimize latency.
Building a Powerful Cue Word
Using Claude for customer support requires that Claude be directional and contextual enough to respond appropriately, and flexible enough to handle a wide range of customer queries.
Let's start with the elements of writing strong cues, starting with system cues:
IDENTITY = """You are Eva, a friendly and knowledgeable AI assistant for Acme Insurance. Your role is to warmly welcome customers and provide information about Acme's insurance products, including car and electric car insurance. You can also help customers get quotes for their insurance needs."""
While you might be inclined to put all the information in the system prompt words to separate the instructions from the user dialog, Claude actually puts most of the prompt content in the first round of the User Performs best when in the middle (the only exception is character prompts). For more information read Use the system prompt to assign a role to ClaudeThe
It is best to break down complex cues into sub-parts and write them one part at a time. For each task, you can get better results by progressively defining the parts of the prompt that Claude needs to complete that task. In the case of auto insurance customer support, for example, we'll write the prompts in stages, starting with the 'Greetings and General Guidance' task. This also makes debugging the prompts easier, as you can tweak the individual parts of the overall prompt more quickly.
We put all these sections in a file called config.py in the document.
STATIC_GREETINGS_AND_GENERAL = """
<static_context
Acme Auto Insurance: Your Reliable Partner on the Road
Brief Description:
At Acme Insurance, we understand that your vehicle is more than just transportation-it's your ticket to life's adventures. Since 1985, we've been crafting auto insurance policies that give drivers the confidence to explore, commute and travel.
Whether you're navigating city streets or traveling cross-country, Acme protects you and your vehicle.
Our innovative auto insurance policies are designed to fit your unique needs and cover everything from minor collisions to major accidents.
With Acme's award-winning customer service and fast claims solutions, you can focus on the fun of driving while we take care of the rest.
We're more than just an insurance provider - we're your co-pilot on the road.
Experience the peace of mind that comes with quality coverage and genuine care when you choose Acme Auto Insurance. Because at Acme, we don't just insure your car - we power the journey you're taking to build your life.
Note: We also offer specialized coverage for electric vehicles, ensuring that drivers of all types of vehicles benefit from our protection.
Acme Insurance offers the following products:
- Car insurance
- Electric Vehicle Insurance
- Two wheeler insurance
Hours of operation: Monday through Friday, 9 a.m. to 5 p.m. (EST)
Customer Service Number: 1-800-123-4567
</static_context
"""
We will treat auto insurance and EV insurance information the same.
STATIC_CAR_INSURANCE="""
<static_context
Auto Insurance Coverage:
Acme's auto insurance policies typically cover:
1. liability insurance: pays for bodily injury and property damage you cause to others.
2. Collision: pays for damage to your car in an accident.
3. Comprehensive: Pays for damage to your car in non-collision accidents. 4.
4. Medical Expenses Coverage: Pays for medical expenses following an accident.
5. uninsured/underinsured motorist coverage: protects you if you are hit by an underinsured motorist.
Optional coverage includes:
- Rental car reimbursement
- Roadside assistance
- New Vehicle Replacement
</static_context
"""
STATIC_ELECTRIC_CAR_INSURANCE="""
<static_context
Electric Vehicle Insurance:
Our specialized electric vehicle insurance goes beyond traditional auto insurance to provide tailored protection for your high-tech investment. In addition to our standard insurance programs, we offer:
- Protection for expensive battery systems
- Cover for home charging equipment
- Adaptation to the rapid development of the electric vehicle sector
Whether you drive a Rivian, a Nissan Leaf, a Chevrolet, an electric car, or a car with a battery, we offer you the following. Bolt or any other electric vehicle model, our experienced agents are ready to create a policy that gives you peace of mind for every electrically powered journey.
</static_context
"""
Now that we have static content, let's add at least 4-5 examples of 'good' interactions to guide Claude's response. These examples should represent your ideal customer interactions and can include safeguards, tool calls, etc.
EXAMPLES="""
Here are some examples of how to interact with your customers:
<example 1
H: Hi, do you offer business insurance for small businesses?
A: Ah! Unfortunately, we do not offer business insurance at this time.
However, we do have a variety of personal insurance products, including auto insurance and electric vehicle insurance.
Would you like to learn more about these products?
</example 1
<example 2
H: Hi, what are your business hours?
A: We are open Monday through Friday, 9am to 5pm (EST).
If you have any questions or need assistance, please feel free to contact us during these hours.
Is there anything else I can help you with today?
</example 2
<example 3
H: I am looking for auto insurance. what services does Acme offer?
A: Great choice! Acme's auto insurance policies typically include:
- Liability: Covers you if you are at fault in an accident.
- Collision: pays for damage to your vehicle in an accident
- Comprehensive coverage: Covers non-collision events such as theft or vandalism
- Medical Expenses Coverage: Pays for medical expenses after an accident
- Uninsured/Underinsured Driver Coverage: protects you from underinsured drivers
We also offer optional coverages such as rental car reimbursement, roadside assistance and new vehicle replacement.
Would you like to know more details about these benefits or have other specific questions about insuring your vehicle?
</example 3
<example 4
H: I'd like to learn more about your electric vehicle coverage.
A: Sure! Our electric vehicle insurance is specifically tailored to the unique needs of electric vehicle owners.
In addition to all the standard coverage we offer for regular vehicles, our EV insurance includes:
- Battery Cover: protects your vehicle's expensive battery system from damage or deterioration
- Technology Pack: covers the repair or replacement of advanced electronics and on-board computer systems
- Charging station cover: protects home and portable charging equipment
- Cyber security cover: protection against hacking or digital threats
- Specialized Roadside Assistance: includes flatbed towing services to protect the vehicle chassis
We offer coverage for a wide range of electric vehicles, including popular models such as Tesla, Rivian, Nissan, Chevrolet and more.
Do you have a specific electric vehicle you need to insure, or would you like more details on certain coverages?
</example 4
<example 5
H: I would like to get a quote for my auto insurance.
A: Sure! I would be happy to provide you with an auto insurance quote.
In order to provide you with an accurate quote, I need to gather some information about your vehicle and the primary driver.
Let's start with basic information:
1. what is the make and model of your vehicle?
2. what year was the vehicle manufactured?
3. approximately how many miles have you driven?
4. what is the age of the primary driver?
Once you provide this information, I will use our quote tool to generate a personalized insurance quote for you.
</example 5
"""
You will also need to include any important instructions on the do's and don'ts of how Claude interacts with customers. These may be taken from brand protection or support policies.
ADDITIONAL_GUARDRAILS = """Please follow the following precautions:
1. only provide information about the types of coverage listed in our products.
2. if asked about a type of coverage we do not offer, politely state that we do not offer it.
3. do not speculate about future products or company plans.
4. do not make unauthorized promises or agreements. You can only provide information and guidance.
5. do not mention any competitor's products or services.
"""
Now let's combine all these parts into a string to use as our prompt.
TASK_SPECIFIC_INSTRUCTIONS = ' '.join([
STATIC_GREETINGS_AND_GENERAL,
STATIC_CAR_INSURANCE, STATIC_ELECTRIC_CAR_INSURANCE
STATIC_ELECTRIC_CAR_INSURANCE.
EXAMPLES.
ADDITIONAL_GUARDRAILS, ])
])
Using tools to empower dynamics and proactivity
Claude can dynamically perform operations and retrieve information by using client-side tooling features. Any external tools or APIs that should be used for the prompt are listed first.
In this example, we'll start with a tool for calculating quotes.
Note that the tool does not actually perform the calculation, it only signals to the application that the tool should be used and passes the specified parameters.
Example insurance quote calculator:
TOOLS = [{
"name": "get_quote",
"description": "Calculates insurance quotes based on user input. The return value is the monthly premium.",
"input_schema": {
"type": "object",
"properties": {
"make": {"type": "string", "description": "The make of the vehicle."},
"model": {"type": "string", "description": "The model of the vehicle."},
"year": {"type": "integer", "description": "The year of manufacture of the vehicle."},
"mileage": {"type": "integer", "description": "Mileage on the vehicle."},
"driver_age": {"type": "integer", "description": "The age of the primary driver."}
},
"required": ["make", "model", "year", "mileage", "driver_age"]
}
}]
def get_quote(make, model, year, mileage, driver_age):
"""Return monthly premium (in dollars)""""
# You can call an HTTP endpoint or a database to get a quote.
# Here, we simulate a 1-second delay and return a fixed $100 quote.
time.sleep(1)
return 100
Deploying your tips
Without deploying the prompt to a test production environment and Operational assessment case, it's hard to know how effective your hints are. So let's build a small application using our hints, the Anthropic SDK, and streamlit, and create a user interface for it.
under the name of chatbot.py file, first set up the ChatBot class, which will encapsulate the interaction with the Anthropic SDK.
The class should contain two main methods:generate_message cap (a poem) process_user_inputThe
from anthropic import Anthropic
from config import IDENTITY, TOOLS, MODEL, get_quote
from dotenv import load_dotenv
load_dotenv()
class ChatBot:
def __init__(self, session_state):
self.anthropic = Anthropic()
self.session_state = session_state
def generate_message(
self,
messages,
max_tokens,
):
try:
response = self.anthropic.messages.create(
model=MODEL,
system=IDENTITY,
max_tokens=max_tokens,
messages=messages,
tools=TOOLS,
)
return response
except Exception as e:
return {"error": str(e)}
def process_user_input(self, user_input):
self.session_state.messages.append({"role": "user", "content": user_input})
response_message = self.generate_message(
messages=self.session_state.messages,
max_tokens=2048,
)
if "error" in response_message:
return f"发生错误: {response_message['error']}"
if response_message.content[-1].type == "tool_use":
tool_use = response_message.content[-1]
func_name = tool_use.name
func_params = tool_use.input
tool_use_id = tool_use.id
result = self.handle_tool_use(func_name, func_params)
self.session_state.messages.append(
{"role": "assistant", "content": response_message.content}
)
self.session_state.messages.append({
"role": "user",
"content": [{
"type": "tool_result",
"tool_use_id": tool_use_id,
"content": f"{result}",
}],
})
follow_up_response = self.generate_message(
messages=self.session_state.messages,
max_tokens=2048,
)
if "error" in follow_up_response:
return f"发生错误: {follow_up_response['error']}"
response_text = follow_up_response.content[0].text
self.session_state.messages.append(
{"role": "assistant", "content": response_text}
)
return response_text
elif response_message.content[0].type == "text":
response_text = response_message.content[0].text
self.session_state.messages.append(
{"role": "assistant", "content": response_text}
)
return response_text
else:
raise Exception("发生错误: 意外的响应类型")
def handle_tool_use(self, func_name, func_params):
if func_name == "get_quote":
premium = get_quote(**func_params)
return f"生成的报价: 每月 ${premium:.2f}"
raise Exception("使用了意外的工具")
Building the User Interface
Deploy this code using Streamlit and a master method test. This main() function sets up a Streamlit-based chat interface.
We'll do this in a file called app.py This is done in the file
import streamlit as st
from chatbot import ChatBot
from config import TASK_SPECIFIC_INSTRUCTIONS
def main():
st.title("Chatting with Eva, assistant at Acme Insurance 🤖")
if "messages" not in st.session_state.
st.session_state.messages = [
{'role': "user", "content": TASK_SPECIFIC_INSTRUCTIONS}, {
{'role': "assistant", "content": "Understood."}
]
chatbot = ChatBot(st.session_state)
# Displays messages from users and assistants, skipping the first two
for message in st.session_state.messages[2:]:
# Ignore message blocks used by the tool
if isinstance(message["content"], str):
with st.chat_message(message["role"]).
st.markdown(message["content"])
if user_msg := st.chat_input("Enter your message here...").
st.chat_message("user").markdown(user_msg)
with st.chat_message("assistant"):
with st.spinner("Eva is thinking...").
response_placeholder = st.empty()
full_response = chatbot.process_user_input(user_msg)
response_placeholder.markdown(full_response)
if __name__ == "__main__":
main()
Use the following command to run the program:
streamlit run app.py
Evaluate your tips
Prompts often require testing and optimization to reach production readiness. To determine if your solution is ready, you can evaluate chatbot performance through a systematic process that combines quantitative and qualitative methods. Based on your defined success criteria, create aStrong empirical assessmentwill enable you to optimize your prompts.
Anthropic Console An evaluation tool is now provided that allows you to test your prompts in different scenarios.
Improve performance
In complex scenarios, in addition to the standardTip Engineeringcap (a poem)Guardrail Implementation StrategyBeyond that, it may be helpful to consider other strategies to improve performance. Here are some common scenarios:
Reducing Long Context Latency with RAG
When dealing with a large number of static and dynamic contexts, including all the information in the cue can lead to high costs, slower response times, and reaching the limits of the context window. In this case, implementing Retrieval Augmented Generation (RAG) techniques can significantly improve performance and efficiency.
By using the Voyage-like embedding model Converting information into a vector representation creates a more scalable and responsive system. This approach allows dynamic retrieval of relevant information based on the current query, rather than including all possible contexts in every hint.
realize RAG Used to support the use of scenarios RAG formulations has been shown to improve accuracy, reduce response times, and lower API costs in systems that require a wide range of contexts.
Integration of real-time data through the use of tools
When dealing with queries that require real-time information, such as account balances or policy details, an embedded RAG-based approach is not enough. Instead, you can leverage tool usage to significantly enhance your chatbot's ability to provide accurate real-time responses. For example, you can use tools to find customer information, retrieve order details, and cancel orders on behalf of customers.
This method is used in our Tool Use: Customer Service Agent Recipe outlined in , allowing you to seamlessly integrate real-time data into Claude's responses, providing a more personalized and efficient customer experience.
Enhanced input and output protection measures
When deploying chatbots, especially in customer service scenarios, it is critical to protect against the risk of abuse, out-of-scope queries, and inappropriate responses. While Claude is inherently resilient to these scenarios, here are additional steps to strengthen your chatbot safeguards:
- Reduction of hallucinations: Implementation of fact-checking mechanisms and quote to ensure that responses are based on the information provided.
- Cross-check information: Ensure that the agent's response is consistent with company policy and known facts.
- Avoiding Contractual Commitments: Ensuring that the agent does not make promises or agreements that he or she does not have the authority to commit to.
- Jailbreak mitigation: Use methods such as innocuous screening and input validation to prevent users from exploiting model vulnerabilities to generate inappropriate content.
- Avoid mentioning competitors: implement a competitor mention filter to keep your brand focused and avoid mentioning any competitor's products or services.
- Keep Claude's role.: Prevents Claude from changing its contextual style during long and complex interactions.
- Remove Personally Identifiable Information (PII): Unless explicitly requested and authorized, delete all PII from responses.
Reducing Perceived Response Time through Streaming Processing
Implementing streaming can significantly increase user engagement and satisfaction when dealing with potentially long responses. In this case, users receive answers gradually, rather than waiting for the entire response to be generated.
Below are the steps on how to implement stream processing:
- utilization Anthropic Streaming API Streaming responses are supported.
- Sets up the front end to process incoming blocks of text.
- Displays a block of text for each arrival, simulating the effect of real-time input.
- Implement a mechanism to save full replies, allowing users to view the full content when they return after navigating away.
In some cases, streaming allows the use of advanced models with higher base latencies, as progressive display mitigates the effects of longer processing times.
Extend your Chatbot
As the complexity of Chatbot increases, the application architecture can evolve with it. Before adding more layers to the architecture, consider the following simpler options:
- Make sure you're making the most of hints and optimizing with hint engineering. Use our Tip Engineering Guide Write the most effective tips.
- Add others to the prompt artifact (which may include cue chain) to see if the desired function can be achieved.
If your Chatbot handles very diverse tasks, you may want to consider adding a Separate intent classifiers to route the initial customer query. For existing applications, this would involve creating a decision tree that routes customer queries through classifiers to specialized conversations (each with its own tools and system prompts). Note that this approach requires an additional call to Claude, which may add latency.
Integrating Claude into Support Workflows
While our examples focus on Python functions that can be called in a Streamlit environment, deploying Claude for real-time support of Chatbot requires an API service.
You can do this:
- Creating API wrappers: Develop a simple API wrapper for your taxonomy functions. For example, you can use the Flask API or Fast API to wrap your code as an HTTP service. This HTTP service can take user input and return a complete Assistant response. As a result, the service can have the following characteristics:
- Server-Sent Events (SSE): SSE allows real-time response streaming between the server and the client. This is critical to providing a smooth, interactive experience when using LLM.
- Caching: Implementing caching can significantly improve responsiveness and reduce unnecessary API calls.
- Context retention: Keeping context is important for the continuity of the dialog as the user navigates away and back.
- Building the Web Interface: Implement a user-friendly Web UI for interacting with Claude-driven agents.
