CLOB: Continuous Learning of a Series of Tasks by a Large Language Model Using Only Cued Words

Original text:https://arxiv.org/pdf/2412.15479

Interpretation:This article itself is not very innovative and has little application. However, it reminds me of three highly informative articles I read a long, long time ago, and reading this article in conjunction with the three previous ones will hopefully bring you more inspiration. Recommended Reading. Intents : Explain with zep how to make a big model understand customer intents. The Convention on the Elimination of All Forms of Discrimination against Women, the CoD: Chain of Density The Convention on the Elimination of All Forms of Discrimination against Women, the Ragas: assessing RAG recall QA accuracy and answer correlation " (focus on the confidence assessment section).

Quick Read: "An In-Depth Analysis of CLOBs and CIS: A New Paradigm for Continuous Learning Based on Black Box Large Language Modeling

In the field of artificial intelligence, continuous learning (CL) has been a much talked about research direction. With the rise of Large Language Modeling (LLM), how to enable the model to continuously learn new tasks without re-training the whole model while avoiding forgetting old knowledge has become an important topic. In this paper, we will introduce a new paradigm of continuous learning - theCLOB (Continuous Learning on Black-box LLMs)and the proposals based on itCIS (contextual CL through incremental summarization)methodology and helps readers gain a deeper understanding of its core mechanisms and benefits through detailed customer service process examples and workflows.

1. Challenges to continuous learning

In traditional continuous learning scenarios, the model needs to learn a series of tasks step by step, and the data for each task is usually discarded after training. While this approach saves storage space, it also introduces the problem of **Catastrophic Forgetting**, where the model forgets previously learned knowledge as it learns new tasks.

1.1 Limitations of traditional methods

• Parameter fine-tuning: Adapting to new tasks by fine-tuning model parameters, but tends to lead to forgetting of old tasks.
• Knowledge Distillation: Migrating knowledge from old models to new ones is complex and computationally expensive to implement.
• data replay: Retain some of the old task data for training, but require additional storage space and may raise privacy concerns.

2. CLOB: A New Paradigm for Continuous Learning with Black Box Large Language Modeling

CLOB (Continual Learning Over Black-box LLMs) is a new continuous learning paradigm that treats a large language model as a black box and enables continuous learning only through **Verbal Prompting** without any fine-tuning of model parameters or addition of trainable parameters.

2.1 Core Benefits of CLOB

• Avoid parameter forgetting: There is no parameter-based catastrophic forgetting because the model parameters are not changed.
• High flexibility: Applies to LLMs accessed via APIs without access to the internal structure of the model.
• Highly scalable: Easily expandable to more tasks and categories.

2.2 How CLOB works

The CLOB workflow can be summarized in the following steps:

1. Mission arrival: When a new task arrives, the system receives a portion of the training data for that task.
2. Abstract Generation: Use LLM to generate summaries for each category that are designed to capture key information about each category.
3. Abstract Saving: The generated summaries are saved in the **Memory Repository** as a basis for subsequent learning and reasoning.

Figure 1: Overview of the CLOB system. The left side shows the use of CIS in CLOB, and the right side shows hints for the use of each component in the learning process.

3. CIS: Continuous learning in context based on incremental summarization

CIS (in-context CL via Incremental Summarization) is a specific implementation of CLOB that leverages the summarization capabilities of LLM to achieve continuous learning by incrementally updating summaries.

3.1 Key Components of CIS

3.1.1 Summary Generator (Reflector)

• functionality: Generate summaries for each category as new tasks arrive.
• workflow::
  1. Receive new mission data: For example, customer service receives frequently asked questions (FAQs) about new products.
  2. Generate a summary: Use the following tips to generate a summary for each category:

     I will show you several examples from the same category. Based on these examples, please provide a summary of the category, no more than 3 sentences. Please note that your summary should not contain any examples.
     Example: [Frequently Asked Questions and Answers from Customers about a New Product]
     

     Example:

     Tip:
     I will show you several examples from the same category. Based on these examples, please provide a summary of the category, no more than 3 sentences. Please note that your summary should not include any examples.
     Examples:
     1. Customer: What colors does this new phone come in?
     Customer Service: This phone comes in black, white, and blue.
     2. Customer: How long is the battery life of this phone?
     Customer Service: The battery life of this phone is two days.
     

     Generate a summary:

     What's New FAQ: This phone comes in black, white and blue and has a two-day battery life.
     

  3. Preservation of abstracts: Save the generated summary to memory.

3.1.2 Summary Updater

• functionality: Update the summary of the relevant category when new data is received for the old task.
• workflow::
  1. Receive new data: For example, customer service receives new questions about new product FAQs.
  2. Updated summary: Use the following tips to update the summary of the relevant category:

     Below is the original summary of a category. I will now provide m additional examples within that category. Based on these, please update the abstract accordingly. Make sure the updated summary is no more than 3 sentences long.
     Original Summary: New Product FAQ: This phone comes in black, white and blue and has a two-day battery life.
     Additional examples:
     1. Customer: Does this phone support 5G network?
     Customer Service: Yes, this phone supports 5G network.
     

     Updated summary:

     What's New FAQ: the phone comes in black, white and blue colors, has a two-day battery life and supports 5G networks.
     

  3. Save the updated summary: Replaces the original summary in the memory bank with the updated summary.

3.1.3 Solver

• functionality: Perform categorical reasoning to handle customer queries.
• workflow::
  1. Receive customer inquiriesFor example, a customer asks, "What colors does this phone come in?"
  2. Step 1::

     Solver Tip (Step 1):
     Please tell me your confidence score that the test sample belongs to each of the categories I have provided. I will provide you with a summary of these categories as a reference. The format of the summary is ': [summary]'. Your answer should contain only the category name and the corresponding confidence score as the answer.
     Sample test: what colors does this phone come in?
     Listed list:
     What's New FAQ: this phone comes in black, white, and blue, has a two-day battery life, and supports 5G networks.
     

     Output:

     New Products FAQ: 0.95
     

  3. Step 2: Select the top k=1 category with the highest confidence level, i.e., "New Product FAQ".
  4. Generate a reply: Generates a response based on the summary of the "New Product FAQ" category:

     Customer Service Response:This phone comes in black, white and blue.
     

  5. Send a reply: Send the response to the customer.

3.2 Access Logic for Memory Banks

• stockpile: The abstracts for each category are stored as a node in the memory bank. For example, "New Products FAQ" is a memory bank node containing its corresponding summary.
• retrieve: In the reasoning process, the system reads summaries of relevant categories and generates responses based on these summaries.
• update: When new data arrives, the summary is updated to reflect the latest knowledge. For example, when a new FAQ about a new product is received, the summary of the "New Product FAQ" node is updated.

4. Difference between CLOB and CIS

• CLOB is a broader continuous learning paradigm that emphasizes learning using linguistic cues without modifying model parameters. It is applicable to various types of tasks and data.
• WASC is a specific implementation of CLOB that focuses on utilizing incremental summaries to manage knowledge and address the input length limitation of LLM. In customer service scenarios, CIS is able to efficiently handle changing customer needs and knowledge by maintaining a dynamically updated summary repository.

5. Example: CIS complete workflow in the customer service process

5.1 Arrival of new missions

Suppose the customer service department has launched a new product and needs to handle related customer inquiries.

1. receive data: The system receives Frequently Asked Questions (FAQ) about new products.
2. Generate a summary::

   Reflector Tips:
   I will show you several examples from the same category. Based on these examples, please provide a summary of the category, no more than 3 sentences. Please note that your summary should not contain any examples.
   Examples:
   1. Customer: What colors does this new phone come in?
   Customer Service: This phone comes in black, white, and blue.
   2. Customer: How long is the battery life of this phone?
   Customer Service: The battery life of this phone is two days.
   

   Generate a summary:

   What's New FAQ: This phone comes in black, white and blue and has a two-day battery life.
   

3. Preservation of abstracts: Saves the summary of the "New Product FAQ" to memory.

5.2 New data arrivals from old missions

Suppose a new question is received about a new product FAQ.

1. Receive new data:: Customers ask "Does this phone support 5G networks?"
2. Updated summary::

   Updater Tip:
   Below is the original summary of a category. I will now provide m additional examples within that category. Based on these, please update the summary accordingly. Make sure the updated summary is no more than 3 sentences long.
   Original Summary: What's New FAQ: This phone comes in black, white and blue and has a two-day battery life.
   Additional examples:
   1. Customer: Does this phone support 5G network?
   Customer Service: Yes, this phone supports 5G network.
   

   Updated summary:

   What's New FAQ: the phone comes in black, white and blue colors, has a two-day battery life and supports 5G networks.
   

3. Save the updated summary: Replaces the original summary in the memory bank with the updated summary.

5.3 Reasoning process

1. Receive customer inquiries: The customer asks "What colors does this phone come in?"
2. Step 1::

   Solver Tip (Step 1):
   Please tell me your confidence score that the test sample belongs to each of the categories I have provided. I will provide you with a summary of these categories as a reference. The format of the summary is ': [summary]'. Your answer should contain only the category name and the corresponding confidence score as the answer.
   Sample test: what colors does this phone come in?
   Listed list:
   What's New FAQ: this phone comes in black, white, and blue, has a two-day battery life, and supports 5G networks.
   

   Output:

   New Products FAQ: 0.95
   

3. Step 2: Select the top k=1 category with the highest confidence level, i.e., "New Product FAQ".
4. Generate a reply: Generates a response based on the summary of the "New Product FAQ" category:

   Customer Service Response:This phone comes in black, white and blue.
   

5. Send a reply: Send the response to the customer.

5.4 Continuous learning

Over time, customer service may receive more inquiries about new products, for example:

• Customers ask "What is the size of this phone?"
• Customers ask "How much does this phone cost?"

This new data will be added to the summary of the "New Products FAQ" category through the CIS update mechanism:

Updater Tip:
Below is the original summary of a category. I will now provide m additional examples within that category. Based on these, please update the summary accordingly. Make sure the updated summary is no more than 3 sentences long.
Original Summary: What's New FAQ: This phone comes in black, white, and blue, has a two-day battery life, and supports 5G networks.
Additional examples:
1. Customer: What is the size of this phone?
Customer: The dimensions of this phone are 146.7mm x 71.5mm x 7.65mm.
2. Customer: What is the price of this phone?
Customer Service: The price of this phone is $999.

Updated summary:

What's New FAQ: the phone comes in black, white and blue, has a two-day battery life, supports 5G networks, measures 146.7mm x 71.5mm x 7.65mm, and costs $999.

6. Summary

With the above examples, we can see that CIS demonstrates strong continuous learning in the customer service process:

• dynamic update (Internet): Ability to dynamically update category summaries based on new data.
• Efficient Reasoning: Quickly locate the category corresponding to a customer query through the summary library and generate an accurate response.
• cumulative knowledge: Over time, the Abstract Library continues to accumulate knowledge so that it can better serve its customers.

This continuous learning paradigm, based on CLOBs and CIS, opens up new opportunities in the field of customer service, enabling organizations to handle customer queries more efficiently and improve customer satisfaction.

7. Future prospects

While CLOBs and CIS show great potential in customer service scenarios, there are still the following issues that deserve further exploration:

• Multimodal data processing: How do you integrate multiple forms of data, such as text, images, and speech, into a summary library?
• Personalized Service: How do you provide personalized responses based on the needs and preferences of different customers?
• topicality: How can the system's ability to handle customer queries in real time be further improved?

Through continuous research and innovation, CLOB and CIS are expected to play a greater role in a wider range of customer service scenarios and create greater value for organizations.