CLOB: Continuous Learning of a Series of Tasks by a Large Language Model Using Only Cued Words

Original text:https://arxiv.org/pdf/2412.15479

Interpretation:This article itself is not very innovative and has little application. However, it reminds me of three highly informative articles I read a long, long time ago, and reading this article in conjunction with the three previous ones will hopefully bring you more inspiration. Recommended Reading. Intents : Explain with zep how to make a big model understand customer intents. The Convention on the Elimination of All Forms of Discrimination against Women, the CoD: Chain of Density The Convention on the Elimination of All Forms of Discrimination against Women, the Ragas: assessing RAG recall QA accuracy and answer correlation " (focus on the confidence assessment section).

Quick Read: "An In-Depth Analysis of CLOBs and CIS: A New Paradigm for Continuous Learning Based on Black Box Large Language Modeling

In the field of artificial intelligence, continuous learning (CL) has been a much talked about research direction. With the rise of Large Language Modeling (LLM), how to enable the model to continuously learn new tasks without re-training the whole model while avoiding forgetting old knowledge has become an important topic. In this paper, we will introduce a new paradigm of continuous learning - theCLOB (Continuous Learning on Black-box LLMs)and the proposals based on itCIS (contextual CL through incremental summarization)methodology and helps readers gain a deeper understanding of its core mechanisms and benefits through detailed customer service process examples and workflows.

1. Challenges to continuous learning

In traditional continuous learning scenarios, the model needs to learn a series of tasks step by step, and the data for each task is usually discarded after training. While this approach saves storage space, it also introduces the problem of **Catastrophic Forgetting**, where the model forgets previously learned knowledge as it learns new tasks.

1.1 Limitations of traditional methods

• Parameter fine-tuning: Adapting to new tasks by fine-tuning model parameters, but tends to lead to forgetting of old tasks.
• Knowledge Distillation: Migrating knowledge from old models to new ones is complex and computationally expensive to implement.
• data replay: Retain some of the old task data for training, but require additional storage space and may raise privacy concerns.

2. CLOB: A New Paradigm for Continuous Learning with Black Box Large Language Modeling

CLOB (Continual Learning Over Black-box LLMs) is a new continuous learning paradigm that treats a large language model as a black box and enables continuous learning only through **Verbal Prompting** without any fine-tuning of model parameters or addition of trainable parameters.

2.1 Core Benefits of CLOB

• Avoid parameter forgetting: There is no parameter-based catastrophic forgetting because the model parameters are not changed.
• High flexibility: Applies to LLMs accessed via APIs without access to the internal structure of the model.
• Highly scalable: Easily expandable to more tasks and categories.

2.2 How CLOB works

The CLOB workflow can be summarized in the following steps:

1. Mission arrival: When a new task arrives, the system receives a portion of the training data for that task.
2. Abstract Generation: Use LLM to generate summaries for each category that are designed to capture key information about each category.
3. Abstract Saving: The generated summaries are saved in the **Memory Repository** as a basis for subsequent learning and reasoning.

Figure 1: Overview of the CLOB system. The left side shows the use of CIS in CLOB, and the right side shows hints for the use of each component in the learning process.

3. CIS: Continuous learning in context based on incremental summarization

CIS (in-context CL via Incremental Summarization) is a specific implementation of CLOB that leverages the summarization capabilities of LLM to achieve continuous learning by incrementally updating summaries.

3.1 Key Components of CIS

3.1.1 Summary Generator (Reflector)

• functionality: Generate summaries for each category as new tasks arrive.
• workflow::
  1. Receive new mission data: For example, customer service receives frequently asked questions (FAQs) about new products.
  2. Generate a summary: Use the following tips to generate a summary for each category:

     我将向你展示来自同一类别的几个示例。基于这些示例，请提供该类别的摘要，不超过 3 句话。请注意，你的摘要不应包含任何示例。
     示例：[客户关于新产品的常见问题及解答]
     

     Example:

     提示：
     我将向你展示来自同一类别的几个示例。基于这些示例，请提供该类别的摘要，不超过 3 句话。请注意，你的摘要不应包含任何示例。
     示例：
     1. 客户：这款新手机有哪些颜色？
     客服：这款手机有黑色、白色和蓝色三种颜色。
     2. 客户：这款手机的电池续航时间有多长？
     客服：这款手机的电池续航时间为两天。
     

     Generate a summary:

     新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天。
     

  3. Preservation of abstracts: Save the generated summary to memory.

3.1.2 Summary Updater

• functionality: Update the summary of the relevant category when new data is received for the old task.
• workflow::
  1. Receive new data: For example, customer service receives new questions about new product FAQs.
  2. Updated summary: Use the following tips to update the summary of the relevant category:

     下面是某个类别的原始摘要。我现在将提供 m 个该类别内的额外示例。基于这些，请相应地更新摘要。确保更新后的摘要不超过 3 句话。
     原始摘要：新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天。
     额外示例：
     1. 客户：这款手机支持5G网络吗？
     客服：是的，这款手机支持5G网络。
     

     Updated summary:

     新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天，支持5G网络。
     

  3. Save the updated summary: Replaces the original summary in the memory bank with the updated summary.

3.1.3 Solver

• functionality: Perform categorical reasoning to handle customer queries.
• workflow::
  1. Receive customer inquiriesFor example, a customer asks, "What colors does this phone come in?"
  2. Step 1::

     Solver 提示（步骤 1）：
     请告诉我你对测试样本属于我提供的每个类别的置信度分数。我将向你提供这些类别的摘要作为参考。摘要的格式为 '<类别名称>: [摘要]'。你的回答应仅包含类别名称和相应的置信度分数作为答案。
     测试样本：这款手机有哪些颜色？
     列出的列表：
     新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天，支持5G网络。
     

     Output:

     新产品 FAQ：0.95
     

  3. Step 2: Select the top k=1 category with the highest confidence level, i.e., "New Product FAQ".
  4. Generate a reply: Generates a response based on the summary of the "New Product FAQ" category:

     客服回复：这款手机有黑色、白色和蓝色三种颜色。
     

  5. Send a reply: Send the response to the customer.

3.2 Access Logic for Memory Banks

• stockpile: The abstracts for each category are stored as a node in the memory bank. For example, "New Products FAQ" is a memory bank node containing its corresponding summary.
• retrieve: In the reasoning process, the system reads summaries of relevant categories and generates responses based on these summaries.
• update: When new data arrives, the summary is updated to reflect the latest knowledge. For example, when a new FAQ about a new product is received, the summary of the "New Product FAQ" node is updated.

4. Difference between CLOB and CIS

• CLOB is a broader continuous learning paradigm that emphasizes learning using linguistic cues without modifying model parameters. It is applicable to various types of tasks and data.
• WASC is a specific implementation of CLOB that focuses on utilizing incremental summaries to manage knowledge and address the input length limitation of LLM. In customer service scenarios, CIS is able to efficiently handle changing customer needs and knowledge by maintaining a dynamically updated summary repository.

5. Example: CIS complete workflow in the customer service process

5.1 Arrival of new missions

Suppose the customer service department has launched a new product and needs to handle related customer inquiries.

1. receive data: The system receives Frequently Asked Questions (FAQ) about new products.
2. Generate a summary::

   Reflector 提示：
   我将向你展示来自同一类别的几个示例。基于这些示例，请提供该类别的摘要，不超过 3 句话。请注意，你的摘要不应包含任何示例。
   示例：
   1. 客户：这款新手机有哪些颜色？
   客服：这款手机有黑色、白色和蓝色三种颜色。
   2. 客户：这款手机的电池续航时间有多长？
   客服：这款手机的电池续航时间为两天。
   

   Generate a summary:

   新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天。
   

3. Preservation of abstracts: Saves the summary of the "New Product FAQ" to memory.

5.2 New data arrivals from old missions

Suppose a new question is received about a new product FAQ.

1. Receive new data:: Customers ask "Does this phone support 5G networks?"
2. Updated summary::

   Updater 提示：
   下面是某个类别的原始摘要。我现在将提供 m 个该类别内的额外示例。基于这些，请相应地更新摘要。确保更新后的摘要不超过 3 句话。
   原始摘要：新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天。
   额外示例：
   1. 客户：这款手机支持5G网络吗？
   客服：是的，这款手机支持5G网络。
   

   Updated summary:

   新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天，支持5G网络。
   

3. Save the updated summary: Replaces the original summary in the memory bank with the updated summary.

5.3 Reasoning process

1. Receive customer inquiries: The customer asks "What colors does this phone come in?"
2. Step 1::

   Solver 提示（步骤 1）：
   请告诉我你对测试样本属于我提供的每个类别的置信度分数。我将向你提供这些类别的摘要作为参考。摘要的格式为 '<类别名称>: [摘要]'。你的回答应仅包含类别名称和相应的置信度分数作为答案。
   测试样本：这款手机有哪些颜色？
   列出的列表：
   新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天，支持5G网络。
   

   Output:

   新产品 FAQ：0.95
   

3. Step 2: Select the top k=1 category with the highest confidence level, i.e., "New Product FAQ".
4. Generate a reply: Generates a response based on the summary of the "New Product FAQ" category:

   客服回复：这款手机有黑色、白色和蓝色三种颜色。
   

5. Send a reply: Send the response to the customer.

5.4 Continuous learning

Over time, customer service may receive more inquiries about new products, for example:

• Customers ask "What is the size of this phone?"
• Customers ask "How much does this phone cost?"

This new data will be added to the summary of the "New Products FAQ" category through the CIS update mechanism:

Updater 提示：
下面是某个类别的原始摘要。我现在将提供 m 个该类别内的额外示例。基于这些，请相应地更新摘要。确保更新后的摘要不超过 3 句话。
原始摘要：新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天，支持5G网络。
额外示例：
1. 客户：这款手机的尺寸是多少？
客服：这款手机的尺寸为 146.7mm x 71.5mm x 7.65mm。
2. 客户：这款手机的价格是多少？
客服：这款手机的价格为 999 美元。

Updated summary:

新产品 FAQ：这款手机有黑色、白色和蓝色三种颜色，电池续航时间为两天，支持5G网络，尺寸为 146.7mm x 71.5mm x 7.65mm，价格为 999 美元。

6. Summary

With the above examples, we can see that CIS demonstrates strong continuous learning in the customer service process:

• dynamic update (Internet): Ability to dynamically update category summaries based on new data.
• Efficient Reasoning: Quickly locate the category corresponding to a customer query through the summary library and generate an accurate response.
• cumulative knowledge: Over time, the Abstract Library continues to accumulate knowledge so that it can better serve its customers.

This continuous learning paradigm, based on CLOBs and CIS, opens up new opportunities in the field of customer service, enabling organizations to handle customer queries more efficiently and improve customer satisfaction.

7. Future prospects

While CLOBs and CIS show great potential in customer service scenarios, there are still the following issues that deserve further exploration:

• Multimodal data processing: How do you integrate multiple forms of data, such as text, images, and speech, into a summary library?
• Personalized Service: How do you provide personalized responses based on the needs and preferences of different customers?
• topicality: How can the system's ability to handle customer queries in real time be further improved?

Through continuous research and innovation, CLOB and CIS are expected to play a greater role in a wider range of customer service scenarios and create greater value for organizations.