Ragas: RAG 리콜 QA 정확도 및 답변 상관관계 평가하기

일반 소개

Ragas는 검색 증강 생성(RAG) 시스템을 평가하고 최적화하기 위해 특별히 고안된 도구입니다. 쿼리, 검색 컨텍스트, 생성된 답변 간의 관계를 분석하여 종합적인 평가 지표를 제공합니다. 이러한 지표에는 충실도, 답변 관련성, 문맥 관련성, 문맥 리콜 및 문맥 정밀도가 포함되며, 개발자는 RAG 시스템의 성능을 평가하고 최적화하여 생성된 답변이 정확하고 관련성이 있으며 검색된 콘텐츠에 기반하고 있는지 확인할 수 있습니다. 새로운 시스템을 개발하든 기존 시스템을 최적화하든 Ragas는 강력한 지원을 제공합니다.

 

기능 목록

• 충실도 평가검색된 컨텍스트에 대한 생성된 답변의 충실도를 측정합니다.
• 답안 관련성 평가: 생성된 답변과 원래 쿼리의 관련성을 평가합니다.
• 문맥 관련성 평가검색된 정보가 문제에 얼마나 잘 맞는지 측정합니다.
• 상황별 회상 평가검색된 컨텍스트에 질문에 답하는 데 필요한 모든 정보가 포함되어 있는지 평가합니다.
• 문맥 정확도 평가결과에서 검색된 관련 컨텍스트의 순서 지정 품질을 측정합니다.

 

도움말 사용

설치 프로세스

1. PyPI를 통한 설치::

   pip install ragas

2. 소스에서 설치::

   pip install git+https://github.com/explodinggradients/ragas

빠른 시작

1. 필요한 모듈 가져오기::

   from ragas import SingleTurnSample
from ragas.metrics import AspectCritic

2. 테스트 데이터 준비::

   test_data = {
"user_input": "summarise given text\nThe company reported an 8% rise in Q3 2024, driven by strong performance in the Asian market. Sales in this region have significantly contributed to the overall growth. Analysts attribute this success to strategic marketing and product localization. The positive trend in the Asian market is expected to continue into the next quarter.",
"response": "The company experienced an 8% increase in Q3 2024, largely due to effective marketing strategies and product adaptation, with expectations of continued growth in the coming quarter."
}

3. RAG 시스템 평가::

   evaluator_llm = LangchainLLMWrapper(ChatOpenAI(model="gpt-4o"))
metric = AspectCritic(name="summary_accuracy", llm=evaluator_llm, definition="Verify if the summary is accurate.")
await metric.single_turn_ascore(SingleTurnSample(**test_data))

세부 기능 작동 흐름

1. 충실도 평가::
   • Ragas에서 제공하는 충실도 메트릭을 사용하여 생성된 답변이 검색된 컨텍스트에 얼마나 충실하게 부합하는지 평가합니다.
   • 해당 평가 함수를 호출하여 충실도 평가 결과를 얻고 분석합니다.
2. 답안 관련성 평가::
   • Ragas의 답변 관련성 메트릭을 사용하여 생성된 답변의 원래 쿼리에 대한 관련성 정도를 평가합니다.
   • 관련성 평가 매개 변수를 구성하여 생성된 답변의 관련성 평가를 사용자 지정합니다.
3. 문맥 관련성 평가::
   • Ragas의 문맥 관련성 메트릭을 사용하여 검색된 정보가 문제에 얼마나 잘 맞는지 측정하세요.
   • 관련성 평가 함수를 호출하여 문맥 관련성 평가 결과를 가져옵니다.
4. 상황별 회상 평가::
   • Ragas의 컨텍스트 리콜 메트릭을 사용하여 검색된 컨텍스트에 질문에 답하는 데 필요한 모든 정보가 포함되어 있는지 평가합니다.
   • 리콜 평가 매개변수를 설정하여 검색된 문맥이 질문에 필요한 정보를 충분히 포함하고 있는지 확인합니다.
5. 문맥 정확도 평가::
   • Ragas의 문맥 정확도 메트릭을 사용하여 결과에서 검색된 관련 문맥의 순서 지정 품질을 측정하세요.
   • 정확도 평가 매개변수를 구성하여 검색 결과의 순위를 최적화하세요.

 

라가스 평가 QA 노트

QA 리콜은 일반적으로 두 가지 구성 요소로 이루어집니다:
검색 - 쿼리에 답변하는 데 필요한 가장 관련성 높은 정보를 검색합니다.
2. 생성기 - 검색된 정보를 사용하여 답을 생성합니다.

 

QA 정확도를 평가하는 것은 항상 큰 문제였습니다.루즈(요약 자동 평가를 위한 패키지그리고BLUE(기계 번역을 자동으로 평가하는 방법)는 효과가 떨어집니다.

현재 방법론: 강력한 LLM을 사용한 참조 없는 평가, Ragas에 대해 자세히 알아보세요.

Ragas는 LLM을 사용하여 QA 파이프라인을 평가할 때 발생하는 이러한 한계를 해결하고, 가능한 한 적은 비용으로 더 빠르고 효율적으로 주석이 달린 데이터를 사용하여 실행 가능한 지표를 제공하도록 설계되었습니다.

Ragas는 QA 파이프라인의 이러한 다양한 측면을 평가하는 데 도움이 되는 프레임워크입니다. QA 시스템의 다양한 측면을 평가할 수 있는 여러 가지 메트릭을 제공합니다.

 

검색을 평가하는 데 사용되는 지표입니다: 는 검색 시스템의 성능을 측정할 수 있는 척도를 제공합니다.
컨텍스트_관련성
context_recall

생성된 지표에 대한 평가: 착각을 측정하는 방법과 질문에 답하는 방법을 제공하세요.
충실성
답변_관련성

충실성

생성된 답변의 사실 정확도를 제공된 문맥과 비교하여 측정합니다.

이 작업은 두 단계로 이루어집니다.

먼저, 질문이 주어지고 답변이 생성되면 Ragas는 LLM을 사용하여 생성된 답변이 작성한 문장을 찾습니다. 이렇게 하면 유효성을 확인해야 하는 문장 목록이 제공됩니다. 2단계에서는 문장의 목록과 반환된 컨텍스트가 주어지면 Ragas는 LLM을 사용하여 제공된 문장이 컨텍스트에서 지원되는지 여부를 확인합니다. 올바른 문장의 수를 더하고 생성된 답안의 총 문장의 수로 나누어 주어진 예제에 대한 점수를 얻습니다.

1단계:

첫 번째 프롬프트에 따라

Given a question and answer, create one or more statements from answer.
question: Who was Albert Einstein and what is he best known for?
answer: He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics.
statements:
Albert Einstein was born in Germany.
Albert Einstein was best known for his theory of relativity.
question: Cadmium Chloride is slightly soluble in this chemical, it is also called what?
answer: alochol
statements:
Cadmium Chloride is slightly soluble in alcohol.
question: Were Shahul and Jithin of the same nationality?
answer: They were from different countries.
statements:
Shahul and Jithin were from different countries.
question:膳食分析是什么?
answer: 膳食分析是一种评估人们摄入的食物的方法，通过详细分析一周内所吃的食物，来确定人们摄入的营养素种类和数量。它可以揭示人们是否缺乏某些营养素，以及食物中是否存在一些对健康有害的物质，如糖、盐、咖啡因、酒精、食品添加剂和防腐剂。膳食分析可以提供关于脂肪、碳水化合物、蛋白质和热量的摄入量的评估。然而，膳食分析并不能全面考虑到食物中营养素含量的变异性、个人的营养素需求，以及营养素的吸收和利用情况。因此，它可能并不是评估个人的确切营养状况最准确的方法。
statements:

결과 1을 얻습니다:

['膳食分析是一种评估人们摄入的食物的方法。', 
'膳食分析可以确定人们摄入的营养素种类和数量。', 
'膳食分析可以揭示人们是否缺乏某些营养素。', 
'膳食分析可以揭示食物中是否存在一些对健康有害的物质。', 
'膳食分析可以提供关于脂肪、碳水化合物、蛋白质和热量的摄入量的评估。', 
'膳食分析可能并不是评估个人的确切营养状况最准确的方法。']

2단계:

두 번째 프롬프트에 따라

Prompt: Natural language inference
Consider the given context and following statements, then determine whether they are supported by the information present in the context.Provide a brief explanation for each statement before arriving at the verdict (Yes/No). Provide a final verdict for each statement in order at the end in the given format. Do not deviate from the specified format.
Context:
John is a student at XYZ University. He is pursuing a degree in Computer Science. He is enrolled in several courses this semester, including Data Structures, Algorithms, and Database Management. John is a diligent student and spends a significant amount of time studying and completing assignments. He often stays late in the library to work on his projects.
statements:
1. John is majoring in Biology.
2. John is taking a course on Artificial Intelligence.
3. John is a dedicated student.
4. John has a part-time job.
5. John is interested in computer programming.
Answer:
1. John is majoring in Biology.
Explanation: John's major is explicitly mentioned as Computer Science. There is no information suggesting he is majoring in Biology.  Verdict: No.
2. John is taking a course on Artificial Intelligence.
Explanation: The context mentions the courses John is currently enrolled in, and Artificial Intelligence is not mentioned. Therefore, it cannot be deduced that John is taking a course on AI. Verdict: No.
3. John is a dedicated student.
Explanation: The prompt states that he spends a significant amount of time studying and completing assignments. Additionally, it mentions that he often stays late in the library to work on his projects, which implies dedication. Verdict: Yes.
4. John has a part-time job.
Explanation: There is no information given in the context about John having a part-time job. Therefore, it cannot be deduced that John has a part-time job.  Verdict: No.
5. John is interested in computer programming.
Explanation: The context states that John is pursuing a degree in Computer Science, which implies an interest in computer programming. Verdict: Yes.
Final verdict for each statement in order: No. No. Yes. No. Yes.
context:
你想要达到什么样的健康程度？如果你想了解自己精神和身体的全部潜在能力，确定你的最佳营养需求是非常必要的。但是如果你的营养需求很特别，那么你该如何发现这一点呢？从1980年开始，我制定并完善了一套分析人体营养需求的体系，此体系建立在评估影响个人营养需求的主要因素的基础上。此体系已经在10万人当中进行了测试并得到了验证，现在已被世界各国的临床营养学家所应用。无数人已经从中获益，因此，我知道从这个体系中可以得到什么样的结果。这些结果包括：更敏锐的思维、更好的记忆力、更强的体力、更好的体重控制效果、降低胆固醇含量、使得病情好转。尽管多数被诊断为有病的人已经受益于个人健康计划，但这个健康计划是以预防为主要目标，而不是以治病为主的。如果你正在进行医药治疗，请确认这个营养补充方案与你接受的治疗措施是否能够很好地相容。影响你营养需求的因素，至少有8项因素会影响你的营养需求量。年龄、性别以及运动量都是很常见的影响因素，但污染、压力、基因遗传、你过去的健康状况，当然还有膳食所能提供的营养素以及反营养物质，却不是能够轻易了解的。但是所有的这些细节以及更多的因素，都必须纳入考虑当中。通过4种分析方法——膳食分析、生化分析、症状分析、生活方式分析，可以了解这些细节。膳食分析这种方式似乎是从容易着手的地方开始：搞清楚人们摄入的食物是什么，就能发现缺乏的营养素是什么。
但不幸的是，如果对一周内所吃的食物进行详细分析，并不能综合考虑到食物营养素含量的变异性、个人的营养素需求，或者营养素是否能被吸收、吸收后的营养素又是否能被有效地利用等情况。我见过许多人的饮食质量很高，却依然有维生素缺乏的症状。对其中绝大部分人来说，问题在于营养素吸收不良。这些可变因素，使得某些通过计算机进行的膳食分析不如设想的准确。膳食分析对于评估那些能影响到人们营养素需求的食物很有用处，比如糖、盐、咖啡、茶、酒精、食品添加剂和防腐剂。另一些因素，如脂肪、碳水化合物、蛋白质和热量的摄入量，也可由膳食分析进行评估。诸如头发矿物质分析与维生素血液分析这样的实验,可以提供关于体内生物化学状态的确切信息，也能让营养顾问了解你身体的确切营养状况。但并不是所有这些实验都能对规划你的营养方案提供有用的信息。为了让这些信息尽可能精确，任何关于维生素或矿物质的检测都必须反映出营养素在体内发挥作用的能力。例如，铁是构成血红细胞的必需元素，有助于体内氧的运输。通过测定细胞中铁的营养状况，可能会对你的铁需求有一个很好的估测。从另一方面来讲，维生素氏在血液中没有像铁那样的直接作用。它主要作用于大脑当中的化学物质5-羟色胺，或在甲基化反应中起作用，降低同型半胱氨酸含量。所以，仅仅测定血液中的维生素B6,并不能真实地反映你是否获得了充足的营养素供给以及体内的维生素B6是否运转正常。相比之下，检测同型半胱氨酸浓度等项目意义更大。如果你已经获得了充足的维生素、维生素B和叶酸，那么你的同型半胱氨酸浓度就很低，但假如同型半胱氨酸浓度很高，则表明你需要更多的其中一种或多种营养素。我们称这种测试方式为功能测试，因为这个测试能够精确地测定你体内生化反应的某项特殊功能是否运转正常。
statements:
1.膳食分析是一种评估人们摄入的食物的方法。
2.膳食分析可以确定人们摄入的营养素种类和数量。
3.膳食分析可以揭示人们是否缺乏某些营养素。
4.膳食分析可以揭示食物中是否存在一些对健康有害的物质。
5.膳食分析可以提供关于脂肪、碳水化合物、蛋白质和热量的摄入量的评估。
6.膳食分析可能并不是评估个人的确切营养状况最准确的方法。
Answer:

결과 2를 가져옵니다:

1. 膳食分析是一种评估人们摄入的食物的方法.
explanation: the context mentions that dietary analysis is a method to understand what food people consume. verdict: yes.
2. 膳食分析可以确定人们摄入的营养素种类和数量.
explanation: the context mentions that dietary analysis can evaluate the intake of nutrients such as sugar, salt, coffee, tea, alcohol, food additives, and preservatives. it does not explicitly state that it can determine the types and quantities of nutrients consumed. verdict: no.
3. 膳食分析可以揭示人们是否缺乏某些营养素.
explanation: the context mentions that dietary analysis can be useful in assessing the intake of nutrients that can affect people's nutritional needs. this implies that it can reveal if someone is lacking certain nutrients. verdict: yes.
4. 膳食分析可以揭示食物中是否存在一些对健康有害的物质.
explanation: the context mentions that dietary analysis can be useful in evaluating the intake of substances that can affect people's nutritional needs, such as food additives and preservatives. this implies that it can reveal the presence of harmful substances in food. verdict: yes.
5. 膳食分析可以提供关于脂肪、碳水化合物、蛋白质和热量的摄入量的评估.
explanation: the context mentions that the intake of factors such as fat, carbohydrates, protein, and calories can be evaluated through dietary analysis. verdict: yes.
6. 膳食分析可能并不是评估个人的确切营养状况最准确的方法.
explanation: the context mentions that while dietary analysis can provide useful information, it may not be the most accurate method for assessing an individual's exact nutritional status. verdict: yes.
final verdict for each statement in order: yes. no. yes. yes. yes. yes.

groundfinal verdict for each statement in order:에서 찾기yes. no. yes. yes. yes. yes.

포인트 계산:

output = "yes. no. yes. yes. yes. yes."
score = sum(0 if "yes" in answer else 1
for answer in output.strip().split(".")
if answer != ""
)
# score = 1 - score/len(statements)
socre = 1 - 1/6
# score = 0.8333333333333334

이것이 페이스풀네스의 최종 점수입니다.

답변_관련성

질문에 대한 답변의 관련성과 집중도를 측정합니다.

주어진 생성된 답변에 대해 Ragas는 LLM을 사용하여 생성된 답변이 답이 될 수 있는 가능한 질문을 찾고 실제 질문과의 유사성을 계산합니다.

답변에 따르면

Generate question for the given answer.
Answer:
The PSLV-C56 mission is scheduled to be launched on Sunday, 30 July 2023 at 06:30 IST / 01:00 UTC. It will be launched from the Satish Dhawan Space Centre, Sriharikota, Andhra Pradesh, India 
Question: When is the scheduled launch date and time for the PSLV-C56 mission, and where will it be launched from?
Answer:膳食分析是一种评估个人饮食中摄入的食物种类和数量的方法。通过对一周内所吃的食物进行详细分析，可以了解人们摄入的营养素种类和含量，从而发现可能存在的营养素缺乏或过量的情况。膳食分析对于评估食物中的营养素含量、摄入量以及一些影响营养需求的因素（如糖、盐、咖啡因、酒精等）是有用的。然而，膳食分析并不能综合考虑到个体的营养素需求、营养素吸收和利用的情况，因此并不是评估个人营养状况的唯一方法。
Question:

Get:

[
"What is dietary analysis and how is it used to evaluate an individual's nutritional intake?", 
"What is dietary analysis and how is it used to assess an individual's nutritional intake?", 
"What is dietary analysis and how is it used to assess an individual's nutritional intake?"
]

그런 다음 [문제]와 이 세 결과 사이의 유사성을 계산합니다:

유사도 계산 프로세스: 사용text-embedding-ada-002벡터화를 수행한 다음 코사인 유사도를 계산합니다.

def calculate_similarity(
self: t.Self, question: str, generated_questions: list[str]
):
question_vec = np.asarray(self.embedding.embed_query(question)).reshape(1, -1)
gen_question_vec = np.asarray(
self.embedding.embed_documents(generated_questions)
)
norm = np.linalg.norm(gen_question_vec, axis=1) * np.linalg.norm(
question_vec, axis=1)
return (
np.dot(gen_question_vec, question_vec.T).reshape(-1,)/ norm
)

Get:

[0.83663467 0.83484782 0.83484782]

그런 다음 평균:

answer_relevancy_score: 0.8354434364200779

컨텍스트_관련성

검색된 문맥에서 신호 대 잡음비를 측정합니다. 질문이 주어지면 Ragas는 LLM을 호출하여 검색된 문맥에서 질문에 답하는 데 필요한 문장을 찾습니다. 필요한 문장과 문맥의 전체 문장 사이의 비율은 점수를 제공합니다.

입력:

Task: Candidate sentence extraction.
Given the question and context, extract minimum number of sentences from context required to answer the question. If the context do not contain information required to answer the question return "No candidate sentences found".
question: Which equation is known as worlds most famous equation?
context:
Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist,[5] widely ranked among the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century.
His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called "the world's most famous equation".
sentences:His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called "the world's most famous equation".
question: Were Scott Derrickson and Ed Wood of the same nationality?
context :
Scott Derrickson (born July 16, 1966) is an American director, screenwriter and producer He lives in Los Angeles, California He is best known for directing horror films such as "Sinister", "The Exorcism of Emily Rose", and "Deliver Us From Evil", as well as the 2016 Marvel Cinematic Universe installment, "Doctor Strange"Tyler Bates is an American musician, music producer, and composer for films, television, and video games. Adam Collis is an American filmmaker and actor.Conrad Brooks is an American actor.Edward Davis Wood Jr. (October 10, 1924 – December 10, 1978) was an American filmmaker, actor, writer, producer, and director.
sentences:Scott Derrickson (born July 16, 1966) is an American director, screenwriter and producer. Edward Davis Wood Jr. (October 10, 1924 – December 10, 1978) was an American filmmaker, actor, writer, producer, and director.
question: How many were killed in the Tiananmen Square incident?
context:
Tiananmen Square incident, also called June Fourth incident or 6/4, series of protests and demonstrations in China in the spring of 1989 that culminated on the night of June 3–4 with a government crackdown on the demonstrators in Tiananmen Square in Beijing.
sentences: No candidate sentences found.
question:膳食分析是什么?
context:
你想要达到什么样的健康程度？如果你想了解自己精神和身体的全部潜在能力，确定你的最佳营养需求是非常必要的。但是如果你的营养需求很特别，那么你该如何发现这一点呢？从1980年开始，我制定并完善了一套分析人体营养需求的体系，此体系建立在评估影响个人营养需求的主要因素的基础上。此体系已经在10万人当中进行了测试并得到了验证，现在已被世界各国的临床营养学家所应用。无数人已经从中获益，因此，我知道从这个体系中可以得到什么样的结果。这些结果包括：更敏锐的思维、更好的记忆力、更强的体力、更好的体重控制效果、降低胆固醇含量、使得病情好转。尽管多数被诊断为有病的人已经受益于个人健康计划，但这个健康计划是以预防为主要目标，而不是以治病为主的。如果你正在进行医药治疗，请确认这个营养补充方案与你接受的治疗措施是否能够很好地相容。影响你营养需求的因素，至少有8项因素会影响你的营养需求量。年龄、性别以及运动量都是很常见的影响因素，但污染、压力、基因遗传、你过去的健康状况，当然还有膳食所能提供的营养素以及反营养物质，却不是能够轻易了解的。但是所有的这些细节以及更多的因素，都必须纳入考虑当中。通过4种分析方法——膳食分析、生化分析、症状分析、生活方式分析，可以了解这些细节。膳食分析这种方式似乎是从容易着手的地方开始：搞清楚人们摄入的食物是什么，就能发现缺乏的营养素是什么。
但不幸的是，如果对一周内所吃的食物进行详细分析，并不能综合考虑到食物营养素含量的变异性、个人的营养素需求，或者营养素是否能被吸收、吸收后的营养素又是否能被有效地利用等情况。我见过许多人的饮食质量很高，却依然有维生素缺乏的症状。对其中绝大部分人来说，问题在于营养素吸收不良。这些可变因素，使得某些通过计算机进行的膳食分析不如设想的准确。膳食分析对于评估那些能影响到人们营养素需求的食物很有用处，比如糖、盐、咖啡、茶、酒精、食品添加剂和防腐剂。另一些因素，如脂肪、碳水化合物、蛋白质和热量的摄入量，也可由膳食分析进行评估。诸如头发矿物质分析与维生素血液分析这样的实验,可以提供关于体内生物化学状态的确切信息，也能让营养顾问了解你身体的确切营养状况。但并不是所有这些实验都能对规划你的营养方案提供有用的信息。为了让这些信息尽可能精确，任何关于维生素或矿物质的检测都必须反映出营养素在体内发挥作用的能力。例如，铁是构成血红细胞的必需元素，有助于体内氧的运输。通过测定细胞中铁的营养状况，可能会对你的铁需求有一个很好的估测。从另一方面来讲，维生素氏在血液中没有像铁那样的直接作用。它主要作用于大脑当中的化学物质5-羟色胺，或在甲基化反应中起作用，降低同型半胱氨酸含量。所以，仅仅测定血液中的维生素B6,并不能真实地反映你是否获得了充足的营养素供给以及体内的维生素B6是否运转正常。相比之下，检测同型半胱氨酸浓度等项目意义更大。如果你已经获得了充足的维生素、维生素B和叶酸，那么你的同型半胱氨酸浓度就很低，但假如同型半胱氨酸浓度很高，则表明你需要更多的其中一种或多种营养素。我们称这种测试方式为功能测试，因为这个测试能够精确地测定你体内生化反应的某项特殊功能是否运转正常。
sentences:

Get:

[
'膳食分析这种方式似乎是从容易着手的地方开始：搞清楚人们摄入的食物是什么，就能发现缺乏的营养素是什么。', 
'膳食分析是从容易着手的地方开始：搞清楚人们摄入的食物是什么，就能发现缺乏的营养素是什么。'
]

각 결과를 분할하여 컨텍스트에서 조회하고 각 결과의 분할된 절 중 몇 퍼센트의 절이 원본 텍스트 overlap_scores에 나타나는지 확인합니다.
다음과 같이 가정합니다.[1.0, 1.0].

컨텍스트:

你想要达到什么样的健康程度？如果你想了解自己精神和身体的全部潜在能力，确定你的最佳营养需求是非常必要的。但是如果你的营养需求很特别，那么你该如何发现这一点呢？从1980年开始，我制定并完善了一套分析人体营养需求的体系，此体系建立在评估影响个人营养需求的主要因素的基础上。此体系已经在10万人当中进行了测试并得到了验证，现在已被世界各国的临床营养学家所应用。无数人已经从中获益，因此，我知道从这个体系中可以得到什么样的结果。这些结果包括：更敏锐的思维、更好的记忆力、更强的体力、更好的体重控制效果、降低胆固醇含量、使得病情好转。尽管多数被诊断为有病的人已经受益于个人健康计划，但这个健康计划是以预防为主要目标，而不是以治病为主的。如果你正在进行医药治疗，请确认这个营养补充方案与你接受的治疗措施是否能够很好地相容。影响你营养需求的因素，至少有8项因素会影响你的营养需求量。年龄、性别以及运动量都是很常见的影响因素，但污染、压力、基因遗传、你过去的健康状况，当然还有膳食所能提供的营养素以及反营养物质，却不是能够轻易了解的。但是所有的这些细节以及更多的因素，都必须纳入考虑当中。通过4种分析方法——膳食分析、生化分析、症状分析、生活方式分析，可以了解这些细节。膳食分析这种方式似乎是从容易着手的地方开始：搞清楚人们摄入的食物是什么，就能发现缺乏的营养素是什么。
但不幸的是，如果对一周内所吃的食物进行详细分析，并不能综合考虑到食物营养素含量的变异性、个人的营养素需求，或者营养素是否能被吸收、吸收后的营养素又是否能被有效地利用等情况。我见过许多人的饮食质量很高，却依然有维生素缺乏的症状。对其中绝大部分人来说，问题在于营养素吸收不良。这些可变因素，使得某些通过计算机进行的膳食分析不如设想的准确。膳食分析对于评估那些能影响到人们营养素需求的食物很有用处，比如糖、盐、咖啡、茶、酒精、食品添加剂和防腐剂。另一些因素，如脂肪、碳水化合物、蛋白质和热量的摄入量，也可由膳食分析进行评估。诸如头发矿物质分析与维生素血液分析这样的实验,可以提供关于体内生物化学状态的确切信息，也能让营养顾问了解你身体的确切营养状况。但并不是所有这些实验都能对规划你的营养方案提供有用的信息。为了让这些信息尽可能精确，任何关于维生素或矿物质的检测都必须反映出营养素在体内发挥作用的能力。例如，铁是构成血红细胞的必需元素，有助于体内氧的运输。通过测定细胞中铁的营养状况，可能会对你的铁需求有一个很好的估测。从另一方面来讲，维生素氏在血液中没有像铁那样的直接作用。它主要作用于大脑当中的化学物质5-羟色胺，或在甲基化反应中起作用，降低同型半胱氨酸含量。所以，仅仅测定血液中的维生素B6,并不能真实地反映你是否获得了充足的营养素供给以及体内的维生素B6是否运转正常。相比之下，检测同型半胱氨酸浓度等项目意义更大。如果你已经获得了充足的维生素、维生素B和叶酸，那么你的同型半胱氨酸浓度就很低，但假如同型半胱氨酸浓度很高，则表明你需要更多的其中一种或多种营养素。我们称这种测试方式为功能测试，因为这个测试能够精确地测定你体内生化反应的某项特殊功能是否运转正常。

그런 다음 bert 또는 jaccard를 사용하여 두 답변의 유사도를 계산하여 agr_score를 구합니다.
그런 다음 context_relevancy를 계산할 수 있습니다:

context_relevancy = agr_score * (overlap_scores的平均值)

context_recall

이 기능을 테스트하려면 먼저ground_truth.
안내를 따르세요:

Given a context, and an answer, analyze each sentence in the answer and classify if the sentence can be attributed to the given context or not.
Think in steps and reason bofore coming to conclusion. 
context: Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist,widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called "the world's most famous equation". He received the 1921 Nobel Prize in Physics "for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect", a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.
answer: Albert Einstein born in 14 March 1879 was  German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics "for his services to theoretical physics. He published 4 papers in 1905.  Einstein moved to Switzerland in 1895 
classification
1. Albert Einstein born in 14 March 1879 was  German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. The date of birth of Einstein is mentioned clearly in the context. So [Attributed]
2. He received the 1921 Nobel Prize in Physics "for his services to theoretical physics. The exact sentence is present in the given context. So [Attributed]
3. He published 4 papers in 1905. There is no mention about papers he wrote in given the context. So [Not Attributed]
4. Einstein moved to Switzerland in 1895. There is not supporting evidence for this in the given the context. So [Not Attributed]
context:{context}
answer:{ground_truth}
classification:

결과 얻기:

[
'#############[Attributed]',
'#############[Attributed]',
'#############[Not Attributed]',
……
]

컨텍스트_리콜인 [어트리뷰티드] 발생의 백분율을 계산합니다.

개인적인 견해:

이 라이브러리 정보: 오픈AI에 대한 auzre 액세스를 지원하지 않으며, 소스 코드를 변경해야 합니다.

네 가지 지표에서

context_relevancy: 의심스러움(agr_score의 유사성 계산에서 stsb-TinyBERT-L-4 중국어 모델의 효과에 대한 불확실성)
context_recall: 어떤 시나리오를 사용할지 잘 모르겠습니다 ......
충실함.
답변_관련성: 신뢰할 수 있는 가능성 (text-embedding-ada-002(벡터화 후 시맨틱의 정밀도)