Learn how Rexera migrated to LangGraph to create powerful quality control intelligence for real estate business processes and significantly improve the accuracy of its Large Language Model (LLM) responses.
Rexera is revolutionizing the $50 billion real estate transaction industry by automating manual processes with AI. By deploying intelligent AI intelligence, Rexera is streamlining real estate business processes and speeding up transactions while significantly reducing costs and error rates.
With LangChain and LangGraph and its Large Language Models (LLMs), Rexera develops sophisticated AI intelligences. These AI intelligences are capable of performing complex cognitive tasks such as:
- Order Payment Settlement Statement
- Extracting key information from documents
- Perform quality control checks
Below, we'll explore how Rexera created a powerful quality control (QC) application that can review real estate business processes as accurately as a human operator and proactively identify issues to prevent delays. We'll explore how their system evolved from an initial single-prompt approach to a more controlled and accurate solution using LangGraph.
Initial Approach: Single-Prompt LLM Checks and Their Limitations
Quality control is critical in real estate transactions, and Rexera has developed a specialized QC application that reviews thousands of workflows every day. This application checks for possible errors at all stages of a real estate transaction, including data processing, client communication, and interactions with homeowners' associations (HOAs), county offices, utility companies, and other interested parties.
To ensure the quality of real estate transactions, Rexera initially implemented multiple single-prompt LLM checks. These checks were designed to validate:
- Accuracy of documentation
- Fulfillment of customer expectations
- Timeliness of workflow (SLA compliance)
- cost control
However, this approach has limitations. Single-prompt LLMs struggle to cope with the complexity of the real estate transaction process for a number of reasons, including the fact that they do not have a full picture of the entire workflow, have limited context, and cannot properly handle multidimensional scenarios.
Rexera evaluates the effectiveness of LLM inspections using the following three key metrics, tested in thousands of workflows:
- accuracy: Correctness scores for problem identification
- efficiency: Speed of execution per transaction
- cost-effectiveness: Costs associated with LLM
This approach simplified quality control by flagging potential issues and reducing the need for manual review. However, Rexera recognized the need for a more advanced solution to effectively handle complex real estate workflows.
Evolving to an AI Intelligence: Try CrewAI
Realizing the limitations of single-prompt LLMs, Rexera tried using the CrewAI of a multi-intelligence approach, where each AI intelligence is responsible for a different part of the transaction process. For example, an intelligence might be defined as:
- Role: "Senior Content Quality Check Analyst"
- TASK: "Check that all HOA documents have been ordered as requested by the customer and verify that the appropriate ETA and cost information has been sent to the customer."
This approach brings a number of improvements over single cue LLM:
- misinformation(incorrectly labeled as non-issue) decreased from 35% to 8%.
- omission(Failure to Flag Real Problems) decreased from 10% to 5%.
However, the CrewAI approach also faces a key challenge. Despite their power, AI intelligences can sometimes take the wrong path in their decision-making, much like a GPS system choosing a longer route. This lack of precise control means that in complex scenarios, intelligences can go off track, leading to false or missed alarms.
Migrate to LangGraph for Accuracy and Control
To overcome the limitations of the CrewAI approach, Rexera turned to LangGraph to customize the design of decision paths for various scenarios, which is especially beneficial when dealing with complex cases.LangGraph is a framework for controlled intelligences built by the LangChain team, which brings additional benefits to Rexera, including human-involved workflow integration, state management, and more. LangGraph
To illustrate the effectiveness of the new LangGraph-based approach, let's use the example of an expedited order. This is a common complication in real estate business processes that require transactions to be completed faster than the standard schedule.
Using LangGraph, Rexera has created a tree-like system for quality control (QC) applications that supports looping and branching. This structure enables QC applications to navigate different paths based on expediting requirements.
When the application recognizes an expedited order, it follows the "Expedited Orders" branch of the tree structure. For standard orders, the application follows another branch and focuses on regular processing checks.
This LangGraph-enabled tree structure dramatically improves the accuracy and integrity of Rexera's QC process by introducing a more deterministic decision-making process and reducing the randomness associated with incorrect path selection. This change ensures consistency of results and enables workflows to be completed accurately and efficiently.
By applying LangGraph to existing CrewAI improvements, the following optimizations were achieved:
- False positive rate increased from 8% Reduced to 2%
- False-negative rate increased from 5% Reduced to 2%
For the same expedited order scenario, the following examples analyze the QC results under each new architecture:
Large Language Model (LLM) output for single prompts:
- Problems found: "True"
- Explanation: "We did not explicitly acknowledge the expedited request in our communications with the client."
- Analysis: false positive --While the expedited request was actually acknowledged and executed, LLM failed to recognize this due to its limited ability to handle complex multi-step interactions, thus incorrectly flagging the issue.
CrewAI Output:
- Problems found: "False"
- Explanation: "The customer requested an expedited order and the team has acknowledged and processed the request."
- Analysis: partially accurate --CrewAI recognized the correct result that the expedited order had been executed, but did not note differences in the type of order, such as whether or not it was properly flagged and processed as an expedited order in the system.
LangGraph output:
- Problems found: "True"
- Explanation: "The order details show 'Expedited Order: no', even though the customer requested an expedite and we have recognized and executed the expedited order."
- Analysis: Completely accurate --LangGraph not only confirmed that expedited orders were acknowledged and executed, but also identified inconsistencies in the order type records. By following a customized decision path, it ensured that both the acknowledgement of the expedited request and the correct processing of the order type were validated, thus avoiding potential delays or problems.
reach a verdict
By leveraging LangGraph's looping and branching capabilities, Rexera has created a smarter and more adaptable QC application that ensures its AI agents are able to perform the right inspection for each specific scenario, improving efficiency and accuracy in real estate transactions.