LangExtract - Google's open source Python library to extract structured information

What is LangExtract?

LangExtract is a Google Open Source Python library that uses large language models (LLMs) to extract structured information from unstructured text. With user-defined commands and a small number of examples, it can efficiently identify and organize key details, such as the names of drugs from clinical notes or character relationships from literature, etc. LangExtract's core strength is its precise source text localization, which maps each extraction to the exact location of the original text, and supports visual highlighting for easy traceability and verification. Supporting multiple language models, including cloud models and local open-source models, LangExtract can handle long documents and optimize extraction efficiency.LangExtract provides interactive visualization functions, and can generate independent HTML files, making it easy for users to view and review the extraction results in the original context. LangExtract can be used in a variety of fields such as healthcare, literature, finance, etc., helping users to quickly extract valuable information from complex text.

LangExtract's main functions

• text extraction: Extract key information from unstructured text and support many types of data such as clinical notes, reports, etc.
• precise positioning: Accurately maps extracted content to source text locations and supports visual highlighting for easy traceability and verification.
• Structured Output: Output the extracted information in a structured format (e.g., JSONL) to facilitate subsequent processing and analysis.
• Long Document Optimization: Efficiently process ultra-long documents and improve recall through text chunking and multi-round extraction strategies.
• Interactive Visualization: Generate interactive HTML files that make it easy for users to view and review the extraction results in their original context.
• Flexible model support: Multiple language models are supported, including cloud-based models (e.g. Google Gemini) and local open source models.
• Domain Adaptation: Extraction tasks for any domain can be defined with a small number of examples, without the need to fine-tune the model, for multiple domains such as healthcare, literature, and finance.
• Efficient processing: Supports parallel processing, improves extraction efficiency, and is suitable for large-scale text processing tasks.

LangExtract's project address

• Project website:: https://pypi.org/project/langextract/
• GitHub repository:: https://github.com/google/langextract

How to use LangExtract

• Installing LangExtract: Install the LangExtract library with pip, Python's package management tool.
• Define the extraction task: Develop extraction instructions based on requirements, specify the type of information to be extracted, and prepare a small amount of sample data.
• configuration model: Choose the appropriate language model, either a cloud model (e.g. Google Gemini) or a local model (e.g. through the Ollama interface).
• Write code: Write code using the API provided by LangExtract to load the model and invoke the extraction function.
• Operational extraction: Execute the code to perform the extraction operation on the target text, LangExtract will perform the information extraction according to the defined task and model.
• Save results: Save the extraction results in a structured format (e.g., JSONL file) for easy subsequent processing.
• Generate visualization reports: Use the tools provided by LangExtract to generate interactive HTML visualization reports for easy viewing and validation of extraction results.
• Optimization and Adjustment: Adjust the extraction instructions or model parameters to optimize the extraction results according to the accuracy and demand of the extraction results.

LangExtract's core strengths

• Accurate source text positioning: It can precisely map each extraction to its position in the original text, and supports visual highlighting for easy traceability and verification.
• Flexible model adaptation: Multiple language models are supported, including cloud models (e.g., Google Gemini) and local open source models (e.g., through the Ollama interface), adapting to the needs of different scenarios.
• Long Document Optimization Processing: Optimized for very long documents, it improves extraction efficiency and recall through text chunking, parallel processing and multi-round extraction strategies.
• Interactive Visualization: Provides interactive HTML visualization reports generated with a single click, making it easy for users to view and review the extraction results in their original context.
• Efficient structured output: Enforcing a consistent output pattern based on a small number of examples ensures that the extraction results are structured and robust.
• Highly adaptable to the field: Define extraction tasks for any domain with only a few examples, without fine-tuning the model, for a wide range of domains such as healthcare, literature, finance, and more.

Who LangExtract is for

• Data Analyst: The need to extract valuable information from large amounts of textual data for data analysis and report generation.
• Medical Industry Practitioners: e.g., doctors, nurses, medical researchers, for processing medical texts such as clinical notes, medical records, etc.
• Legal professionals: e.g. lawyers, legal staff, for analyzing legal documents, contracts, etc. and extracting key terms and information.
• Financial industry personnel: e.g., financial analysts, risk managers, for processing financial reports and transaction records.
• Academic researchers: Data and conclusions need to be extracted from academic literature for research and synthesis.
• literary researcher: Used to analyze literary works and extract information about characters, plot, and theme.