LangWatch: A Visualization Tool for Monitoring and Optimizing LLM Processes Based on the DSPy Framework

General Introduction

LangWatch is a comprehensive platform designed for Large Language Model (LLM) operations, providing monitoring, analysis, evaluation, dataset management, and cued optimization. The platform is based on Stanford University's DSPy framework and is designed to help users better manage and optimize LLM pipelines.LangWatch provides an intuitive drag-and-drop interface that enables users to easily perform experiment tracking and version control. In addition, the platform supports multiple evaluation tools and custom evaluation builders to ensure model quality and compliance. Whether it's for real-time debugging, performance tracking, or user analytics, LangWatch provides a comprehensive solution.

Function List

• Visual Interface: Provides a drag-and-drop interface for LLM pipeline optimization.
• Based on the DSPy framework: Built on top of Stanford University's DSPy framework.
• Auto-generation: Automatically generates tips and a few examples.
• Experiment tracking: Visualization of experiment tracking and version control.
• Evaluators: More than 30 off-the-shelf evaluators are available, and custom evaluation builders are supported.
• Dataset Management: Provides complete dataset management functionality.
• Compliance Checks: Provides compliance and security checks.
• DSPy Visualization Tools: Built-in DSPy visualization tools.

Using Help

Installation and Setup

local installation

1. clone warehouse::

   git clone https://github.com/langwatch/langwatch.git

2. Copy the environment configuration file::

   cp langwatch/.env.example langwatch/.env

3. Build and start the container::

   docker compose up --build

4. Access platforms: Open your browser and visit http://localhost:5560.

Development Environment Settings

1. Starting the database::

   docker compose up redis postgres opensearch

2. Install dependencies and launch the platform::

   make install
make start

Function Operation Guide

Optimization Studio

• drag and drop interface: With drag-and-drop components, users can easily build and optimize LLM pipelines.
• Experimental tracking: Record detailed information about each experiment for version control and comparison of results.

quality assurance (QA)

• Built-in evaluation tools: The platform has more than 30 built-in assessment tools, allowing users to choose the appropriate assessment method according to their needs.
• Custom Evaluation Builder: Users can customize the assessment criteria and methodology according to their specific needs.

Data set management

• Data upload and management: Users can upload and manage their own datasets, and the platform provides comprehensive data compliance checks.
• data security: Ensure the security of data during transmission and storage.

Monitoring and Analysis

• real time debugging: Provide real-time debugging tools to help users quickly locate and solve problems.
• Performance Tracking: Detailed records of the model's performance metrics to help users optimize the model.
• user analysis: Provide user behavior analysis tools to help users understand how the model is being used.
• Customized business metrics: Users can customize monitoring metrics and alerts according to business needs.

LangWatch Cloud

• Register & Login: Users can register for a free account on the LangWatch Cloud Platform and log in to access the full functionality of the platform.
• cloud service: Provides high availability cloud services so users don't have to worry about infrastructure maintenance and management.

Self-hosted support

• Business Support: LangWatch provides commercial support to help users self-host the platform on their own infrastructure.
• Detailed Documentation: The platform provides detailed self-hosted documentation to help users successfully complete installation and configuration.