Laminar: open source AI product engineering platform for easy data visualization tracking and evaluation

General Introduction

Laminar is an open source AI engineering optimization platform focused on AI engineering from first principles. It helps users collect, understand and use data to improve the quality of LLM (Large Language Model) applications.Laminar provides comprehensive observability, text analytics, evaluation and cue chain management capabilities to support users in building and optimizing complex AI products. Whether it's data tracking, online evaluation or dataset building, Laminar provides powerful support to help users achieve efficient AI development and deployment.

Its modern, open source technology stack includes Rust, RabbitMQ, Postgres, Clickhouse, and more to ensure high performance and low overhead. Users can deploy quickly with Docker Compose or enjoy full functionality using a hosted platform.

DEMO: https://www.lmnr.ai/

Function List

• Data tracking: Document each step of the execution of the LLM application and collect valuable data for better evaluation and fine-tuning.
• Online Assessment: Set up LLM as a rater or use a Python script evaluator for each received span.
• Data set construction: Constructing datasets from tracking data for evaluating, fine-tuning, and prompting engineering.
• Cue Chain Management: Support for building and hosting complex cue chains, including agent hybrid or self-reflexive LLM pipelines.
• Open source and self-hosted: Completely open source, easily self-hosted, and ready to go with just a few commands.

Using Help

Installation process

1. Cloning GitHub repositories:

   git clone https://github.com/lmnr-ai/lmnr
   

2. Go to the project catalog:

   cd lmnr
   

3. Use Docker Compose to start:

   docker compose up -d
   

Function Operation Guide

Data tracking

• initialization: Import Laminar in the code and initialize the project API key.

  from lmnr import Laminar, observe
  Laminar.initialize(project_api_key="...")
  

• comment function: Use@observeAnnotate functions that need to be traced.

  @observe()
  def my_function():
  ...
  

Online Assessment

• Setting up the Evaluator: The LLM can be set up to act as a rater or use a Python script evaluator to evaluate and label each received span.

  # Example code
  evaluator = LLMJudge()
  evaluator.evaluate(span)
  

Data set construction

• Creating Data Sets: Construct datasets from tracking data for subsequent evaluation and fine-tuning.

  dataset = create_dataset_from_traces(traces)
  

Cue Chain Management

• Build a cue chain: Support for building complex cue chains, including agent mixing or self-reflective LLM pipelines.

  chain = PromptChain()
  chain.add_prompt(prompt)
  

self-hosted

• Self-hosting Steps: To start self-hosting with just a few commands, make sure Docker and Docker Compose are installed in your environment.

  git clone https://github.com/lmnr-ai/lmnr
  cd lmnr
  docker compose up -d