Eino: ByteDance's open source Golang big model application development framework

General Introduction

Eino is a Golang-based open source framework launched by CloudWeGo team, aiming to be the ultimate development tool for Large Model (LLM) applications. It draws on the excellent design of open source frameworks such as LangChain and LlamaIndex, and combines cutting-edge research results and ByteDance's internal practical experience to provide a concise, scalable, and efficient development experience.Eino helps developers quickly build reliable AI applications through well-designed component abstractions, powerful orchestration capabilities, complete streaming support, and type safety features. Eino Eino has been widely used within ByteDance (e.g., Jitter, Doubao), and is now open-sourced to GitHub, with high code coverage and an active community dedicated to promoting the standardization and efficiency of AI development in the Golang ecosystem.

Function List

• Rich component abstraction: Provides reusable components such as ChatModel, Tool, ChatTemplate, etc., supporting nesting and complex logic encapsulation.
• Powerful orchestration capabilities: Supports both Chain and Graph modes, handling type checking, concurrency management, and data flow orchestration.
• Complete Streaming Processing: Automatically handle splicing, splitting, merging and copying of streaming data, adapting to multiple streaming paradigms.
• Highly scalable cutouts: Built-in callback mechanism to support injection of cross-cutting concerns such as logging and tracing.
• type safety: Utilize Golang compile-time checks to ensure that inputs and outputs match.
• Development Toolset: Provide visual debugging, evaluators, etc., covering the full development cycle.
• Best Practice Examples: Built-in ReAct Agent and other process realization, with rich sample code.

Using Help

Installation process

Eino is a Golang framework that requires a Go environment for installation. Here are the detailed steps:

1. Install Go
   • Visit the Go website to download version 1.18 or above.
   • After the installation is complete, run go version Check the version (e.g. go1.21.0).
2. Pulling Eino Core Libraries
   • Runs in the terminal:

     go get github.com/cloudwego/eino@latest
     

   • If you need to extend the component (e.g. OpenAI support), run:

     go get github.com/cloudwego/eino-ext@latest
     

3. Verify Installation
   • Creating Documents main.go::

     package main
     import "github.com/cloudwego/eino"
     func main() {
     println("Eino installed successfully!")
     }
     

   • fulfillment go run main.goIf the output is successful, the process is complete.
4. dependency statement
   • Eino Dependencies kin-openapi version v0.118.0 for Go 1.18 compatibility.
   • Optional:EinoExt Provide more component implementations.
5. Community Support
   • Visit GitHub Issues to submit an issue or join the Flying Book user group (see QR code on official page).

Main function operation flow

1. Use ChatModel directly

ChatModel is the base component, which is used to call the big model to generate content.

• code example

  package main
  import (
  "context"
  "github.com/cloudwego/eino-ext/components/model/openai"
  "github.com/cloudwego/eino/schema"
  )
  func main() {
  ctx := context.Background()
  config := &openai.Config{APIKey: "Your OpenAI-API-Key"}
  model, err := openai.NewChatModel(ctx, config)
  if err ! = nil {
  panic(err)
  }
  messages := []*schema.Message{
  {Role: schema.System, Content: "You are an assistant."} ,
  {Role: schema.User, Content: "What will AI apps look like in the future?"} ,
  }
  msg, err := model.Generate(ctx, messages)
  if err ! = nil {
  panic(err)
  }
  println(msg.Content)
  }

• Operating Instructions
  • expense or outlay NewChatModel Initialize the model instance, passing in the context and API configuration.
  • invocations Generate method generates non-streaming output.
  • Other models (e.g. LLaMA) can be supported through extension components.

2. Orchestrating simple processes with Chain

Chain is suitable for linear processes such as templates + models.

• code example

  package main
  import (
  "context"
  "github.com/cloudwego/eino"
  "github.com/cloudwego/eino-ext/components/model/openai"
  )
  func main() {
  ctx := context.Background()
  model, _ := openai.NewChatModel(ctx, &openai.Config{APIKey: "Your API-Key"})
  prompt := eino.NewChatTemplate("Hello, my name is {{.name}}")
  chain, _ := eino.NewChain[map[string]any, *eino.Message]().
  AppendChatTemplate(prompt).
  AppendChatModel(model).
  Compile(ctx)
  result, _ := chain.Invoke(ctx, map[string]any{"name": "Eino"})
  println(result.Content)
  }
  

• Operating Instructions
  • NewChain Define the input and output types thatAppend Add node.
  • Compile The compilation chain.Invoke Execute and return the result.
  • Good for simple back-and-forth sequential tasks.

3. Using Graph for complex organization

Graph supports cyclic or acyclic directed graphs, suitable for scenarios such as tool invocation.

• code example

  package main
  import (
  "context"
  "github.com/cloudwego/eino"
  "github.com/cloudwego/eino-ext/components/model/openai"
  "github.com/cloudwego/eino/schema"
  )
  func main() {
  ctx := context.Background()
  model, _ := openai.NewChatModel(ctx, &openai.Config{APIKey: "Your API-Key"})
  graph := eino.NewGraph[map[string]any, *schema.Message]()
  graph.AddChatModelNode("model", model)
  graph.AddEdge(eino.START, "model")
  graph.AddEdge("model", eino.END)
  runnable, _ := graph.Compile(ctx)
  result, _ := runnable.Invoke(ctx, map[string]any{"query": "Hello, world!"})
  println(result.Content)
  }
  

• Operating Instructions
  • Add*Node Adding nodes.AddEdge Define the data flow direction.
  • Support for branching (AddBranch) and tool calls.
  • stream (computing) interchangeable Invoke Get streaming output.

4. Configuring streaming

Eino provides four streaming paradigms: Invoke, Stream, Collect, and Transform.

• Streaming Output Example

  stream, _ := runnable.Stream(ctx, map[string]any{"query": "Hello, world!"})
  for chunk := range stream {
  if chunk.Err ! = nil {
  panic(chunk.Err)
  }
  print(chunk.Value.Content)
  }
  

• Functional Analysis
  • Eino automatically handles splicing (e.g., to non-stream nodes) and splitting (e.g., copying when branching) of streams.
  • Downstream nodes do not need to care about the upstream flow state and the framework handles it transparently.

5. Adding a callback mechanism

Callbacks are used for logging, tracing, and other extensions.

• code example

  handler := eino.NewHandlerBuilder().
  OnStartFn(func(ctx context.Context, info *eino.RunInfo, input eino.CallbackInput) context.Context {
  println("Starting:", info.NodeID)
  return ctx
  }).Build()
  result, _ := runnable.Invoke(ctx, map[string]any{"query": "test"}, eino.WithCallbacks(handler))
  

• Operating Points
  • be in favor of OnStart,OnEnd Five types of cuts such as.
  • transferring entity WithCallbacks Specifies that the node or global takes effect.

Featured Function Operation

1. ReAct Agent

• Functional Description
  The ReAct Agent combines ChatModel and tool calls to realize autonomous decision loops.
• procedure
  • consultation react.goThe
  • Run the example: configure the model and tools, call the Invoke Implementation.

2. Visual debugging

• Functional Description
  pass (a bill or inspection etc) Eino Devops Provides UI debugging.
• procedure
  • pull eino-ext Warehouse, run devops module, view the document configuration.