Ollama in LangChain - JavaScript Integration

summary

This document describes how to use the Ollama Integrate with LangChain to create powerful AI applications.Ollama is an open source deployment tool for large language models, and LangChain is a framework for building language model-based applications. By combining the two, we can quickly deploy and use advanced AI models in a local environment.

Note: This document contains core code snippets and detailed explanations. The complete code can be found in notebook/c5/ollama_langchain_javascript.

1. Environmental settings

Configuring the Node.js environment

First, make sure you have Node.js installed on your system; you can download and install the latest version from the Node.js website.

Create a project and install dependencies

1. Switch to the run directory to run it:

cd notebook/C5/ollama_langchain_javascript
npm init -y

2. Install the necessary dependencies:

npm install @langchain/ollama @langchain/core @langchain/community zod

3. exist package.json file by adding the "type": "module" to enable ES module support:

{
"type": "module",
// ... 其他配置
}

2. Download the required model and initialize Ollama

Download llama3.1 model

1. Go to the official website https://ollama.com/download to download and install Ollama on available supported platforms.
2. Check out https://ollama.ai/library for all the available models.
3. pass (a bill or inspection etc) ollama pull <name-of-model> command to get the available LLM models (for example:ollama pull llama3.1).

The command line is executed as shown in the figure:

Model storage location:

• Mac. ~/.ollama/models/
• Linux (or WSL). /usr/share/ollama/.ollama/models
• Windows. C:\Users\Administrator\.ollama\models

Once the download is complete, you need to make sure that the Ollama service is started:

ollama ps

3. Examples of basic use

Simple conversations with ChatOllama

runnable base_chat.js file, the specific code is as follows:

import { Ollama } from "@langchain/community/llms/ollama";
const ollama = new Ollama({
baseUrl: "http://localhost:11434",  // 确保Ollama服务已经启动
model: "llama3.1", // 替换为实际使用的模型
});
const stream = await ollama.stream(
`你比GPT4厉害吗?`
);
const chunks = [];
for await (const chunk of stream) {
chunks.push(chunk);
}
console.log(chunks.join(""));

Run the code:

node base_chat.js

This code does several things:

1. Import the Ollama class and initialize it, specifying the model and base URL.
2. utilization stream method sends a question to the model, which allows us to receive the response block by block.
3. Use a for-await loop to collect all response blocks.
4. Combine all blocks and print the full response.

Multimodal model use

(of a computer) runbase_multimodal.jsfile, the specific code is as follows:

import { Ollama } from "@langchain/community/llms/ollama";
import * as fs from "node:fs/promises";
const imageData = await fs.readFile("../../../docs/images/C5-1-4.png"); // 可以替换为你想询问的图片
const model = new Ollama({
model: "llava",
baseUrl: "http://127.0.0.1:11434",
}).bind({
images: [imageData.toString("base64")],
});
const res = await model.invoke("图片里是什么动物呀？");
console.log({ res });

Run the code:

node base_multimodal.js

This code demonstrates how to process image and text inputs using a multimodal model such as llava:

1. Reads an image file and converts it to base64 encoding.
2. Initialize the Ollama model and use the bind method binds the image data to the model.
3. utilization invoke method sends a question about an image.
4. Print the response of the model.

Tool Call

(of a computer) run base_tool.js file with the following code:

import { tool } from "@langchain/core/tools";
import { ChatOllama } from "@langchain/ollama";
import { z } from "zod";
// 定义简单计算器工具
const simpleCalculatorTool = tool((args) => {
const { operation, x, y } = args;
switch (operation) {
case "add":
return x + y;
case "subtract":
return x - y;
case "multiply":
return x * y;
case "divide":
if (y !== 0) {
return x / y;
} else {
throw new Error("Cannot divide by zero");
}
default:
throw new Error("Invalid operation");
}
}, {
name: "simple_calculator",
description: "Perform simple arithmetic operations",
schema: z.object({
operation: z.enum(["add", "subtract", "multiply", "divide"]),
x: z.number(),
y: z.number(),
}),
});
// 定义模型
const llm = new ChatOllama({
model: "llama3.1",
temperature: 0,
});
// 将工具绑定到模型
const llmWithTools = llm.bindTools([simpleCalculatorTool]);
// 使用模型进行工具调用
const result = await llmWithTools.invoke(
"你知道一千万乘二是多少吗？请使用 'simple_calculator' 工具来计算。"
);
console.log(result);

Run the code:

node base_tool.js

This code shows how to define and use tools:

1. utilization tool function defines a simple calculator tool, including the operation logic and parameter schema.
2. Initialize the ChatOllama model.
3. utilization bindTools method binds the tool to the model.
4. utilization invoke method sends a problem that needs to be computed and the model automatically calls the appropriate tool.

4. Advanced usage

Customized prompt templates

Customized prompt templates not only improve the efficiency of content generation, but also ensure consistency and relevance of the output. With well-designed templates, we can fully utilize the capabilities of the AI model while maintaining control and guidance over the output content:

import { ChatOllama } from "@langchain/ollama";
import { ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate } from "@langchain/core/prompts";
// 初始化ChatOllama模型
const model = new ChatOllama({
model: "llama3.1",
temperature: 0.7,
});
const systemMessageContent = `
你是一位经验丰富的电商文案撰写专家。你的任务是根据给定的产品信息创作吸引人的商品描述。
请确保你的描述简洁、有力，并且突出产品的核心优势。
`;
const humanMessageTemplate = `
请为以下产品创作一段吸引人的商品描述：
产品类型: {product_type}
核心特性: {key_feature}
目标受众: {target_audience}
价格区间: {price_range}
品牌定位: {brand_positioning}
请提供以下三种不同风格的描述，每种大约50字：
1. 理性分析型
2. 情感诉求型
3. 故事化营销型
`;
const prompt = ChatPromptTemplate.fromMessages([
SystemMessagePromptTemplate.fromTemplate(systemMessageContent),
HumanMessagePromptTemplate.fromTemplate(humanMessageTemplate),
]);
const chain = prompt.pipe(model);
async function generateProductDescriptions(productInfo) {
const response = await chain.invoke(productInfo);
return response.content;
}
// 示例使用
const productInfo = {
product_type: "智能手表",
key_feature: "心率监测和睡眠分析",
target_audience: "注重健康的年轻专业人士",
price_range: "中高端",
brand_positioning: "科技与健康的完美结合"
};
generateProductDescriptions(productInfo)
.then((result) => console.log(result))
.catch((error) => console.error("Error:", error));

Run the code:

node advanced_prompt.js

This code shows how to create and use a custom prompt template:

1. Define system message and human message templates.
2. utilization ChatPromptTemplate.fromMessages Create a complete prompt template.
3. utilization pipe method connects the cue template to the model, creating a processing chain.
4. Define a function to generate a product description that uses a processing chain to process the input product information.

Custom prompt templates have a wide range of uses in practice, especially when you need to generate content in a specific format or style. Below are some practical application scenarios:

1. E-commerce Product Description Generation: As shown in this example, it can be used to automatically generate different styles of product descriptions to improve the attractiveness and conversion rate of product pages.
2. Customer Service Response Templates: You can create response templates for various scenarios, such as handling complaints, providing product information, etc., to ensure the consistency and professionalism of customer service responses.
3. News Report Generation: Templates can be designed to generate different types of news reports, such as breaking news, in-depth analysis, etc., to help journalists quickly draft their first drafts.
4. Personalized marketing emails: Generate personalized marketing email content based on customer data and marketing objectives to improve the effectiveness of email marketing.

Advanced JSON Output and Knowledge Graph Generation

In this example, we show how Ollama and LangChain can be used to generate structured JSON output, especially for creating knowledge graphs. This approach is closely related to Microsoft's open source project GraphRAG, especially in terms of automated knowledge extraction and ternary generation.

import { ChatOllama } from "@langchain/ollama";
import { PromptTemplate } from "@langchain/core/prompts";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
const systemTemplate = `
你是一位医疗领域的专家，擅长创建知识图谱。请将所有响应格式化为具有以下结构的JSON对象：
{
"节点": [
{"id": "string", "标签": "string", "类型": "string"}
],
"关系": [
{"源": "string", "目标": "string", "关系": "string"}
]
}
确保所有节点id都是唯一的，并且关系引用的是已存在的节点id。
`;
const humanTemplate = `
请为医疗主题"{topic}"创建一个知识图谱。包括以下相关概念: {concepts}。
提供至少5个节点和5个关系。请确保使用中文回答。
`;
const systemMessage = new SystemMessage(systemTemplate);
const humanPrompt = PromptTemplate.fromTemplate(humanTemplate);
const llmJsonMode = new ChatOllama({
baseUrl: "http://localhost:11434", // 默认值
model: "llama3.1",
format: "json",
});
async function generateMedicalKnowledgeGraph(topic, concepts) {
try {
const humanMessageContent = await humanPrompt.format({
topic: topic,
concepts: concepts.join("、"),
});
const humanMessage = new HumanMessage(humanMessageContent);
const messages = [systemMessage, humanMessage];
const result = await llmJsonMode.call(messages);
console.log(JSON.stringify(result, null, 2));
return result;
} catch (error) {
console.error("生成知识图谱时出错:", error);
}
}
// 使用示例
const topic = "糖尿病";
const concepts = ["胰岛素", "血糖", "并发症", "饮食管理", "运动疗法"];
generateMedicalKnowledgeGraph(topic, concepts);

Run the code:

node advanced_json.js

This code demonstrates how to generate structured JSON output using Ollama:

1. Defines a system template that specifies the desired JSON structure.
2. Create a human prompt template for generating requests for the Knowledge Graph.
3. Initialize the ChatOllama model, set the format: "json" to get the JSON output.
4. Define a function to generate a medical knowledge graph that combines system messages and human messages and calls the model.

We can get a glimpse of the many ways in which this can be utilized through specific outputs in the json format:

1. Automated ternary generation : In traditional approaches, the creation of knowledge graphs usually requires extensive manual annotation work. Experts need to carefully read documents, identify important concepts and their relationships, and then manually create triples (subject-relationship-object). This process is not only time-consuming but also error-prone, especially when dealing with a large number of documents. Using our approach, the Big Language Model can automatically generate relevant nodes and relations from given subjects and concepts. This greatly accelerates the process of knowledge graph construction.
2. data enhancement : This approach can be used not only for the direct generation of knowledge graphs, but also for data enhancement. For example:
   • Extending the existing training dataset: by having the model generate new relevant triples based on existing triples.
   • Creating diverse examples: generating knowledge graphs for different medical conditions adds diversity to the data.
   • Cross-language data generation: by adjusting the prompts, knowledge graphs in different languages can be generated to support multilingual applications.
3. Improving data quality : The Big Language Model can be leveraged to generate high-quality, coherent knowledge graphs that capitalize on its broad knowledge base. With well-designed prompts, we can ensure that the generated data meets specific quality standards and domain specifications.
4. Flexibility and scalability : This approach is very flexible and can be easily adapted to different areas and needs:
   • By modifying the system prompts, we can change the JSON structure of the output to accommodate different knowledge graph formats.
   • It is easy to extend this approach to other areas such as technology, finance, education, etc.

reach a verdict

With these examples, we show how to use Ollama and LangChain to build a variety of AI applications in a JavaScript environment, from simple dialog systems to complex knowledge graph generation. These tools and techniques provide a solid foundation for developing powerful AI applications.

The combination of Ollama and LangChain provides developers with great flexibility and possibilities. You can choose the right models and components according to your specific needs and build an AI system that suits your application scenario.

As the technology continues to evolve, we expect to see more innovative applications emerge. Hopefully, this guide will help you get started on your AI development journey and inspire your creativity to explore the endless possibilities of AI technology.