This article describes how to use the Ollama API . This documentation is intended to help developers get up to speed quickly and take full advantage of Ollama's capabilities. You can use it in a Node.js environment or import the module directly in your browser. By studying this document, you can easily integrate Ollama into your projects.
Install Ollama
npm i ollama
Usage
import ollama from 'ollama'
const response = await ollama.chat({
model: 'ollama3.1', message: [{ role: 'user', content: 'Why is the sky blue?
messages: [{ role: 'user', content: 'Why is the sky blue?' }]
})
console.log(response.message.content)
Browser Use
To use this library without Node.js, import the browser module.
import ollama from 'ollama/browser'
Streaming Response
This can be done by setting the stream: true
Enable response streaming so that a function call returns a AsyncGenerator
, where each part is an object in the stream.
import ollama from 'ollama'
const message = { role: 'user', content: 'Why is the sky blue?' }
const response = await ollama.chat({ model: 'ollama3.1', messages: [message], stream: true })
for await (const part of response) {
process.stdout.write(part.message.content)
}
Structured Output
Using the Ollama JavaScript library, the architecture as a JSON
object is passed to the format
parameter, you can optionally use the object
format, or you can use Zod (recommended) to pass the zodToJsonSchema()
Method Serialization Architecture.
import ollama from 'ollama';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';
const Country = z.object({
name: z.string(), capital: z.string(), {
languages: z.array(z.string()),
}); const response = await ollama.
const response = await ollama.chat({
messages: [{ role: 'user', content: 'Tell me about Canada.' }], format: zodToJson(z.string()), }
format: zodToJsonSchema(Country), }); }
}).
const country = Country.parse(JSON.parse(response.message.content)); console.log(country.log(response.message);)
console.log(country);
Creating Models
import ollama from 'ollama'
const modelfile = `
FROM llama3.1
SYSTEM "You are Mario from Super Mario Bros."
`
await ollama.create({ model: 'example', modelfile: modelfile })
API
The Ollama JavaScript library API is designed around the Ollama REST API. If you want to learn more about the underlying implementation and full API endpoint information, we recommend referring to the Ollama API User's Guide
chats
ollama.chat(request)
request
: A request object containing chat parameters.
model
The name of the model to be used for the chat.
messages
: An array of message objects representing the chat history.
role
: The role of the message sender ('user', 'system' or 'assistant').
content
: The content of the message.
images
: (Optional) The image to include in the message, either a Uint8Array or base64 encoded string.
format
: (Optional) Set the expected format of the response (
json
).stream
: (Optional) If true, returns
AsyncGenerator
Thekeep_alive
: (Optional) Holds the length of time the model is loaded.
tools
: (Optional) A list of tools that may be called by the model.
options
: (Optional) Configure runtime options.
- Returns.
generating
ollama.generate(request)
request
: The request object containing the generated parameters.
model
The name of the model to be used for the chat.
prompt
: Sends a prompt to the model.
suffix
: (Optional) The suffix is the text that follows the inserted text.
system
: (Optional) Override modeling system prompts.
template
: (Optional) Override model templates.
raw
: (Optional) Bypasses the prompt template and passes the prompt directly to the model.
images
: (Optional) The image to include, either a Uint8Array or base64 encoded string.
format
: (Optional) Set the expected format of the response (
json
).stream
: (Optional) If true, returns
AsyncGenerator
Thekeep_alive
: (Optional) Holds the length of time the model is loaded.
options
: (Optional) Configure runtime options.
- Returns.
pull model
ollama.pull(request)
request
: The request object containing the pull parameters.
model
The name of the model to pull.
insecure
: (Optional) Pulls from servers that cannot authenticate.
stream
: (Optional) If true, returns
AsyncGenerator
The
- Returns.
push model
ollama.push(request)
request
: A request object containing push parameters.
model
The name of the model to push.
insecure
: (Optional) Push to a server that cannot authenticate the identity.
stream
: (Optional) If true, returns
AsyncGenerator
The
- Returns.
Creating Models
ollama.create(request)
request
: Contains the request object for which the parameters were created.
model
The name of the model to be created.
path
: (Optional) Path to the model file to be created.
modelfile
: (Optional) The contents of the model file to be created.
stream
: (Optional) If true, returns
AsyncGenerator
The
- Returns.
Delete Model
ollama.delete(request)
request
: The request object containing the deletion parameter.
model
The name of the model to be deleted.
- Returns.
Replication models
ollama.copy(request)
request
: The request object containing the replication parameters.
source
The name of the model to copy from.
destination
The name of the model to copy to.
- Returns.
List of Local Models
ollama.list()
- Returns.
Displaying model information
ollama.show(request)
request
: A request object containing the display parameters.
model
The name of the model to be displayed.
system
: (Optional) Overrides the model system prompt return value.
template
: (Optional) Overrides the model template return value.
options
: (Optional) Configure runtime options.
- Returns.
Generate Embedding
ollama.embed(request)
request
: A request object containing embedded parameters.
model
The name of the model used to generate the embedding.
input
|
: Input used to generate the embedding.truncate
: (Optional) Truncate the input to fit the maximum context length supported by the model.
keep_alive
: (Optional) Holds the length of time the model is loaded.
options
: (Optional) Configure runtime options.
- Returns.
step
ollama.ps()
- Returns.
Customized Clients
Custom clients can be created using the following fields:
host
: (Optional) Ollama host address. Default.
"http://127.0.0.1:11434"
Thefetch
: (Optional) The fetch library used to make requests to the Ollama host.
import { Ollama } from 'ollama'
const ollama = new Ollama({ host: 'http://127.0.0.1:11434' })
const response = await ollama.chat({
model: 'ollama3.1', { messages: [{ role: 'user', 'user', 'user', 'user'] }
messages: [{ role: 'user', content: 'Why is the sky blue?' }]
})
construct (sth abstract)
To build the project file, run:
npm run build
Refer to the documentation:ollama-js