Using the Ollama API in C++

This article describes how to use the Ollama API . This document is designed to help C++ developers get up to speed quickly and take full advantage of Ollama's capabilities. By studying this document, you can easily integrate Ollama into your projects.

Note that Ollama's community and documentation may be more focused on integrating Python with other high-level languages. When using big models in C++ projects, it is common to use the Llama.cpp (llama.cpp tutorial) Llama.cpp is an open source project developed by Georgi Gerganov that provides an efficient C/C++ implementation of Meta's LLaMa architecture, focusing on the reasoning process for large language models (LLMs).

I. Environmental preparedness

The development environment for C++ projects is not described here, but only how to integrate Ollama. make sure that you have Ollama installed and running, and that you have downloaded the required models (see Chapter 2).

II. Calling the Ollama API

Calling the Ollama API directly is cumbersome, so here are two ideas.

• Use a wrapped C++ class library.
• Call Ollama's model via Python.

1. Use of packaged C++ libraries

You can find packaged C++ libraries on Github, which can be used just by introducing header files to avoid duplicating the wheel. Here's an example of a ollama-hpp As an example:

#include "ollama.hpp"
std::cout << ollama::generate("ollama3.1:latest", "how to use ollama in a cpp project?") << std::endl;

Currently, there is only one C++ class library encapsulated on Github, and there are some bugs that need to be fixed manually during the testing process, so it is not recommended to use this method.

2. Calling Ollama's model via Python

Integration of Ollama in a C++ project allows you to call Ollama's models from Python and pass the results to a C++ program:

1. Writing Python code to call the Ollama API: Write code in Python to call the desired model to generate text or have a conversation through the API interface provided by Ollama. Below is a simple Python call example:

   import requests
   def call_ollama_api(prompt, model_name="llama3.1:latest"):: url = "".
   url = "http://localhost:11434/api/generate"
   headers = {"Content-Type": "application/json"}
   payload = {
   "model": model_name,
   "prompt": prompt, "stream": False
   "stream": False
   }
   response = requests.post(url, json=payload, headers=headers)
   return response.json()
   # Example call
   response = call_ollama_api("Is Black Myth fun?")
   print(response)
   

2. Calling Python Scripts from C++: In a C++ program, you can use system calls to execute Python scripts and pass the required text or questions as arguments to the Python script. For example, a Python script can be called in C++ in the following way:

   #include 
   int main() {
   system("python call_ollama.py 'Is Black Myth fun?'") ;
   return 0;
   }
   

reference document

• ollama-hpp
• llama.cpp Tutorial