GLM-4V Series

The GLM-4V series contains 3 models for different application scenarios.

• GLM-4V-Plus: With excellent multimodal comprehension capability, it can process up to 5 images simultaneously and supports video content comprehension, which is suitable for complex multimedia analysis scenarios.
• GLM-4V: Focusing on image content understanding, it is capable of parsing up to 5 images simultaneously, and is suitable for occasions where deep image understanding is required.
• GLM-4V-Flash (Free): Focuses on efficient single-image understanding for scenarios with fast image parsing, such as real-time image analysis or batch image processing.

synchronous call

interface request

Request Parameters

Messages format

Acceptable message types for the model include User message, Assistant message, and the format varies from one message type to another. The details are as follows:

User message

Assistant message

response parameter

Example of a request

Upload Video URL

# video comprehension example, upload video URLs
from zhipuai import ZhipuAI
client = ZhipuAI(api_key="YOUR API KEY") # Fill in your own APIKey
response = client.chat.completions.create(
model="glm-4v-plus", # Fill in the name of the model to be called.
messages=[
{
"role": "user", "content": [ {
"content": [
{
"type": "video_url", "content": [ {
"video_url": {
"url" : "https://sfile.chatglm.cn/testpath/video/xxxxx.mp4"
}
},
{
"type": "text".
"text": "Please describe this video carefully"
}
]
}
]
)
print(response.choices[0].message)

Upload video Base64

import base64
from zhipuai import ZhipuAI
video_path = "/Users/YourCompluter/xxxx.mp4"
with open(video_path, 'rb') as video_file.
video_base = base64.b64encode(video_file.read()).decode('utf-8')
client = ZhipuAI(api_key="YOUR API KEY") # complete your own APIKey
response = client.chat.completions.create(
model="glm-4v-plus", # Fill in the name of the model to be called.
messages=[
{
"role": "user", "content": [ {
"content": [
{
"type": "video_url", "content": [ {
"video_url": {
"url" : video_base
}
},
{
"type": "text", "text": "Please describe this video carefully", {
"text": "Please describe this video carefully"
}
]
}
]
)
print(response.choices[0].message)

Upload Image URL

from zhipuai import ZhipuAI
client = ZhipuAI(api_key="") # Fill in your own APIKey
response = client.chat.completions.create(
model="glm-4v", # Fill in the name of the model you want to call.
messages=[
{
"role": "user", "content": [ {
"content": [
{
"type": "text", "text": "What's in the picture", "text": "What's in the picture
"text": "What's in the picture"
},
{
"type": "image_url", {
"image_url": {
"url" : "https://img1.baidu.com/it/u=1369931113,3388870256&fm=253&app=138&size=w931&n=0&f=JPEG&fmt=auto?sec=1703696400&t= f3028c7a1dca43a080aeb8239f09cc2f"
}
}
]
}
]
)
print(response.choices[0].message)

Upload image Base64

import base64
from zhipuai import ZhipuAI
img_path = "/Users/YourCompluter/xxxx.jpeg"
with open(img_path, 'rb') as img_file: img_base = base64.
img_base = base64.b64encode(img_file.read()).decode('utf-8')
client = ZhipuAI(api_key="YOUR API KEY") # Fill in your own APIKey
response = client.chat.completions.create(
model="glm-4v-plus", # Fill in the name of the model to be called.
messages=[
{
"role": "user", "content": [ {
"content": [
{
"type": "image_url",
"image_url": {
"url": img_base
}
},
{
"type": "text",
"text": "Please describe this image"
}
]
}
]
)
print(response.choices[0].message)

Multiple rounds of picture dialog

from zhipuai import ZhipuAI
client = ZhipuAI(api_key="YOUR API KEY") # Fill in your own APIKey
response = client.chat.completions.create(
model="glm-4v-plus", # Fill in the name of the model to be called.
messages=[
{
"content": [
{
"image_url": {
"url": "https://cdn.bigmodel.cn/xxxxxx.png"
},
"type": "image_url"
},
{
"text": "What's in the picture",
"type": "text"
}
], { "role": "user".
"role": "user"
},
{
"content": [
{
"text": "This is a painting depicting a natural landscape, showing a peaceful lake with water in varying shades of blue-green hues. A blooming cherry blossom tree grows near the lake, its pinkish-white blossoms looking vibrant in the sunlight. Under the tree there is a man dressed in blue sitting on a boat, seemingly enjoying the good time. In the \n\n background are rolling hills covered in green vegetation, while the tops of the hills are surrounded by clouds, adding a sense of mystery. A small pavilion can also be seen in the distance, which sits on a rocky outcrop near the lake, harmonizing with the surrounding natural environment. \n\nThe whole picture gives a feeling of tranquility and harmony, as if one is in a paradise." ,
"type": "text"
}
],
"role": "assistant"
}, { "role": "assistant".
{
"content": [
{
"image_url": {
"url": "https://cdn.bigmodel.cn/xxxxxxx.jpeg"
},
"type": "image_url"
},
{
"text": "How is this image different from the above image?", {
"type": "text"
}
], { "role": "user".
"role": "user"
}, { "role": "user".
{
"content": [
{
"text": "The first picture and the second picture are very different in terms of picture content, composition and color, forming a sharp contrast. \n\n1.The first picture is a landscape painting, mainly depicting natural scenery such as cherry blossom trees, lakes, mountains, etc., which is rich in color and full of spring, while the second picture is a landscape photo depicting the Great Wall in winter, with the Great Wall meandering through the snow-covered mountains, and the overall color tone is cold, giving a feeling of coldness and serenity. \n\. Compositionally, the first picture adopts a symmetrical composition, with the cherry blossom tree as the center and the scenery on both sides echoing each other to form a balanced and harmonious picture effect, while the second picture adopts a diagonal composition, with the Great Wall serving as a visual guide line to lead the viewer's eyes across the whole picture, increasing the depth and dynamic sense of the picture. \n\nOverall, these two pictures show a clear contrast in picture content, color, composition and atmosphere, one is a spring landscape full of vitality and color, the other is a serene and mysterious winter landscape photo of the Great Wall." ,
"type": "text"
}
],.
"role": "assistant"
}, { "role": "assistant".
{
"content": [
{
"image_url": {
"url": "https://cdn.bigmodel.cn/xxxxxxx.jpeg"
},
"type": "image_url"
},
{
"text": "What is the difference between this image and the previous one?", {
"type": "text"
}
], { "role": "user".
"role": "user"
}
]
)
print(response.choices[0].message)

Response Example

{
"created": 1703487403,.
"id": "8239375684858666781",
"model": "glm-4v-plus",
"request_id": "8239375684858666781",
"choices": [
{
"finish_reason": "stop",

"message": {
"content": "The picture has a blue sea and blue sky with white clouds in the sky. In the bottom right corner of the picture there is an island or rock with dark green trees growing on it." ,
"role": "assistant"
}
}
],.
"usage": {

"prompt_tokens": 1037,
"total_tokens": 1074
}
}

streaming output

response parameter

Example of a request

from zhipuai import ZhipuAI
client = ZhipuAI(api_key="") # Please fill in your own APIKey
response = client.chat.completions.create(
model="glm-4v-plus", # Fill in the name of the model you want to call.
messages=[
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url" : "sfile.chatglm.co.uk/testpath/xxxx.jpg"
}
},
{
"type": "text", {
"text": "What's in the picture"
}
]
}
],
stream=True, )
)
for chunk in response.
print(chunk.choices[0].delta)

Response Example

data: {"id": "8305986882425703351", "created":1705476637, "model": "glm-4v-plus", "choices":[{"index":0, "delta":{"role": "assistant", " content": "next"}}]}
data: {"id": "8305986882425703351", "created":1705476637, "model": "glm-4v-plus", "choices":[{"index":0, "delta":{"role": "assistant"," content": "corner"}}]}
data: {"id": "8305986882425703351", "created":1705476637, "model": "glm-4v-plus", "choices":[{"index":0, "delta":{"role": "assistant"," content": "There is one"}}]}
... ...
data: {"id": "8305986882425703351", "created":1705476637, "model": "glm-4v-plus", "choices":[{"index":0, "delta":{"role": "assistant", " content": "trees"}}]}
data: {"id": "8305986882425703351", "created":1705476637, "model": "glm-4v-plus", "choices":[{"index":0, "delta":{"role": "assistant"," content":"."}}]}
data: {"id": "8305986882425703351", "created":1705476637, "model": "glm-4v-plus", "choices":[{"index":0, "finish_reason": "stop", "delta":{ "role": "assistant", "content":""}}], "usage":{"prompt_tokens":1037, "completion_tokens":37, "total_tokens":1074}}