本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。

# Cohere Embed v4
<a name="model-parameters-embed-v4"></a>

Cohere Embed v4 是一款支持文本输入与图像输入的多模态嵌入模型。该模型可处理文本与图像交错的内容，适用于文档理解、视觉搜索及多模态检索应用场景。该模型支持包括 float、int8、uint8、binary 和 ubinary 格式在内的多种嵌入类型，可在 256 至 1536 之间配置输出维度。

Cohere Embed v4 的模型 ID 为 `cohere.embed-v4`。

**其他使用说明**  

+ **上下文长度：**每个文档最多 12.8 万个令牌；对于 RAG 来说，较小的区块通常可以提高检索率和成本。
+ **图像尺寸：**像素数超过 2458624 的图像会下采样至该尺寸；像素数不足 3136 的图像会进行上采样处理。
+ **交错输入：**对于页面类多模态内容，建议使用 inputs.content[]，确保文本上下文（例如，文件名、实体）与图像一起传递。

**Topics**
+ [请求和响应](#model-parameters-embed-v4-request-response)
+ [不同的 input\_type 的请求和响应](#api-inference-examples-cohere-embed-v4)
+ [代码示例](#code-examples-cohere-embed-v4)

## 请求和响应
<a name="model-parameters-embed-v4-request-response"></a>

------
#### [ Request ]

内容类型： application/json

```
{
  "input_type": "search_document | search_query | classification | clustering",
  "texts": ["..."],                      // optional; text-only
  "images": ["data:<mime>;base64,..."],  // optional; image-only
  "inputs": [
    { "content": [
        { "type": "text",      "text": "..." },
        { "type": "image_url", "image_url": {"url": "data:<mime>;base64,..."} }
      ]
    }
  ],                                     // optional; mixed (interleaved) text+image
  "embedding_types": ["float" | "int8" | "uint8" | "binary" | "ubinary"],
  "output_dimension": 256 | 512 | 1024 | 1536,
  "max_tokens": 128000,
  "truncate": "NONE | LEFT | RIGHT"
}
```

**参数**  

+ **input\_type**（必填）- 添加专用词元以区分各种使用案例。可用项：`search_document`、`search_query`、`classification`、`clustering`。对于 search/RAG，使用嵌入语料库`search_document`并使用查询。`search_query`
+ **texts**（可选）- 要嵌入的字符串数组。每次调用最多支持 96 个字符串。如果您使用 `texts`，请不要在同一调用中发送 `images`。
+ **images**（可选）– 要嵌入的 data-URI base64 图像数组。每次调用最多支持 96 张图像。请不要将 `texts` 和 `images` 一起发送。（将 `inputs` 用于交错型内容。）
+ **inp** uts（可选 mixed/fused ；modality）-一个列表，其中每个项目都有部分内容列表。每个部分的格式为 `{ "type": "text", "text": ... }` 或 `{ "type": "image_url", "image_url": {"url": "data:<mime>;base64,..."} }`。在此处发送类似页面的交错内容（例如，PDF 页面图片 \+）。 caption/metadata最多 96 个项目。
+ **embedding\_types**（可选）– 以下项中的一个或多个：`float`、`int8`、`uint8`、`binary`、`ubinary`。如果省略，则返回浮点型嵌入。
+ **output\_dimension**（可选）- 选择向量长度。可用项：`256`、`512`、`1024`、`1536`（如果未指定，则为默认值 `1536`）。
+ **max\_token** s（可选）-每个输入文档的截断预算。该模型支持每个文档多达 128,000 个令牌；对于 RAG，请酌情缩小区块数量。
+ **truncate**（可选）– 处理超长输入的方式：`LEFT` 表示从开头删除词元；`RIGHT` 表示从结尾删除词元；`NONE` 表示在输入超出限制时返回错误。

**限制&amp;大小**  

+ 每次请求的项目数：最多 96 张图像。原始图像文件类型必须采用 png、jpeg、webp 或 gif 格式，且大小不超过 5 MB。
+ 请求大小上限：总有效载荷约为 20 MB。
+ 最大输入词元数：最多 128000 个词元。图像文件将转换为词元，总词元数应少于 128000 个。
+ 图像：下采样前最大像素数为 2458624；对像素数少于 3136 的图像进行上采样处理。以 `data:<mime>;base64,....` 格式提供图像
+ 词元计数（按每个 `inputs` 项目）：来自图像输入的词元数 ≈（图像像素数 ÷ 784）x 4；来自交错型文本和图像输入的词元数 =（图像像素数 ÷ 784）x 4 \+（文本词元数）

**提示：**对于 PDF 文件，可将每页转换为图像，通过 `inputs` 发送，并在相邻的文本部分中附带页面元数据（例如，文件名、实体）。

------
#### [ Response ]

内容类型： application/json

如果您已请求单一嵌入类型（例如，仅 `float`）：

```
{
"id": "string",
"embeddings": [[ /* length = output_dimension */ ]],
"response_type": "embeddings_floats",
"texts": ["..."], // present if text was provided
"inputs": [ { "content": [ ... ] } ] // present if 'inputs' was used
}
```

如果您已请求多种嵌入类型（例如，`["float","int8"]`）：

```
{
  "id": "string",
  "embeddings": {
    "float": [[ ... ]],
    "int8":  [[ ... ]]
  },
  "response_type": "embeddings_by_type",
  "texts": ["..."],     // when text used
  "inputs": [ { "content": [ ... ] } ] // when 'inputs' used
}
```
+ 返回的向量数量与 `texts` 数组的长度或 `inputs` 项目数相匹配。
+ 每个向量的长度均等于 `output_dimension`（默认值为 `1536`）。

------

## 不同的 input\_type 的请求和响应
<a name="api-inference-examples-cohere-embed-v4"></a>

**A）带紧凑的 int8 向量的交错页面（图像 \+ 标题）**

**请求**  


```
{
  "input_type": "search_document",
  "inputs": [
    {
      "content": [
        { "type": "text", "text": "Quarterly ARR growth chart; outlier in Q3." },
        { "type": "image_url", "image_url": {"url": "data:image/png;base64,{{BASE64_PAGE_IMG}}"} }
      ]
    }
  ],
  "embedding_types": ["int8"],
  "output_dimension": 512,
  "truncate": "RIGHT",
  "max_tokens": 128000
}
```

**响应（已截断）**  


```
{
  "id": "836a33cc-61ec-4e65-afaf-c4628171a315",
  "embeddings": { "int8": [[ 7, -3, ... ]] },
  "response_type": "embeddings_by_type",
  "inputs": [
    { "content": [
      { "type": "text", "text": "Quarterly ARR growth chart; outlier in Q3." },
      { "type": "image_url", "image_url": {"url": "data:image/png;base64,{{...}}"} }
    ] }
  ]
}
```

**B) Text-only 语料库索引（默认浮点数，1536-dim）**

**请求**  


```
{
  "input_type": "search_document",
  "texts": [
    "RAG system design patterns for insurance claims",
    "Actuarial loss triangles and reserving primer"
  ]
}
```

**响应（示例）**  


```
{
  "response_type": "embeddings_floats",
  "embeddings": [
    [0.0135, -0.0272, ...],   // length 1536
    [0.0047,  0.0189, ...]
  ]
}
```

## 代码示例
<a name="code-examples-cohere-embed-v4"></a>

------
#### [ Text input ]

```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to generate embeddings using the Cohere Embed v4 model.
"""
import json
import logging
import boto3


from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def generate_text_embeddings(model_id, body, region_name):
    """
    Generate text embedding by using the Cohere Embed model.
    Args:
        model_id (str): The model ID to use.
        body (str) : The reqest body to use.
        region_name (str): The AWS region to invoke the model on
    Returns:
        dict: The response from the model.
    """

    logger.info("Generating text embeddings with the Cohere Embed model %s", model_id)

    accept = '*/*'
    content_type = 'application/json'

    bedrock = boto3.client(service_name='bedrock-runtime', region_name=region_name)

    response = bedrock.invoke_model(
        body=body,
        modelId=model_id,
        accept=accept,
        contentType=content_type
    )

    logger.info("Successfully generated embeddings with Cohere model %s", model_id)

    return response


def main():
    """
    Entrypoint for Cohere Embed example.
    """

    logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
    
    region_name = 'us-east-1'

    model_id = 'cohere.embed-v4:0'
    text1 = "hello world"
    text2 = "this is a test"
    input_type = "search_document"
    embedding_types = ["float"]

    try:
        body = json.dumps({
            "texts": [
                text1,
                text2],
            "input_type": input_type,
            "embedding_types": embedding_types
        })
        
        response = generate_text_embeddings(model_id=model_id, body=body, region_name=region_name)

        response_body = json.loads(response.get('body').read())

        print(f"ID: {response_body.get('id')}")
        print(f"Response type: {response_body.get('response_type')}")

        print("Embeddings")
        embeddings = response_body.get('embeddings')
        for i, embedding_type in enumerate(embeddings):
            print(f"\t{embedding_type} Embeddings:")
            print(f"\t{embeddings[embedding_type]}")

        print("Texts")
        for i, text in enumerate(response_body.get('texts')):
            print(f"\tText {i}: {text}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    else:
        print(
            f"Finished generating text embeddings with Cohere model {model_id}.")


if __name__ == "__main__":
    main()
```

------
#### [ Mixed modalities ]

```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to generate image embeddings using the Cohere Embed v4 model.
"""
import json
import logging
import boto3
import base64


from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

def get_base64_image_uri(image_file_path: str, image_mime_type: str):
    with open(image_file_path, "rb") as image_file:
        image_bytes = image_file.read()
        base64_image = base64.b64encode(image_bytes).decode("utf-8")
    return f"data:{image_mime_type};base64,{base64_image}"


def generate_embeddings(model_id, body, region_name):
    """
    Generate image embedding by using the Cohere Embed model.
    Args:
        model_id (str): The model ID to use.
        body (str) : The reqest body to use.
        region_name (str): The AWS region to invoke the model on
    Returns:
        dict: The response from the model.
    """

    logger.info("Generating image embeddings with the Cohere Embed model %s", model_id)

    accept = '*/*'
    content_type = 'application/json'

    bedrock = boto3.client(service_name='bedrock-runtime', region_name=region_name)

    response = bedrock.invoke_model(
        body=body,
        modelId=model_id,
        accept=accept,
        contentType=content_type
    )

    logger.info("Successfully generated embeddings with Cohere model %s", model_id)

    return response


def main():
    """
    Entrypoint for Cohere Embed example.
    """

    logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
    
    region_name = 'us-east-1'

    image_file_path = "image.jpg"
    image_mime_type = "image/jpg"
    text = "hello world"

    model_id = 'cohere.embed-v4:0'
    input_type = "search_document"
    image_base64_uri = get_base64_image_uri(image_file_path, image_mime_type)
    embedding_types = ["int8","float"]

    try:
        body = json.dumps({
            "inputs": [
                {
                  "content": [
                    { "type": "text", "text": text },
                    { "type": "image_url", "image_url": {"url": "data:image/png;base64,{{image_base64_uri}}"} }
                  ]
                }
              ],
            "input_type": input_type,
            "embedding_types": embedding_types
        })
        
        response = generate_embeddings(model_id=model_id, body=body, region_name=region_name)

        response_body = json.loads(response.get('body').read())

        print(f"ID: {response_body.get('id')}")
        print(f"Response type: {response_body.get('response_type')}")

        print("Embeddings")
        embeddings = response_body.get('embeddings')
        for i, embedding_type in enumerate(embeddings):
            print(f"\t{embedding_type} Embeddings:")
            print(f"\t{embeddings[embedding_type]}")

        print("inputs")
        for i, input in enumerate(response_body.get('inputs')):
            print(f"\tinput {i}: {input}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    else:
        print(
            f"Finished generating embeddings with Cohere model {model_id}.")


if __name__ == "__main__":
    main()
```

------