本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

# 建立使用人力工作者之模型評估任務的自訂提示資料集
<a name="model-evaluation-prompt-datasets-custom-human"></a>

若要建立使用人力工作者的模型評估任務，您必須指定自訂提示資料集。然後，這些提示會在推論期間與您選取要評估的模型搭配使用。

如果您想要使用已產生的回應來評估非 Amazon Bedrock 模型，請在提示資料集中包含它們，如 [使用您自己的推論回應資料執行評估任務](#model-evaluation-prompt-datasets-custom-human-byoir) 中所述。當您提供自己的推論回應資料時，Amazon Bedrock 會略過模型調用步驟，並使用您提供的資料執行評估任務。

自訂提示資料集必須存放在 Amazon S3 中，並使用 JSON 行格式及使用 `.jsonl` 檔案副檔名。每行都必須是有效的 JSON 物件。每個自動評估任務在您的資料集中最多可有 1000 個提示。

對於使用主控台建立的任務，您必須更新 S3 儲存貯體上的跨來源資源分享 (CORS) 組態。若要進一步了解必要的 CORS 許可，請參閱 [S3 儲存貯體上所需的跨來源資源分享 (CORS) 許可](model-evaluation-security-cors.md)。

## 執行 Amazon Bedrock 為您調用模型的評估任務
<a name="model-evaluation-prompt-datasets-custom-human-invoke"></a>

若要執行 Amazon Bedrock 為您調用模型的評估任務，請提供包含下列鍵值對的提示資料集：
+ `prompt` – 您想要讓模型回應的提示。
+ `referenceResponse` – (選用) 您的工作者可以在評估期間參考的基本事實回應。
+ `category`– (選用) 在模型評估報告卡中檢閱結果時可用來篩選結果的索引鍵。

在工作者 UI 中，您的人力工作者可看見您為 `prompt` 和 `referenceResponse` 指定的內容。

以下是包含 6 個輸入並使用 JSON 行格式的自訂資料集範例。

```
{"prompt":"Provide the prompt you want the model to use during inference","category":"(Optional) Specify an optional category","referenceResponse":"(Optional) Specify a ground truth response."}
{"prompt":"Provide the prompt you want the model to use during inference","category":"(Optional) Specify an optional category","referenceResponse":"(Optional) Specify a ground truth response."}
{"prompt":"Provide the prompt you want the model to use during inference","category":"(Optional) Specify an optional category","referenceResponse":"(Optional) Specify a ground truth response."}
{"prompt":"Provide the prompt you want the model to use during inference","category":"(Optional) Specify an optional category","referenceResponse":"(Optional) Specify a ground truth response."}
{"prompt":"Provide the prompt you want the model to use during inference","category":"(Optional) Specify an optional category","referenceResponse":"(Optional) Specify a ground truth response."}
{"prompt":"Provide the prompt you want the model to use during inference","category":"(Optional) Specify an optional category","referenceResponse":"(Optional) Specify a ground truth response."}
```

下列範例是為了加強清晰度而擴充的單一項目。在實際提示資料集中，每行都必須是有效的 JSON 物件。

```
{
  "prompt": "What is high intensity interval training?",
  "category": "Fitness",
  "referenceResponse": "High-Intensity Interval Training (HIIT) is a cardiovascular exercise approach that involves short, intense bursts of exercise followed by brief recovery or rest periods."
}
```

## 使用您自己的推論回應資料執行評估任務
<a name="model-evaluation-prompt-datasets-custom-human-byoir"></a>

若要使用您已產生的回應執行評估任務，請提供包含下列鍵值對的提示資料集：
+ `prompt` – 您的模型用來產生回應的提示。
+ `referenceResponse` – (選用) 您的工作者可以在評估期間參考的基本事實回應。
+ `category`– (選用) 在模型評估報告卡中檢閱結果時可用來篩選結果的索引鍵。
+ `modelResponses` – 您要評估之自己推論的回應。您可以在 `modelResponses` 清單中提供一或兩個具有下列屬性的項目。
  + `response` – 包含模型推論回應的字串。
  + `modelIdentifier` – 識別產生回應之模型的字串。

提示資料集中的每一行都必須包含相同數量的回應 (一到兩個)。此外，您必須在每一行中指定相同的模型 ID 或識別符，且單一資料集中的 `modelIdentifier` 不能使用超過 2 個唯一值。

以下是包含 6 個輸入並使用 JSON 行格式的自訂範例資料集。

```
{"prompt":"The prompt you used to generate the model responses","referenceResponse":"(Optional) a ground truth response","category":"(Optional) a category for the prompt","modelResponses":[{"response":"The response your first model generated","modelIdentifier":"A string identifying your first model"},{"response":"The response your second model generated","modelIdentifier":"A string identifying your second model"}]}
{"prompt":"The prompt you used to generate the model responses","referenceResponse":"(Optional) a ground truth response","category":"(Optional) a category for the prompt","modelResponses":[{"response":"The response your first model generated","modelIdentifier":"A string identifying your first model"},{"response":"The response your second model generated","modelIdentifier":"A string identifying your second model"}]}
{"prompt":"The prompt you used to generate the model responses","referenceResponse":"(Optional) a ground truth response","category":"(Optional) a category for the prompt","modelResponses":[{"response":"The response your first model generated","modelIdentifier":"A string identifying your first model"},{"response":"The response your second model generated","modelIdentifier":"A string identifying your second model"}]}
{"prompt":"The prompt you used to generate the model responses","referenceResponse":"(Optional) a ground truth response","category":"(Optional) a category for the prompt","modelResponses":[{"response":"The response your first model generated","modelIdentifier":"A string identifying your first model"},{"response":"The response your second model generated","modelIdentifier":"A string identifying your second model"}]}
{"prompt":"The prompt you used to generate the model responses","referenceResponse":"(Optional) a ground truth response","category":"(Optional) a category for the prompt","modelResponses":[{"response":"The response your first model generated","modelIdentifier":"A string identifying your first model"},{"response":"The response your second model generated","modelIdentifier":"A string identifying your second model"}]}
{"prompt":"The prompt you used to generate the model responses","referenceResponse":"(Optional) a ground truth response","category":"(Optional) a category for the prompt","modelResponses":[{"response":"The response your first model generated","modelIdentifier":"A string identifying your first model"},{"response":"The response your second model generated","modelIdentifier":"A string identifying your second model"}]}
```

下列範例顯示提示資料集中為了加強清晰度而擴充的單一項目。

```
{
    "prompt": "What is high intensity interval training?",
    "referenceResponse": "High-Intensity Interval Training (HIIT) is a cardiovascular exercise approach that involves short, intense bursts of exercise followed by brief recovery or rest periods.",
    "category": "Fitness",
     "modelResponses": [
        {
            "response": "High intensity interval training (HIIT) is a workout strategy that alternates between short bursts of intense, maximum-effort exercise and brief recovery periods, designed to maximize calorie burn and improve cardiovascular fitness.",
            "modelIdentifier": "Model1"
        },
        {
            "response": "High-intensity interval training (HIIT) is a cardiovascular exercise strategy that alternates short bursts of intense, anaerobic exercise with less intense recovery periods, designed to maximize calorie burn, improve fitness, and boost metabolic rate.",
            "modelIdentifier": "Model2"
        }
    ]
}
```