/AWS1/IF_BDO=>CREATEEVALUATOR()

About CreateEvaluator

Creates a custom evaluator for agent quality assessment. Custom evaluators can use either LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings, or code-based configurations with customer-managed Lambda functions to evaluate agent performance at tool call, trace, or session levels.

Method Signature

METHODS /AWS1/IF_BDO~CREATEEVALUATOR
  IMPORTING
    !IV_CLIENTTOKEN TYPE /AWS1/BDOCLIENTTOKEN OPTIONAL
    !IV_EVALUATORNAME TYPE /AWS1/BDOCUSTOMEVALUATORNAME OPTIONAL
    !IV_DESCRIPTION TYPE /AWS1/BDOEVALUATORDESCRIPTION OPTIONAL
    !IO_EVALUATORCONFIG TYPE REF TO /AWS1/CL_BDOEVALUATORCONFIG OPTIONAL
    !IV_LEVEL TYPE /AWS1/BDOEVALUATORLEVEL OPTIONAL
    !IT_TAGS TYPE /AWS1/CL_BDOTAGSMAP_W=>TT_TAGSMAP OPTIONAL
  RETURNING
    VALUE(OO_OUTPUT) TYPE REF TO /aws1/cl_bdocreateevaluatorrsp
  RAISING
    /AWS1/CX_BDOACCESSDENIEDEX
    /AWS1/CX_BDOCONFLICTEXCEPTION
    /AWS1/CX_BDOINTERNALSERVEREX
    /AWS1/CX_BDOSERVICEQUOTAEXCDEX
    /AWS1/CX_BDOTHROTTLINGEX
    /AWS1/CX_BDOVALIDATIONEX
    /AWS1/CX_BDOCLIENTEXC
    /AWS1/CX_BDOSERVEREXC
    /AWS1/CX_RT_TECHNICAL_GENERIC
    /AWS1/CX_RT_SERVICE_GENERIC.

IMPORTING

Required arguments:

iv_evaluatorname TYPE /AWS1/BDOCUSTOMEVALUATORNAME /AWS1/BDOCUSTOMEVALUATORNAME

The name of the evaluator. Must be unique within your account.

io_evaluatorconfig TYPE REF TO /AWS1/CL_BDOEVALUATORCONFIG /AWS1/CL_BDOEVALUATORCONFIG

The configuration for the evaluator. Specify either LLM-as-a-Judge settings with instructions, rating scale, and model configuration, or code-based settings with a customer-managed Lambda function.

iv_level TYPE /AWS1/BDOEVALUATORLEVEL /AWS1/BDOEVALUATORLEVEL

The evaluation level that determines the scope of evaluation. Valid values are TOOL_CALL for individual tool invocations, TRACE for single request-response interactions, or SESSION for entire conversation sessions.

Optional arguments:

iv_clienttoken TYPE /AWS1/BDOCLIENTTOKEN /AWS1/BDOCLIENTTOKEN

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.

iv_description TYPE /AWS1/BDOEVALUATORDESCRIPTION /AWS1/BDOEVALUATORDESCRIPTION

The description of the evaluator that explains its purpose and evaluation criteria.

it_tags TYPE /AWS1/CL_BDOTAGSMAP_W=>TT_TAGSMAP TT_TAGSMAP

A map of tag keys and values to assign to an AgentCore Evaluator. Tags enable you to categorize your resources in different ways, for example, by purpose, owner, or environment.

RETURNING

oo_output TYPE REF TO /aws1/cl_bdocreateevaluatorrsp /AWS1/CL_BDOCREATEEVALUATORRSP

Examples

Syntax Example

This is an example of the syntax for calling the method. It includes every possible argument and initializes every possible value. The data provided is not necessarily semantically accurate (for example the value "string" may be provided for something that is intended to be an instance ID, or in some cases two arguments may be mutually exclusive). The syntax shows the ABAP syntax for creating the various data structures.

DATA(lo_result) = lo_client->createevaluator(
  io_evaluatorconfig = new /aws1/cl_bdoevaluatorconfig(
    io_codebased = new /aws1/cl_bdocodebasedevaltrcfg(
      io_lambdaconfig = new /aws1/cl_bdolambdaevaltrconfig(
        iv_lambdaarn = |string|
        iv_lambdatimeoutinseconds = 123
      )
    )
    io_llmasajudge = new /aws1/cl_bdollmasajudgeevalu00(
      io_modelconfig = new /aws1/cl_bdoevaluatormodelcfg(
        io_bdrkevaluatormodelconfig = new /aws1/cl_bdobdrevaluatormdel00(
          io_addlmodelrequestfields = /AWS1/CL_RT_DOCUMENT=>FROM_JSON_STR( |\{"foo":"this is a JSON object..."\}| )
          io_inferenceconfig = new /aws1/cl_bdoinferenceconf(
            it_stopsequences = VALUE /aws1/cl_bdononemptystrlist_w=>tt_nonemptystringlist(
              ( new /aws1/cl_bdononemptystrlist_w( |string| ) )
            )
            iv_maxtokens = 123
            iv_temperature = '0.1'
            iv_topp = '0.1'
          )
          iv_modelid = |string|
        )
      )
      io_ratingscale = new /aws1/cl_bdoratingscale(
        it_categorical = VALUE /aws1/cl_bdocatgclscaledefn=>tt_categoricalscaledefinitions(
          (
            new /aws1/cl_bdocatgclscaledefn(
              iv_definition = |string|
              iv_label = |string|
            )
          )
        )
        it_numerical = VALUE /aws1/cl_bdonumericalscaledefn=>tt_numericalscaledefinitions(
          (
            new /aws1/cl_bdonumericalscaledefn(
              iv_definition = |string|
              iv_label = |string|
              iv_value = '0.1'
            )
          )
        )
      )
      iv_instructions = |string|
    )
  )
  it_tags = VALUE /aws1/cl_bdotagsmap_w=>tt_tagsmap(
    (
      VALUE /aws1/cl_bdotagsmap_w=>ts_tagsmap_maprow(
        value = new /aws1/cl_bdotagsmap_w( |string| )
        key = |string|
      )
    )
  )
  iv_clienttoken = |string|
  iv_description = |string|
  iv_evaluatorname = |string|
  iv_level = |string|
).

This is an example of reading all possible response values

lo_result = lo_result.
IF lo_result IS NOT INITIAL.
  lv_customevaluatorarn = lo_result->get_evaluatorarn( ).
  lv_evaluatorid = lo_result->get_evaluatorid( ).
  lv_timestamp = lo_result->get_createdat( ).
  lv_evaluatorstatus = lo_result->get_status( ).
ENDIF.