Package com.google.cloud.aiplatform.v1beta1

Interface CustomCodeExecutionSpecOrBuilder

All Superinterfaces:
com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder
All Known Implementing Classes:
CustomCodeExecutionSpec, CustomCodeExecutionSpec.Builder
@Generated public interface CustomCodeExecutionSpecOrBuilder extends com.google.protobuf.MessageOrBuilder

  • Method Summary

    Modifier and Type
    Method
    Description
    String
    getEvaluationFunction()
    Required.
    com.google.protobuf.ByteString
    getEvaluationFunctionBytes()
    Required.
    boolean
    hasEvaluationFunction()
    Required.

    Methods inherited from interface com.google.protobuf.MessageLiteOrBuilder

    isInitialized

    Methods inherited from interface com.google.protobuf.MessageOrBuilder

    findInitializationErrors, getAllFields, getDefaultInstanceForType, getDescriptorForType, getField, getInitializationErrorString, getOneofFieldDescriptor, getRepeatedField, getRepeatedFieldCount, getUnknownFields, hasField, hasOneof

  • Method Details

    • hasEvaluationFunction

      boolean hasEvaluationFunction()
       Required. Python function.
 Expected user to define the following function, e.g.:
 def evaluate(instance: dict[str, Any]) -> float:
 Please include this function signature in the code snippet.
 Instance is the evaluation instance, any fields populated in the instance
 are available to the function as instance[field_name].

 Example:
 Example input:
 ```
 instance= EvaluationInstance(
 response=EvaluationInstance.InstanceData(text="The answer is 4."),
 reference=EvaluationInstance.InstanceData(text="4")
 )
 ```

 Example converted input:
 ```
 {
 'response': {'text': 'The answer is 4.'},
 'reference': {'text': '4'}
 }
 ```

 Example python function:
 ```
 def evaluate(instance: dict[str, Any]) -> float:
 if instance['response']['text'] == instance['reference']['text']:
 return 1.0
 return 0.0
 ```

 CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset
 RPC) and Tuning Evaluation. Each line in the input jsonl file will be
 converted to dict[str, Any] and passed to the evaluation function.
      optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
      Returns:
      Whether the evaluationFunction field is set.

    • getEvaluationFunction

      String getEvaluationFunction()
       Required. Python function.
 Expected user to define the following function, e.g.:
 def evaluate(instance: dict[str, Any]) -> float:
 Please include this function signature in the code snippet.
 Instance is the evaluation instance, any fields populated in the instance
 are available to the function as instance[field_name].

 Example:
 Example input:
 ```
 instance= EvaluationInstance(
 response=EvaluationInstance.InstanceData(text="The answer is 4."),
 reference=EvaluationInstance.InstanceData(text="4")
 )
 ```

 Example converted input:
 ```
 {
 'response': {'text': 'The answer is 4.'},
 'reference': {'text': '4'}
 }
 ```

 Example python function:
 ```
 def evaluate(instance: dict[str, Any]) -> float:
 if instance['response']['text'] == instance['reference']['text']:
 return 1.0
 return 0.0
 ```

 CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset
 RPC) and Tuning Evaluation. Each line in the input jsonl file will be
 converted to dict[str, Any] and passed to the evaluation function.
      optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
      Returns:
      The evaluationFunction.

    • getEvaluationFunctionBytes

      com.google.protobuf.ByteString getEvaluationFunctionBytes()
       Required. Python function.
 Expected user to define the following function, e.g.:
 def evaluate(instance: dict[str, Any]) -> float:
 Please include this function signature in the code snippet.
 Instance is the evaluation instance, any fields populated in the instance
 are available to the function as instance[field_name].

 Example:
 Example input:
 ```
 instance= EvaluationInstance(
 response=EvaluationInstance.InstanceData(text="The answer is 4."),
 reference=EvaluationInstance.InstanceData(text="4")
 )
 ```

 Example converted input:
 ```
 {
 'response': {'text': 'The answer is 4.'},
 'reference': {'text': '4'}
 }
 ```

 Example python function:
 ```
 def evaluate(instance: dict[str, Any]) -> float:
 if instance['response']['text'] == instance['reference']['text']:
 return 1.0
 return 0.0
 ```

 CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset
 RPC) and Tuning Evaluation. Each line in the input jsonl file will be
 converted to dict[str, Any] and passed to the evaluation function.
      optional string evaluation_function = 1 [(.google.api.field_behavior) = REQUIRED];
      Returns:
      The bytes for evaluationFunction.