Package com.google.cloud.aiplatform.v1beta1

Class EvaluationParserConfig.CustomCodeParserConfig

java.lang.Object
com.google.protobuf.AbstractMessageLite
com.google.protobuf.AbstractMessage
com.google.protobuf.GeneratedMessage
com.google.cloud.aiplatform.v1beta1.EvaluationParserConfig.CustomCodeParserConfig
All Implemented Interfaces:
EvaluationParserConfig.CustomCodeParserConfigOrBuilder, com.google.protobuf.Message, com.google.protobuf.MessageLite, com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder, Serializable
Enclosing class:
EvaluationParserConfig
public static final class EvaluationParserConfig.CustomCodeParserConfig extends com.google.protobuf.GeneratedMessage implements EvaluationParserConfig.CustomCodeParserConfigOrBuilder
 Configuration for parsing the LLM response using custom code.
Protobuf type google.cloud.aiplatform.v1beta1.EvaluationParserConfig.CustomCodeParserConfig
See Also:

  • Field Details

    • PARSING_FUNCTION_FIELD_NUMBER

      public static final int PARSING_FUNCTION_FIELD_NUMBER
      See Also:

  • Method Details

    • getDescriptor

      public static final com.google.protobuf.Descriptors.Descriptor getDescriptor()

    • internalGetFieldAccessorTable

      protected com.google.protobuf.GeneratedMessage.FieldAccessorTable internalGetFieldAccessorTable()
      Specified by:
      internalGetFieldAccessorTable in class com.google.protobuf.GeneratedMessage

    • hasParsingFunction

      public boolean hasParsingFunction()
       Required. Python function for parsing results. The function should be
 defined within this string.

 The function takes a list of strings (LLM responses) and should return
 either a list of dictionaries (for rubrics) or a single dictionary
 (for a metric result).

 Example function signature:
 def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

 When parsing rubrics, return a list of dictionaries, where each
 dictionary represents a Rubric.
 Example for rubrics:
 [
 {
 "content": {"property": {"description": "The response is
 factual."}},
 "type": "FACTUALITY",
 "importance": "HIGH"
 },
 {
 "content": {"property": {"description": "The response is
 fluent."}},
 "type": "FLUENCY",
 "importance": "MEDIUM"
 }
 ]

 When parsing critique results, return a dictionary representing a
 MetricResult.
 Example for a metric result:
 {
 "score": 0.8,
 "explanation": "The model followed most instructions.",
 "rubric_verdicts": [...]
 }

 ... code for result extraction and aggregation
      optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];
      Specified by:
      hasParsingFunction in interface EvaluationParserConfig.CustomCodeParserConfigOrBuilder
      Returns:
      Whether the parsingFunction field is set.

    • getParsingFunction

      public String getParsingFunction()
       Required. Python function for parsing results. The function should be
 defined within this string.

 The function takes a list of strings (LLM responses) and should return
 either a list of dictionaries (for rubrics) or a single dictionary
 (for a metric result).

 Example function signature:
 def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

 When parsing rubrics, return a list of dictionaries, where each
 dictionary represents a Rubric.
 Example for rubrics:
 [
 {
 "content": {"property": {"description": "The response is
 factual."}},
 "type": "FACTUALITY",
 "importance": "HIGH"
 },
 {
 "content": {"property": {"description": "The response is
 fluent."}},
 "type": "FLUENCY",
 "importance": "MEDIUM"
 }
 ]

 When parsing critique results, return a dictionary representing a
 MetricResult.
 Example for a metric result:
 {
 "score": 0.8,
 "explanation": "The model followed most instructions.",
 "rubric_verdicts": [...]
 }

 ... code for result extraction and aggregation
      optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];
      Specified by:
      getParsingFunction in interface EvaluationParserConfig.CustomCodeParserConfigOrBuilder
      Returns:
      The parsingFunction.

    • getParsingFunctionBytes

      public com.google.protobuf.ByteString getParsingFunctionBytes()
       Required. Python function for parsing results. The function should be
 defined within this string.

 The function takes a list of strings (LLM responses) and should return
 either a list of dictionaries (for rubrics) or a single dictionary
 (for a metric result).

 Example function signature:
 def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

 When parsing rubrics, return a list of dictionaries, where each
 dictionary represents a Rubric.
 Example for rubrics:
 [
 {
 "content": {"property": {"description": "The response is
 factual."}},
 "type": "FACTUALITY",
 "importance": "HIGH"
 },
 {
 "content": {"property": {"description": "The response is
 fluent."}},
 "type": "FLUENCY",
 "importance": "MEDIUM"
 }
 ]

 When parsing critique results, return a dictionary representing a
 MetricResult.
 Example for a metric result:
 {
 "score": 0.8,
 "explanation": "The model followed most instructions.",
 "rubric_verdicts": [...]
 }

 ... code for result extraction and aggregation
      optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];
      Specified by:
      getParsingFunctionBytes in interface EvaluationParserConfig.CustomCodeParserConfigOrBuilder
      Returns:
      The bytes for parsingFunction.

    • isInitialized

      public final boolean isInitialized()
      Specified by:
      isInitialized in interface com.google.protobuf.MessageLiteOrBuilder
      Overrides:
      isInitialized in class com.google.protobuf.GeneratedMessage

    • writeTo

      public void writeTo(com.google.protobuf.CodedOutputStream output) throws IOException
      Specified by:
      writeTo in interface com.google.protobuf.MessageLite
      Overrides:
      writeTo in class com.google.protobuf.GeneratedMessage
      Throws:
      IOException

    • getSerializedSize

      public int getSerializedSize()
      Specified by:
      getSerializedSize in interface com.google.protobuf.MessageLite
      Overrides:
      getSerializedSize in class com.google.protobuf.GeneratedMessage

    • equals

      public boolean equals(Object obj)
      Specified by:
      equals in interface com.google.protobuf.Message
      Overrides:
      equals in class com.google.protobuf.AbstractMessage

    • hashCode

      public int hashCode()
      Specified by:
      hashCode in interface com.google.protobuf.Message
      Overrides:
      hashCode in class com.google.protobuf.AbstractMessage

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(ByteBuffer data) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(ByteBuffer data, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(com.google.protobuf.ByteString data) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(com.google.protobuf.ByteString data, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(byte[] data) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(byte[] data, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(InputStream input) throws IOException
      Throws:
      IOException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(InputStream input, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws IOException
      Throws:
      IOException

    • parseDelimitedFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseDelimitedFrom(InputStream input) throws IOException
      Throws:
      IOException

    • parseDelimitedFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseDelimitedFrom(InputStream input, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws IOException
      Throws:
      IOException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(com.google.protobuf.CodedInputStream input) throws IOException
      Throws:
      IOException

    • parseFrom

      public static EvaluationParserConfig.CustomCodeParserConfig parseFrom(com.google.protobuf.CodedInputStream input, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws IOException
      Throws:
      IOException

    • newBuilderForType

      public EvaluationParserConfig.CustomCodeParserConfig.Builder newBuilderForType()
      Specified by:
      newBuilderForType in interface com.google.protobuf.Message
      Specified by:
      newBuilderForType in interface com.google.protobuf.MessageLite

    • newBuilder

      public static EvaluationParserConfig.CustomCodeParserConfig.Builder newBuilder()

    • newBuilder

      public static EvaluationParserConfig.CustomCodeParserConfig.Builder newBuilder(EvaluationParserConfig.CustomCodeParserConfig prototype)

    • toBuilder

      public EvaluationParserConfig.CustomCodeParserConfig.Builder toBuilder()
      Specified by:
      toBuilder in interface com.google.protobuf.Message
      Specified by:
      toBuilder in interface com.google.protobuf.MessageLite

    • newBuilderForType

      protected EvaluationParserConfig.CustomCodeParserConfig.Builder newBuilderForType(com.google.protobuf.AbstractMessage.BuilderParent parent)
      Overrides:
      newBuilderForType in class com.google.protobuf.AbstractMessage

    • getDefaultInstance

      public static EvaluationParserConfig.CustomCodeParserConfig getDefaultInstance()

    • parser

      public static com.google.protobuf.Parser<EvaluationParserConfig.CustomCodeParserConfig> parser()

    • getParserForType

      public com.google.protobuf.Parser<EvaluationParserConfig.CustomCodeParserConfig> getParserForType()
      Specified by:
      getParserForType in interface com.google.protobuf.Message
      Specified by:
      getParserForType in interface com.google.protobuf.MessageLite
      Overrides:
      getParserForType in class com.google.protobuf.GeneratedMessage

    • getDefaultInstanceForType

      public EvaluationParserConfig.CustomCodeParserConfig getDefaultInstanceForType()
      Specified by:
      getDefaultInstanceForType in interface com.google.protobuf.MessageLiteOrBuilder
      Specified by:
      getDefaultInstanceForType in interface com.google.protobuf.MessageOrBuilder