Class ReinforcementTuningCloudRunRewardScorer

java.lang.Object
com.google.genai.JsonSerializable
com.google.genai.types.ReinforcementTuningCloudRunRewardScorer

public abstract class ReinforcementTuningCloudRunRewardScorer extends JsonSerializable
ReinforcementTuningCloudRunRewardScorer allows users to implement a reward function through GCP Cloud Run. Comparing with ReinforcementTuningCodeExecutionRewardScorer that runs in a Sandbox and has no internet access, Cloud Run reward scorer is fully controlled by users. The Cloud Run service should implement the following HTTP API: HTTP method: `POST` HTTP request body: ``` { "example": ReinforcementTuningExample, "response": Content, "metadata": { "step": int "tuning_job_id": int64 } } ``` * `example` is a ReinforcementTuningExample in ProtoJSON format, (i.e., the format is the same as as one line in the training/validation dataset except that the keys must be in camel case). System instructions (i.e., `example.get("systemInstruction")`) and references (i.e., `example.get("references")`) are also included in the `example` provided that they are set in the training/validation dataset. * `response` is a Content in ProtoJSON format (i.e., keys must be in camel case), which is the same as the Online Prediction response for Gemini models. HTTP response body: { "reward": float, "user_requested_aux_info": str // Optional } where the field "user_requested_aux_info" is any (optional) string provided by users for assisting debugging. It's in snake case. This field is mostly useful when calling the GenAiTuningService.ValidateReinforcementTuningReward API, where the proto field (not Cloud Run HTTP response body) userRequestedAuxInfo will be populated if the Cloud Run reward function sets this field in the HTTP response. The following are examples for the HTTP request and response body. Example HTTP request body: ``` { "example": { "contents": [ { "role": "user", "parts": [ { "text": "What is the capital of France?" } ] } ], "references": { "answer": "Paris" } }, "response": { "parts": [ { "text": "London" } ] }, "metadata": { "step": 1, "tuning_job_id": 123456789 } } ``` Example HTTP response body: ``` { "reward": -1.0 } ``` Note: Reward output by Cloud Run reward function is clipped to be within `[-1, 1]`, i.e., `reward = max(min(reward, 1.0), -1.0)`. This data type is not supported in Gemini API.
  • Constructor Details

    • ReinforcementTuningCloudRunRewardScorer

      public ReinforcementTuningCloudRunRewardScorer()
  • Method Details

    • cloudRunUri

      public abstract Optional<String> cloudRunUri()
      URI of the Cloud Run service that will be used to compute the reward. The [Vertex AI Secure Fine Tuning Service Agent](https://docs.cloud.google.com/iam/docs/service-agents#vertex-ai-secure-fine-tuning-service-agent) (`service-@gcp-sa-vertex-tune.iam.gserviceaccount.com`) must be granted the permission (e.g. by granting `roles/run.invoker` in IAM) to invoke the Cloud Run service.
    • builder

      Instantiates a builder for ReinforcementTuningCloudRunRewardScorer.
    • toBuilder

      Creates a builder with the same values as this instance.
    • fromJson

      public static ReinforcementTuningCloudRunRewardScorer fromJson(String jsonString)
      Deserializes a JSON string to a ReinforcementTuningCloudRunRewardScorer object.