Package com.google.genai.types
Class ReinforcementTuningCloudRunRewardScorer
java.lang.Object
com.google.genai.JsonSerializable
com.google.genai.types.ReinforcementTuningCloudRunRewardScorer
ReinforcementTuningCloudRunRewardScorer allows users to implement a reward function through GCP
Cloud Run. Comparing with ReinforcementTuningCodeExecutionRewardScorer that runs in a Sandbox and
has no internet access, Cloud Run reward scorer is fully controlled by users. The Cloud Run
service should implement the following HTTP API: HTTP method: `POST` HTTP request body: ``` {
"example": ReinforcementTuningExample, "response": Content, "metadata": { "step": int
"tuning_job_id": int64 } } ``` * `example` is a ReinforcementTuningExample in ProtoJSON format,
(i.e., the format is the same as as one line in the training/validation dataset except that the
keys must be in camel case). System instructions (i.e., `example.get("systemInstruction")`) and
references (i.e., `example.get("references")`) are also included in the `example` provided that
they are set in the training/validation dataset. * `response` is a Content in ProtoJSON format
(i.e., keys must be in camel case), which is the same as the Online Prediction response for
Gemini models. HTTP response body: { "reward": float, "user_requested_aux_info": str // Optional
} where the field "user_requested_aux_info" is any (optional) string provided by users for
assisting debugging. It's in snake case. This field is mostly useful when calling the
GenAiTuningService.ValidateReinforcementTuningReward API, where the proto field (not Cloud Run
HTTP response body) userRequestedAuxInfo will be populated if the Cloud Run reward function sets
this field in the HTTP response. The following are examples for the HTTP request and response
body. Example HTTP request body: ``` { "example": { "contents": [ { "role": "user", "parts": [ {
"text": "What is the capital of France?" } ] } ], "references": { "answer": "Paris" } },
"response": { "parts": [ { "text": "London" } ] }, "metadata": { "step": 1, "tuning_job_id":
123456789 } } ``` Example HTTP response body: ``` { "reward": -1.0 } ``` Note: Reward output by
Cloud Run reward function is clipped to be within `[-1, 1]`, i.e., `reward = max(min(reward,
1.0), -1.0)`. This data type is not supported in Gemini API.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic classBuilder for ReinforcementTuningCloudRunRewardScorer. -
Field Summary
Fields inherited from class com.google.genai.JsonSerializable
MAX_READ_LENGTH_PROPERTY -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionbuilder()Instantiates a builder for ReinforcementTuningCloudRunRewardScorer.URI of the Cloud Run service that will be used to compute the reward.Deserializes a JSON string to a ReinforcementTuningCloudRunRewardScorer object.Creates a builder with the same values as this instance.Methods inherited from class com.google.genai.JsonSerializable
fromJsonNode, fromJsonString, objectMapper, setMaxReadLength, stringToJsonNode, toJson, toJsonNode, toJsonString
-
Constructor Details
-
ReinforcementTuningCloudRunRewardScorer
public ReinforcementTuningCloudRunRewardScorer()
-
-
Method Details
-
cloudRunUri
URI of the Cloud Run service that will be used to compute the reward. The [Vertex AI Secure Fine Tuning Service Agent](https://docs.cloud.google.com/iam/docs/service-agents#vertex-ai-secure-fine-tuning-service-agent) (`service-@gcp-sa-vertex-tune.iam.gserviceaccount.com`) must be granted the permission (e.g. by granting `roles/run.invoker` in IAM) to invoke the Cloud Run service. -
builder
Instantiates a builder for ReinforcementTuningCloudRunRewardScorer. -
toBuilder
Creates a builder with the same values as this instance. -
fromJson
Deserializes a JSON string to a ReinforcementTuningCloudRunRewardScorer object.
-