Class ReinforcementTuningAutoraterScorerExactMatchScorer.Builder
- Enclosing class:
- ReinforcementTuningAutoraterScorerExactMatchScorer
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionbuild()Clears the value of correctAnswerReward field.Clears the value of expression field.Clears the value of wrongAnswerReward field.correctAnswerReward(Float correctAnswerReward) Setter for correctAnswerReward.expression(String expression) Setter for expression.wrongAnswerReward(Float wrongAnswerReward) Setter for wrongAnswerReward.
-
Constructor Details
-
Builder
public Builder()
-
-
Method Details
-
correctAnswerReward
public abstract ReinforcementTuningAutoraterScorerExactMatchScorer.Builder correctAnswerReward(Float correctAnswerReward) Setter for correctAnswerReward.correctAnswerReward: Assigns this reward score if the parsed response string equals the expression.
-
clearCorrectAnswerReward
@CanIgnoreReturnValue public ReinforcementTuningAutoraterScorerExactMatchScorer.Builder clearCorrectAnswerReward()Clears the value of correctAnswerReward field. -
wrongAnswerReward
public abstract ReinforcementTuningAutoraterScorerExactMatchScorer.Builder wrongAnswerReward(Float wrongAnswerReward) Setter for wrongAnswerReward.wrongAnswerReward: Assigns this reward score if the parsed reward value does not equal the expression.
-
clearWrongAnswerReward
@CanIgnoreReturnValue public ReinforcementTuningAutoraterScorerExactMatchScorer.Builder clearWrongAnswerReward()Clears the value of wrongAnswerReward field. -
expression
public abstract ReinforcementTuningAutoraterScorerExactMatchScorer.Builder expression(String expression) Setter for expression.expression: The string expression to match against for scoring. This field supports placeholders in the format of {{references.key}} that will be replaced before matching. Regex is not supported for this expression. For example, users can define an ExactMatchScorer as follows: { "correctAnswerReward": 1.0, "wrongAnswerReward": -1.0, "expression": "{{references.concise_answer}}" } When evaluating the reward for each parsed autorater response, if the prompt references in the training/validation dataset has the following fields: ``` { "example": ..., "references": { "concise_ansser": "Yes", "verbose_answer": "The answer is Yes" } } ``` The above ExactMatchScorer will be replaced as follows for scoring: ``` { "correctAnswerReward": 1.0, "wrongAnswerReward": -1.0, "expression": "Yes" } ``` If the *parsed* autorater response is equal to the string `"Yes"`, then the reward is `1.0`, otherwise the reward is `-1.0`.
-
clearExpression
@CanIgnoreReturnValue public ReinforcementTuningAutoraterScorerExactMatchScorer.Builder clearExpression()Clears the value of expression field. -
build
-