Package dev.dokimos.core.evaluators.agents

Class ToolErrorEvaluator.Builder

java.lang.Object
dev.dokimos.core.evaluators.agents.ToolErrorEvaluator.Builder
Enclosing class:
ToolErrorEvaluator
public static class ToolErrorEvaluator.Builder extends Object
Builder for constructing the evaluator.

  • Constructor Details

    • Builder

      public Builder()

  • Method Details

    • name

      public ToolErrorEvaluator.Builder name(String name)
      Sets the evaluator name.
      Parameters:
      name - the evaluator name
      Returns:
      this builder

    • threshold

      public ToolErrorEvaluator.Builder threshold(double threshold)
      Sets the minimum score threshold for success.
      Parameters:
      threshold - the threshold value
      Returns:
      this builder

    • evaluationParams

      public ToolErrorEvaluator.Builder evaluationParams(List<EvalTestCaseParam> params)
      Sets which test case parameters to validate.
      Parameters:
      params - the parameters
      Returns:
      this builder

    • toolCallsKey

      public ToolErrorEvaluator.Builder toolCallsKey(String toolCallsKey)
      Sets the key used to retrieve tool calls from actualOutputs.
      Parameters:
      toolCallsKey - the key
      Returns:
      this builder

    • treatBlankAsError

      public ToolErrorEvaluator.Builder treatBlankAsError(boolean treatBlankAsError)
      Controls whether a null or blank result counts as an error.
      Parameters:
      treatBlankAsError - true to treat blank results as errors (default true)
      Returns:
      this builder

    • detectJsonErrorField

      public ToolErrorEvaluator.Builder detectJsonErrorField(boolean detectJsonErrorField)
      Controls whether a JSON object with a top-level "error" field counts as an error.
      Parameters:
      detectJsonErrorField - true to detect JSON error fields (default true)
      Returns:
      this builder

    • errorDetector

      public ToolErrorEvaluator.Builder errorDetector(Predicate<String> errorDetector)
      Sets a custom predicate that flags a result string as an error.
      Parameters:
      errorDetector - the predicate (receives the raw result string)
      Returns:
      this builder

    • build

      public ToolErrorEvaluator build()
      Builds the evaluator.
      Returns:
      a new evaluator