Package dev.dokimos.core.evaluators.agents

Class PlanAdherenceEvaluator.Builder

java.lang.Object
dev.dokimos.core.evaluators.agents.PlanAdherenceEvaluator.Builder
Enclosing class:
PlanAdherenceEvaluator
public static class PlanAdherenceEvaluator.Builder extends Object
Builder for constructing the evaluator.

  • Constructor Details

    • Builder

      public Builder()

  • Method Details

    • name

      public PlanAdherenceEvaluator.Builder name(String name)
      Sets the evaluator name.
      Parameters:
      name - the evaluator name
      Returns:
      this builder

    • threshold

      public PlanAdherenceEvaluator.Builder threshold(double threshold)
      Sets the minimum score threshold for success.
      Parameters:
      threshold - the threshold value
      Returns:
      this builder

    • evaluationParams

      public PlanAdherenceEvaluator.Builder evaluationParams(List<EvalTestCaseParam> params)
      Sets which test case parameters to validate.
      Parameters:
      params - the parameters
      Returns:
      this builder

    • reasoningStepsKey

      public PlanAdherenceEvaluator.Builder reasoningStepsKey(String reasoningStepsKey)
      Sets the actualOutputs key for the plan (reasoning steps).
      Parameters:
      reasoningStepsKey - the key
      Returns:
      this builder

    • toolCallsKey

      public PlanAdherenceEvaluator.Builder toolCallsKey(String toolCallsKey)
      Sets the actualOutputs key for the tool calls.
      Parameters:
      toolCallsKey - the key
      Returns:
      this builder

    • judge

      public PlanAdherenceEvaluator.Builder judge(JudgeLM judge)
      Sets an optional judge LLM for the adherence check. Without a judge, only the rule-based plan_present check runs.
      Parameters:
      judge - the judge LLM
      Returns:
      this builder

    • build

      public PlanAdherenceEvaluator build()
      Builds the evaluator.
      Returns:
      a new evaluator