Package dev.dokimos.core.evaluators

Class ContextualRelevanceEvaluator.Builder

java.lang.Object
dev.dokimos.core.evaluators.ContextualRelevanceEvaluator.Builder
Enclosing class:
ContextualRelevanceEvaluator
public static class ContextualRelevanceEvaluator.Builder extends Object
Builder for constructing ContextualRelevanceEvaluator instances.

  • Constructor Details

    • Builder

      public Builder()

  • Method Details

    • name

      public ContextualRelevanceEvaluator.Builder name(String name)
      Sets the evaluator name.
      Parameters:
      name - the evaluator name
      Returns:
      this builder

    • retrievalContextKey

      public ContextualRelevanceEvaluator.Builder retrievalContextKey(String retrievalContextKey)
      Sets the key used to retrieve context chunks from actualOutputs. Defaults to "retrievalContext".
      Parameters:
      retrievalContextKey - the key for retrieval context
      Returns:
      this builder

    • threshold

      public ContextualRelevanceEvaluator.Builder threshold(double threshold)
      Sets the minimum score threshold for success. Default is 0.5.
      Parameters:
      threshold - the threshold value (0.0 to 1.0)
      Returns:
      this builder

    • strictMode

      public ContextualRelevanceEvaluator.Builder strictMode(boolean strictMode)
      Enables or disables strict mode. When enabled, the threshold is automatically set to 1.0. Default is false.
      Parameters:
      strictMode - true to enable strict mode
      Returns:
      this builder

    • evaluationParams

      public ContextualRelevanceEvaluator.Builder evaluationParams(List<EvalTestCaseParam> params)
      Sets which test case parameters to validate before evaluation.
      Parameters:
      params - the parameters to validate
      Returns:
      this builder

    • judge

      public ContextualRelevanceEvaluator.Builder judge(JudgeLM judge)
      Sets the LLM judge to use for scoring context relevance.
      Parameters:
      judge - the LLM judge implementation
      Returns:
      this builder

    • includeReason

      public ContextualRelevanceEvaluator.Builder includeReason(boolean includeReason)
      Sets whether to include a detailed reason in the result. Default is true.
      Parameters:
      includeReason - true to include reasoning
      Returns:
      this builder

    • build

      public ContextualRelevanceEvaluator build()
      Builds the ContextualRelevanceEvaluator.
      Returns:
      a new ContextualRelevanceEvaluator instance
      Throws:
      IllegalStateException - if judge is not set