Package dev.dokimos.core

Interface Evaluator

All Known Implementing Classes:
BaseEvaluator, ContextualRelevanceEvaluator, ExactMatchEvaluator, FaithfulnessEvaluator, HallucinationEvaluator, LLMJudgeEvaluator, PrecisionEvaluator, RecallEvaluator, RegexEvaluator, TaskCompletionEvaluator, ToolArgumentHallucinationEvaluator, ToolCallValidityEvaluator, ToolCorrectnessEvaluator, ToolDescriptionReliabilityEvaluator, ToolNameReliabilityEvaluator, TrajectoryEvaluator
public interface Evaluator
Evaluates test cases and produces scored results.

  • Method Summary

    Modifier and Type
    Method
    Description
    EvalResult
    evaluate(EvalTestCase testCase)
    Evaluates the test case and returns a scored result.
    String
    name()
    Returns the evaluator name.
    double
    threshold()
    Returns the minimum score threshold for success.

  • Method Details

    • evaluate

      EvalResult evaluate(EvalTestCase testCase)
      Evaluates the test case and returns a scored result.
      Parameters:
      testCase - the test case to evaluate
      Returns:
      the evaluation result

    • name

      String name()
      Returns the evaluator name.
      Returns:
      the evaluator name

    • threshold

      double threshold()
      Returns the minimum score threshold for success.
      Returns:
      the threshold value