Package dev.dokimos.core.conversation

Class TrajectoryEvaluationCriteria

java.lang.Object
dev.dokimos.core.conversation.TrajectoryEvaluationCriteria
public final class TrajectoryEvaluationCriteria extends Object
Factory for creating pre-built evaluation criteria for trajectory evaluation.

This class provides ready-to-use criteria for common conversation evaluation dimensions. Each criterion includes a detailed description that guides the LLM judge in scoring the conversation.

Example usage: 


 TrajectoryEvaluator evaluator = TrajectoryEvaluator.builder()
         .judge(judgeLM)
         .criteria(List.of(
                 TrajectoryEvaluationCriteria.userSatisfaction(),
                 TrajectoryEvaluationCriteria.problemResolution(),
                 TrajectoryEvaluationCriteria.professionalTone()))
         .build();

  • Method Details

    • userSatisfaction

      public static EvaluationCriterion userSatisfaction()
      Creates a criterion for evaluating user satisfaction.

      Assesses whether the user's concerns were adequately addressed and whether they seem satisfied by the end of the conversation.

      Returns:
      a user satisfaction criterion

    • goalCompletion

      public static EvaluationCriterion goalCompletion()
      Creates a criterion for evaluating goal completion.

      Assesses whether the assistant successfully achieved the stated or implied goal of the conversation.

      Returns:
      a goal completion criterion

    • conversationQuality

      public static EvaluationCriterion conversationQuality()
      Creates a criterion for evaluating conversation quality.

      Assesses the overall flow, coherence, and naturalness of the dialogue.

      Returns:
      a conversation quality criterion

    • responseRelevance

      public static EvaluationCriterion responseRelevance()
      Creates a criterion for evaluating response relevance.

      Assesses whether the assistant's responses were on-topic and directly addressed the user's questions.

      Returns:
      a response relevance criterion

    • professionalTone

      public static EvaluationCriterion professionalTone()
      Creates a criterion for evaluating professional tone.

      Assesses whether the assistant maintained appropriate language, demeanor, and professionalism throughout.

      Returns:
      a professional tone criterion

    • problemResolution

      public static EvaluationCriterion problemResolution()
      Creates a criterion for evaluating problem resolution.

      Assesses whether issues or problems raised were effectively resolved.

      Returns:
      a problem resolution criterion

    • informationAccuracy

      public static EvaluationCriterion informationAccuracy()
      Creates a criterion for evaluating information accuracy.

      Assesses whether the information provided was correct and reliable.

      Returns:
      an information accuracy criterion

    • clarity

      public static EvaluationCriterion clarity()
      Creates a criterion for evaluating clarity of communication.

      Assesses how clearly and understandably the assistant communicated.

      Returns:
      a clarity criterion

    • helpfulness

      public static EvaluationCriterion helpfulness()
      Creates a criterion for evaluating helpfulness.

      Assesses how genuinely helpful and service-oriented the assistant was.

      Returns:
      a helpfulness criterion

    • consistency

      public static EvaluationCriterion consistency()
      Creates a criterion for evaluating consistency.

      Assesses whether the assistant maintained consistent information and behavior throughout the conversation.

      Returns:
      a consistency criterion

    • safety

      public static EvaluationCriterion safety()
      Creates a criterion for evaluating safety and appropriateness.

      Assesses whether the assistant avoided harmful, inappropriate, or unsafe responses.

      Returns:
      a safety criterion

    • custom

      public static EvaluationCriterion custom(String name, String description)
      Creates a custom evaluation criterion.

      Use this method when the pre-built criteria don't fit your evaluation needs.

      Parameters:
      name - the criterion name
      description - detailed instructions for evaluation
      Returns:
      a custom criterion with weight 1.0

    • custom

      public static EvaluationCriterion custom(String name, String description, double weight)
      Creates a custom evaluation criterion with specified weight.
      Parameters:
      name - the criterion name
      description - detailed instructions for evaluation
      weight - the weight for score aggregation
      Returns:
      a custom criterion with the specified weight