Package dev.dokimos.core.conversation

Class TrajectoryEvaluator

java.lang.Object

dev.dokimos.core.BaseEvaluator

dev.dokimos.core.conversation.TrajectoryEvaluator

All Implemented Interfaces:

Evaluator

public class TrajectoryEvaluator
extends BaseEvaluator

Evaluates complete conversation trajectories using LLM-as-judge patterns.

This evaluator assesses holistic qualities of multi-turn conversations such as user satisfaction, goal completion, conversation quality, and other custom criteria. It processes the entire conversation history rather than individual turns.

Example usage:

 TrajectoryEvaluator evaluator = TrajectoryEvaluator.builder()
         .name("Customer Service Quality")
         .threshold(0.7)
         .judge(judgeLM)
         .criteria(List.of(
                 TrajectoryEvaluationCriteria.userSatisfaction(),
                 TrajectoryEvaluationCriteria.problemResolution()))
         .aggregationStrategy(AggregationStrategy.WEIGHTED_MEAN)
         .build();

 EvalTestCase testCase = EvalTestCase.builder()
         .actualOutput("trajectory", trajectory)
         .build();

 EvalResult result = evaluator.evaluate(testCase);
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TrajectoryEvaluator.Builder	Builder for constructing trajectory evaluators.

Method Summary

Modifier and Type	Method	Description
static TrajectoryEvaluator.Builder	builder()	Creates a new builder for constructing a trajectory evaluator.

Methods inherited from class dev.dokimos.core.BaseEvaluator

evaluate, evaluateAsync, evaluateAsync, name, threshold

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static TrajectoryEvaluator.Builder builder()

Creates a new builder for constructing a trajectory evaluator.

Returns:

a new builder