Package dev.dokimos.core.evaluators.agents

Class PlanQualityEvaluator

java.lang.Object

dev.dokimos.core.BaseEvaluator

dev.dokimos.core.evaluators.agents.PlanQualityEvaluator

All Implemented Interfaces:

Evaluator

public class PlanQualityEvaluator
extends BaseEvaluator

Evaluates whether an agent's plan is coherent and goal-directed using a rule-based check and an optional LLM check.

The plan is read from actualOutputs[reasoningStepsKey] (default "reasoningSteps") as a List<String>, matching AgentTrace.toOutputMap().

Checks performed:

• non_empty_reasoning (rule): a plan is present (non-empty reasoning)
• quality (LLM): the plan is logical, coherent, and goal-directed given the input

Without a judge LLM, only the rule-based check runs. The score is the fraction of checks that passed. An empty plan short-circuits to a score of 1.0, and a missing key throws.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	PlanQualityEvaluator.Builder	Builder for constructing the evaluator.

Method Summary

Modifier and Type	Method	Description
static PlanQualityEvaluator.Builder	builder()	Creates a new builder for constructing the evaluator.

Methods inherited from class dev.dokimos.core.BaseEvaluator

evaluate, evaluateAsync, evaluateAsync, name, threshold

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static PlanQualityEvaluator.Builder builder()

Creates a new builder for constructing the evaluator.

Returns:

a new builder