Package dev.dokimos.core.evaluators.agents

Class ToolErrorEvaluator

java.lang.Object

dev.dokimos.core.BaseEvaluator

dev.dokimos.core.evaluators.agents.ToolErrorEvaluator

All Implemented Interfaces:

Evaluator

public class ToolErrorEvaluator
extends BaseEvaluator

Detects tool execution failures by inspecting ToolCall.result().

This is a deterministic, glass-box evaluator (no LLM). A tool call is considered failed when any of the configured detectors fire:

• the result is null or blank (when treatBlankAsError, the default);
• the result is a JSON object with a non-null top-level "error" field (when detectJsonErrorField, the default);
• a custom Predicate supplied via ToolErrorEvaluator.Builder.errorDetector(Predicate) matches.

The score is the fraction of tool calls that did not fail. An empty tool-call list scores 1.0, consistent with the other agent evaluators.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ToolErrorEvaluator.Builder	Builder for constructing the evaluator.

Method Summary

Modifier and Type	Method	Description
static ToolErrorEvaluator.Builder	builder()	Creates a new builder for constructing the evaluator.

Methods inherited from class dev.dokimos.core.BaseEvaluator

evaluate, evaluateAsync, evaluateAsync, name, threshold

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static ToolErrorEvaluator.Builder builder()

Creates a new builder for constructing the evaluator.

Returns:

a new builder