Package dev.dokimos.core.evaluators.agents

Class ToolTrajectoryEvaluator.Builder

java.lang.Object

dev.dokimos.core.evaluators.agents.ToolTrajectoryEvaluator.Builder

Enclosing class:

ToolTrajectoryEvaluator

public static class ToolTrajectoryEvaluator.Builder
extends Object

Builder for constructing the evaluator.

Constructor Summary

Constructors

Constructor	Description
Builder()

Method Summary

Modifier and Type	Method	Description
ToolTrajectoryEvaluator.Builder	argumentMatcher(ArgumentMatcher matcher)	Sets the default argument matcher applied to every tool unless overridden.
ToolTrajectoryEvaluator.Builder	argumentMatcher(String toolName, ArgumentMatcher matcher)	Overrides the argument matcher for a specific tool by name.
ToolTrajectoryEvaluator	build()	Builds the evaluator.
ToolTrajectoryEvaluator.Builder	evaluationParams(List<EvalTestCaseParam> params)	Sets which test case parameters to validate.
ToolTrajectoryEvaluator.Builder	expectedToolCallsKey(String expectedToolCallsKey)	Sets the key used to retrieve expected tool calls from expectedOutputs.
ToolTrajectoryEvaluator.Builder	matchMode(ToolTrajectoryEvaluator.MatchMode matchMode)	Sets the trajectory match mode.
ToolTrajectoryEvaluator.Builder	name(String name)	Sets the evaluator name.
ToolTrajectoryEvaluator.Builder	threshold(double threshold)	Sets the minimum score threshold for success.
ToolTrajectoryEvaluator.Builder	toolCallsKey(String toolCallsKey)	Sets the key used to retrieve actual tool calls from actualOutputs.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Builder

public Builder()

Method Details

name

public ToolTrajectoryEvaluator.Builder name(String name)

Sets the evaluator name.

Parameters:

name - the evaluator name

Returns:

this builder

threshold

public ToolTrajectoryEvaluator.Builder threshold(double threshold)

Sets the minimum score threshold for success.

Parameters:

threshold - the threshold value

Returns:

this builder

evaluationParams

public ToolTrajectoryEvaluator.Builder evaluationParams(List<EvalTestCaseParam> params)

Sets which test case parameters to validate.

Parameters:

params - the parameters

Returns:

this builder

toolCallsKey

public ToolTrajectoryEvaluator.Builder toolCallsKey(String toolCallsKey)

Sets the key used to retrieve actual tool calls from actualOutputs.

Parameters:

toolCallsKey - the key

Returns:

this builder

expectedToolCallsKey

public ToolTrajectoryEvaluator.Builder expectedToolCallsKey(String expectedToolCallsKey)

Sets the key used to retrieve expected tool calls from expectedOutputs.

Parameters:

expectedToolCallsKey - the key

Returns:

this builder

matchMode

public ToolTrajectoryEvaluator.Builder matchMode(ToolTrajectoryEvaluator.MatchMode matchMode)

Sets the trajectory match mode.

Parameters:

matchMode - the match mode (default ToolTrajectoryEvaluator.MatchMode.IN_ORDER)

Returns:

this builder

argumentMatcher

public ToolTrajectoryEvaluator.Builder argumentMatcher(ArgumentMatcher matcher)

Sets the default argument matcher applied to every tool unless overridden.

Defaults to ArgumentMatcher.tolerant(), so arguments are compared by default. Supply ArgumentMatcher.of(ArgMatchMode.IGNORE) to compare tool names and order only.

Parameters:

matcher - the default argument matcher

Returns:

this builder

argumentMatcher

public ToolTrajectoryEvaluator.Builder argumentMatcher(String toolName,
 ArgumentMatcher matcher)

Overrides the argument matcher for a specific tool by name.

Parameters:

toolName - the tool name

matcher - the argument matcher to use for that tool

Returns:

this builder

build

public ToolTrajectoryEvaluator build()

Builds the evaluator.

Returns:

a new evaluator