Package dev.dokimos.core.evaluators.agents

Class ToolDescriptionReliabilityEvaluator.Builder

java.lang.Object

dev.dokimos.core.evaluators.agents.ToolDescriptionReliabilityEvaluator.Builder

Enclosing class:

ToolDescriptionReliabilityEvaluator

public static class ToolDescriptionReliabilityEvaluator.Builder
extends Object

Builder for constructing the evaluator.

Constructor Summary

Constructors

Constructor	Description
Builder()

Method Summary

Modifier and Type	Method	Description
ToolDescriptionReliabilityEvaluator	build()	Builds the evaluator.
ToolDescriptionReliabilityEvaluator.Builder	evaluationParams(List<EvalTestCaseParam> params)	Sets which test case parameters to validate.
ToolDescriptionReliabilityEvaluator.Builder	judge(JudgeLM judge)	Sets an optional judge LLM for semantic checks.
ToolDescriptionReliabilityEvaluator.Builder	maxInputArgs(int maxInputArgs)	Sets the maximum number of total input parameters allowed per tool.
ToolDescriptionReliabilityEvaluator.Builder	maxOptionalArgs(int maxOptionalArgs)	Sets the maximum number of optional parameters allowed per tool.
ToolDescriptionReliabilityEvaluator.Builder	name(String name)	Sets the evaluator name.
ToolDescriptionReliabilityEvaluator.Builder	threshold(double threshold)	Sets the minimum score threshold for success.
ToolDescriptionReliabilityEvaluator.Builder	toolsKey(String toolsKey)	Sets the metadata key for tool definitions.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Builder

public Builder()

Method Details

name

public ToolDescriptionReliabilityEvaluator.Builder name(String name)

Sets the evaluator name.

Parameters:

name - the evaluator name

Returns:

this builder

threshold

public ToolDescriptionReliabilityEvaluator.Builder threshold(double threshold)

Sets the minimum score threshold for success.

Parameters:

threshold - the threshold value

Returns:

this builder

evaluationParams

public ToolDescriptionReliabilityEvaluator.Builder evaluationParams(List<EvalTestCaseParam> params)

Sets which test case parameters to validate.

Parameters:

params - the parameters

Returns:

this builder

toolsKey

public ToolDescriptionReliabilityEvaluator.Builder toolsKey(String toolsKey)

Sets the metadata key for tool definitions.

Parameters:

toolsKey - the key

Returns:

this builder

judge

public ToolDescriptionReliabilityEvaluator.Builder judge(JudgeLM judge)

Sets an optional judge LLM for semantic checks. Without a judge, only rule-based checks run (4 out of 13).

Parameters:

judge - the judge LLM

Returns:

this builder

maxOptionalArgs

public ToolDescriptionReliabilityEvaluator.Builder maxOptionalArgs(int maxOptionalArgs)

Sets the maximum number of optional parameters allowed per tool. Default: 3.

Parameters:

maxOptionalArgs - the maximum count

Returns:

this builder

maxInputArgs

public ToolDescriptionReliabilityEvaluator.Builder maxInputArgs(int maxInputArgs)

Sets the maximum number of total input parameters allowed per tool. Default: 5.

Parameters:

maxInputArgs - the maximum count

Returns:

this builder

build

public ToolDescriptionReliabilityEvaluator build()

Builds the evaluator.

Returns:

a new evaluator