Uses of Interface
dev.dokimos.core.Evaluator
Packages that use Evaluator
Package
Description
-
Uses of Evaluator in dev.dokimos.core
Classes in dev.dokimos.core that implement EvaluatorModifier and TypeClassDescriptionclassBase class for implementing concrete evaluators.Methods in dev.dokimos.core with parameters of type EvaluatorModifier and TypeMethodDescriptionstatic voidAssertions.assertEval(EvalTestCase testCase, Evaluator... evaluators) Asserts that the test case passes all evaluators.Adds a single evaluator to the experiment.Method parameters in dev.dokimos.core with type arguments of type EvaluatorModifier and TypeMethodDescriptionstatic voidAssertions.assertEval(EvalTestCase testCase, List<Evaluator> evaluators) Asserts that the test case passes all evaluators.Experiment.Builder.evaluators(List<Evaluator> evaluators) Adds multiple evaluators to the experiment. -
Uses of Evaluator in dev.dokimos.core.conversation
Classes in dev.dokimos.core.conversation that implement EvaluatorModifier and TypeClassDescriptionclassEvaluates complete conversation trajectories using LLM-as-judge patterns. -
Uses of Evaluator in dev.dokimos.core.evaluators
Classes in dev.dokimos.core.evaluators that implement EvaluatorModifier and TypeClassDescriptionclassEvaluator that measures how relevant retrieved context chunks are to a user's input query.classEvaluator that checks for exact string match between actual and expected outputs.classEvaluator that uses an LLM to check how much of the actual output is backed by the given context.classEvaluator that uses an LLM to detect hallucinations in the actual output.classEvaluator that uses an LLM to evaluate outputs based on the specified criteria.classEvaluator that measures retrieval precision.classEvaluator that measures retrieval recall.classEvaluator that checks if the actual output matches a regular expression pattern.classCompares the actual output against the expected output as JSON structures rather than as opaque strings.final classEvaluates selected fields of a structured task output with an existing evaluator.Methods in dev.dokimos.core.evaluators with parameters of type EvaluatorModifier and TypeMethodDescriptionTypedEvaluator.Builder.evaluateWith(Evaluator delegate) Finishes the builder and returns the wrapping evaluator. -
Uses of Evaluator in dev.dokimos.core.evaluators.agents
Classes in dev.dokimos.core.evaluators.agents that implement EvaluatorModifier and TypeClassDescriptionclassEvaluates whether an agent's executed tool calls followed its stated plan using a rule-based check and an optional LLM check.classEvaluates whether an agent's plan is coherent and goal-directed using a rule-based check and an optional LLM check.classEvaluates whether an AI agent completed the user's requested tasks.classUses a judge LLM to assess whether tool call argument values are factually grounded in the user's input and preceding tool call results.classValidates that tool calls are syntactically correct per their JSON schema definitions.classChecks whether the agent used the expected set of tools.classEvaluates tool description quality using a mix of rule-based checks and optional LLM checks.classMeasures how efficiently an agent used its tools by detecting redundant calls.classDetects tool execution failures by inspectingToolCall.result().classEvaluates tool naming quality using a mix of rule-based checks and optional LLM checks.classEvaluates an agent's tool-call trajectory against an expected trajectory using a selectable match mode.