Package dev.dokimos.core.evaluators

Class PrecisionEvaluator

java.lang.Object

dev.dokimos.core.BaseEvaluator

dev.dokimos.core.evaluators.PrecisionEvaluator

All Implemented Interfaces:

Evaluator

public class PrecisionEvaluator
extends BaseEvaluator

Evaluator that measures retrieval precision.

Precision is the fraction of retrieved items that are relevant:

 precision = |relevant ∩ retrieved| / |retrieved|
 

A precision of 1.0 means every retrieved item was relevant (no false positives). A precision of 0.0 means no retrieved items were relevant.

This evaluator supports various RAG use cases beyond document retrieval, including knowledge graph triples, API responses, and semantic matching.

Example usage:

 var evaluator = PrecisionEvaluator.builder()
         .name("retrieval-precision")
         .retrievedKey("retrievedDocs")
         .expectedKey("relevantDocs")
         .matchingStrategy(MatchingStrategy.byEquality())
         .threshold(0.8)
         .build();

 var testCase = EvalTestCase.builder()
         .input("What causes diabetes?")
         .actualOutput("retrievedDocs", List.of("doc_1", "doc_2", "doc_3"))
         .expectedOutput("relevantDocs", List.of("doc_1", "doc_3", "doc_5"))
         .build();

 EvalResult result = evaluator.evaluate(testCase);
 // precision = 2/3 = 0.667 (doc_1 and doc_3 are relevant)
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	PrecisionEvaluator.Builder	Builder for constructing PrecisionEvaluator instances.

Method Summary

Modifier and Type	Method	Description
static PrecisionEvaluator.Builder	builder()	Creates a new builder for constructing the precision evaluator.

Methods inherited from class dev.dokimos.core.BaseEvaluator

evaluate, evaluateAsync, evaluateAsync, name, threshold

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static PrecisionEvaluator.Builder builder()

Creates a new builder for constructing the precision evaluator.

Returns:

a new builder