Package dev.dokimos.server.repository

Interface ItemResultRepository

All Superinterfaces:
org.springframework.data.repository.CrudRepository<ItemResult,UUID>, org.springframework.data.jpa.repository.JpaRepository<ItemResult,UUID>, org.springframework.data.repository.ListCrudRepository<ItemResult,UUID>, org.springframework.data.repository.ListPagingAndSortingRepository<ItemResult,UUID>, org.springframework.data.repository.PagingAndSortingRepository<ItemResult,UUID>, org.springframework.data.repository.query.QueryByExampleExecutor<ItemResult>, org.springframework.data.repository.Repository<ItemResult,UUID>
public interface ItemResultRepository extends org.springframework.data.jpa.repository.JpaRepository<ItemResult,UUID>

  • Method Summary

    Modifier and Type
    Method
    Description
    Double
    avgLatencyByRun(ExperimentRun run)
    Averages the call latency across a run's items.
    long
    countByRun(ExperimentRun run)
     
    long
    countItemsWithAllEvalsPassed(ExperimentRun run)
     
    long
    countPricedItemsByRun(ExperimentRun run)
    Counts the run's items that carry a non-null cost, i.e. the items that contributed to sumCostByRun(ExperimentRun).
    long
    countTokenizedItemsByRun(ExperimentRun run)
    Counts the run's items that carry a non-null prompt-token count, used as the denominator of the cost-coverage signal: an item with tokens but no cost is one a PriceTable could not price (unknown model or null token count), which is exactly the gap the signal reports.
    List<ItemResult>
    findAllWithEvalsByIdIn(Collection<UUID> ids)
    Loads the given items with their eval results fetch-joined, used to initialize the lazy evalResults collection on a page of items in one query rather than per row.
    List<ItemResult>
    findByRunIdWithEvals(UUID runId)
    Loads all item results for a run with their eval results fetch-joined in one query.
    org.springframework.data.domain.Page<ItemResult>
    findByRunOrderByCreatedAtAsc(ExperimentRun run, org.springframework.data.domain.Pageable pageable)
     
    List<ItemResult>
    findByRunWithEvals(ExperimentRun run)
    Loads all item results for a run with their eval results and dataset-item link fetch-joined in one query, avoiding the N+1 the lazy evalResults collection would otherwise trigger when building a core RunResult for the gate comparison.
    org.springframework.data.domain.Page<ItemResult>
    findItemsNeedingReview(String projectName, UUID experimentId, UUID runId, boolean restricted, String tenantId, org.springframework.data.domain.Pageable pageable)
    Page of items still awaiting a human verdict, optionally scoped to a project, experiment, or run.
    List<ItemResult>
    findItemsNotYetEvaluated(UUID runId, String evaluatorName, UUID afterId, org.springframework.data.domain.Pageable pageable)
    Seek-based page of run items that carry no eval result for the given evaluator yet.
    Double
    sumCostByRun(ExperimentRun run)
    Sums the call cost across a run's items.
    Long
    sumTokensInByRun(ExperimentRun run)
    Sums the prompt tokens across a run's items.
    Long
    sumTokensOutByRun(ExperimentRun run)
    Sums the completion tokens across a run's items.

    Methods inherited from interface org.springframework.data.repository.CrudRepository

    count, delete, deleteAll, deleteAll, deleteAllById, deleteById, existsById, findById, save

    Methods inherited from interface org.springframework.data.jpa.repository.JpaRepository

    deleteAllByIdInBatch, deleteAllInBatch, deleteAllInBatch, deleteInBatch, findAll, findAll, flush, getById, getOne, getReferenceById, saveAllAndFlush, saveAndFlush

    Methods inherited from interface org.springframework.data.repository.ListCrudRepository

    findAll, findAllById, saveAll

    Methods inherited from interface org.springframework.data.repository.ListPagingAndSortingRepository

    findAll

    Methods inherited from interface org.springframework.data.repository.PagingAndSortingRepository

    findAll

    Methods inherited from interface org.springframework.data.repository.query.QueryByExampleExecutor

    count, exists, findAll, findBy, findOne

  • Method Details

    • findByRunOrderByCreatedAtAsc

      org.springframework.data.domain.Page<ItemResult> findByRunOrderByCreatedAtAsc(ExperimentRun run, org.springframework.data.domain.Pageable pageable)

    • countByRun

      long countByRun(ExperimentRun run)

    • findByRunWithEvals

      @Query("SELECT DISTINCT i FROM ItemResult i\nLEFT JOIN FETCH i.evalResults\nLEFT JOIN FETCH i.datasetItem\nWHERE i.run = :run\nORDER BY i.createdAt ASC, i.id ASC\n") List<ItemResult> findByRunWithEvals(ExperimentRun run)
      Loads all item results for a run with their eval results and dataset-item link fetch-joined in one query, avoiding the N+1 the lazy evalResults collection would otherwise trigger when building a core RunResult for the gate comparison. Ordered by createdAt for a stable positional pairing fallback.

    • countItemsWithAllEvalsPassed

      @Query("SELECT COUNT(i) FROM ItemResult i\nWHERE i.run = :run\nAND NOT EXISTS (SELECT e FROM EvalResult e WHERE e.itemResult = i AND e.success = false)\nAND EXISTS (SELECT e FROM EvalResult e WHERE e.itemResult = i)\n") long countItemsWithAllEvalsPassed(ExperimentRun run)

    • sumTokensInByRun

      @Query("SELECT SUM(i.tokensIn) FROM ItemResult i WHERE i.run = :run") Long sumTokensInByRun(ExperimentRun run)
      Sums the prompt tokens across a run's items.
      Parameters:
      run - the run to aggregate
      Returns:
      the total prompt tokens, or null if no item carries a token count

    • sumTokensOutByRun

      @Query("SELECT SUM(i.tokensOut) FROM ItemResult i WHERE i.run = :run") Long sumTokensOutByRun(ExperimentRun run)
      Sums the completion tokens across a run's items.
      Parameters:
      run - the run to aggregate
      Returns:
      the total completion tokens, or null if no item carries a token count

    • sumCostByRun

      @Query("SELECT SUM(i.costUsd) FROM ItemResult i WHERE i.run = :run") Double sumCostByRun(ExperimentRun run)
      Sums the call cost across a run's items.
      Parameters:
      run - the run to aggregate
      Returns:
      the total cost in USD, or null if no item carries a cost

    • avgLatencyByRun

      @Query("SELECT AVG(i.latencyMs) FROM ItemResult i WHERE i.run = :run") Double avgLatencyByRun(ExperimentRun run)
      Averages the call latency across a run's items.
      Parameters:
      run - the run to aggregate
      Returns:
      the mean latency in milliseconds, or null if no item carries a latency

    • countPricedItemsByRun

      @Query("SELECT COUNT(i) FROM ItemResult i WHERE i.run = :run AND i.costUsd IS NOT NULL") long countPricedItemsByRun(ExperimentRun run)
      Counts the run's items that carry a non-null cost, i.e. the items that contributed to sumCostByRun(ExperimentRun). Read together with countTokenizedItemsByRun(ExperimentRun) this is the numerator of the cost-coverage signal: when fewer items are priced than tokenized, sumCostByRun omits the unpriced items and the surfaced total understates the true cost rather than failing.
      Parameters:
      run - the run to aggregate
      Returns:
      the number of items with a non-null costUsd; zero if none

    • countTokenizedItemsByRun

      @Query("SELECT COUNT(i) FROM ItemResult i WHERE i.run = :run AND i.tokensIn IS NOT NULL") long countTokenizedItemsByRun(ExperimentRun run)
      Counts the run's items that carry a non-null prompt-token count, used as the denominator of the cost-coverage signal: an item with tokens but no cost is one a PriceTable could not price (unknown model or null token count), which is exactly the gap the signal reports. An item with no tokens at all was never measured (a plain .task) and counts toward neither this nor countPricedItemsByRun(ExperimentRun).
      Parameters:
      run - the run to aggregate
      Returns:
      the number of items with a non-null tokensIn; zero if none

    • findItemsNotYetEvaluated

      @Query("SELECT i FROM ItemResult i\nWHERE i.run.id = :runId\nAND i.id > :afterId\nAND NOT EXISTS (\n SELECT e FROM EvalResult e\n WHERE e.itemResult = i AND e.evaluatorName = :evaluatorName\n)\nORDER BY i.id ASC\n") List<ItemResult> findItemsNotYetEvaluated(@Param("runId") UUID runId, @Param("evaluatorName") String evaluatorName, @Param("afterId") UUID afterId, org.springframework.data.domain.Pageable pageable)
      Seek-based page of run items that carry no eval result for the given evaluator yet. Items are ordered by id so the worker can page forward by passing the last id it saw as afterId; the NOT EXISTS filter is scoped by evaluator name so adding an evaluator to a previously scored run still picks up every item for that evaluator. Pass the all-zero UUID for the first page.
      Parameters:
      runId - the run whose items to scan
      evaluatorName - the evaluator whose results gate the scan
      afterId - the seek cursor; rows with a strictly greater id are returned
      pageable - a one-page request bounding the result size
      Returns:
      the next page of unevaluated items, ordered by id

    • findByRunIdWithEvals

      @Query("SELECT DISTINCT i FROM ItemResult i\nLEFT JOIN FETCH i.evalResults\nWHERE i.run.id = :runId\nORDER BY i.createdAt ASC, i.id ASC\n") List<ItemResult> findByRunIdWithEvals(@Param("runId") UUID runId)
      Loads all item results for a run with their eval results fetch-joined in one query. Annotations are deliberately not fetch-joined here: each item has at most one annotation but many eval results, so joining both would multiply rows in a Cartesian product. Callers that also need annotations batch-load them separately by item id.
      Parameters:
      runId - the run whose items to load
      Returns:
      the run's item results, each with its eval results initialized

    • findItemsNeedingReview

      @Query(value="SELECT i FROM ItemResult i\nJOIN FETCH i.run r\nJOIN FETCH r.experiment e\nJOIN FETCH e.project p\nWHERE (:projectName IS NULL OR p.name = :projectName)\nAND (:experimentId IS NULL OR e.id = :experimentId)\nAND (:runId IS NULL OR r.id = :runId)\nAND (:restricted = false OR i.tenantId = :tenantId OR i.tenantId IS NULL)\nAND NOT EXISTS (\n SELECT a FROM Annotation a\n WHERE a.itemResult = i AND a.verdict <> dev.dokimos.server.entity.AnnotationVerdict.UNSURE\n)\nORDER BY i.createdAt ASC, i.id ASC\n", countQuery="SELECT COUNT(i) FROM ItemResult i\nWHERE (:projectName IS NULL OR i.run.experiment.project.name = :projectName)\nAND (:experimentId IS NULL OR i.run.experiment.id = :experimentId)\nAND (:runId IS NULL OR i.run.id = :runId)\nAND (:restricted = false OR i.tenantId = :tenantId OR i.tenantId IS NULL)\nAND NOT EXISTS (\n SELECT a FROM Annotation a\n WHERE a.itemResult = i AND a.verdict <> dev.dokimos.server.entity.AnnotationVerdict.UNSURE\n)\n") org.springframework.data.domain.Page<ItemResult> findItemsNeedingReview(@Param("projectName") String projectName, @Param("experimentId") UUID experimentId, @Param("runId") UUID runId, @Param("restricted") boolean restricted, @Param("tenantId") String tenantId, org.springframework.data.domain.Pageable pageable)
      Page of items still awaiting a human verdict, optionally scoped to a project, experiment, or run. An item needs review when it carries no annotation or only an UNSURE one: the NOT EXISTS clause treats a CORRECT/INCORRECT annotation as resolved (there is at most one annotation per item). Run, experiment, and project are fetch-joined so the queue can render each item's context without a per-row lookup, and the null-guarded filters let the one query serve both the global queue and the scoped views. The tenant predicate filters on the item's own stamped tenant so a reviewer only sees items of their own tenant plus shared (null-tenant) items; it is applied to both the page query and the count query so paging metadata stays consistent.
      Parameters:
      projectName - restrict to this project, or null for any
      experimentId - restrict to this experiment, or null for any
      runId - restrict to this run, or null for any
      restricted - whether the tenant predicate applies; false sees every tenant
      tenantId - the tenant to filter by when restricted, or null for shared-only
      pageable - the page to return, ordered oldest-first
      Returns:
      the matching items, each with run, experiment, and project initialized

    • findAllWithEvalsByIdIn

      @Query("SELECT DISTINCT i FROM ItemResult i LEFT JOIN FETCH i.evalResults WHERE i.id IN :ids") List<ItemResult> findAllWithEvalsByIdIn(@Param("ids") Collection<UUID> ids)
      Loads the given items with their eval results fetch-joined, used to initialize the lazy evalResults collection on a page of items in one query rather than per row.
      Parameters:
      ids - the item ids to load
      Returns:
      the items with eval results initialized