Package dev.dokimos.core

Class Assertions

java.lang.Object
dev.dokimos.core.Assertions
public class Assertions extends Object
Assertion utilities for evaluation-based testing.

assertEval(dev.dokimos.core.EvalTestCase, java.util.List<dev.dokimos.core.Evaluator>) checks a single test case against its evaluators and throws AssertionError when a score falls below its threshold. assertNoRegression(dev.dokimos.core.ExperimentResult) compares an experiment result against a committed baseline and throws on a regression on every runner (CI and local alike), so a failing test is the gate without CI-specific wiring.

  • Method Details

    • assertEval

      public static void assertEval(EvalTestCase testCase, List<Evaluator> evaluators)
      Asserts that the test case passes all evaluators.
      Parameters:
      testCase - the test case to evaluate
      evaluators - the evaluators to run
      Throws:
      AssertionError - if any evaluator score falls below its threshold

    • assertEval

      public static void assertEval(EvalTestCase testCase, Evaluator... evaluators)
      Asserts that the test case passes all evaluators.
      Parameters:
      testCase - the test case to evaluate
      evaluators - the evaluators to run
      Throws:
      AssertionError - if any evaluator score falls below its threshold

    • assertNoRegression

      public static void assertNoRegression(ExperimentResult candidate)
      Asserts that the candidate has not regressed against its committed baseline, resolving the baseline from the experiment name.

      The baseline lives at src/test/resources/dokimos/baselines/<name>.json, resolved relative to the module directory. With no baseline yet, the first local run writes one and passes, so the new file can be reviewed and committed; set GateConfig.Builder.bootstrapPasses(boolean) to false to fail that first run instead, until the baseline is committed. A CI run never writes a baseline (the checkout is ephemeral) and reports NO_BASELINE. To accept an intended change, re-run with DOKIMOS_UPDATE_BASELINE=true (or -Ddokimos.updateBaseline) to overwrite the baseline.

      Parameters:
      candidate - the candidate experiment result
      Throws:
      IllegalArgumentException - if the experiment name is blank or the default "unnamed" (two unnamed experiments would collide on one baseline); name the experiment or pass a Path
      AssertionError - on a regression, or on the first local baseline write when bootstrapPasses is false

    • assertNoRegression

      public static void assertNoRegression(ExperimentResult candidate, String baselineName)
      Asserts that the candidate has not regressed against the baseline named baselineName.
      Parameters:
      candidate - the candidate experiment result
      baselineName - the logical baseline name (resolved to src/test/resources/dokimos/baselines/<name>.json)
      Throws:
      IllegalArgumentException - if baselineName is blank or "unnamed"
      AssertionError - on a regression, or on the first local baseline write when bootstrapPasses is false

    • assertNoRegression

      public static void assertNoRegression(ExperimentResult candidate, String baselineName, GateConfig config)
      Asserts that the candidate has not regressed against the baseline named baselineName, using the supplied configuration.
      Parameters:
      candidate - the candidate experiment result
      baselineName - the logical baseline name (resolved to src/test/resources/dokimos/baselines/<name>.json)
      config - the gate configuration
      Throws:
      IllegalArgumentException - if baselineName is blank or "unnamed"
      AssertionError - on a regression, or on the first local baseline write when bootstrapPasses is false

    • assertNoRegression

      public static void assertNoRegression(ExperimentResult candidate, Path baseline)
      Asserts that the candidate has not regressed against the baseline at the given path.
      Parameters:
      candidate - the candidate experiment result
      baseline - the baseline file path
      Throws:
      AssertionError - on a regression, or on the first local baseline write when bootstrapPasses is false

    • assertNoRegression

      public static void assertNoRegression(ExperimentResult candidate, Path baseline, GateConfig config)
      Asserts that the candidate has not regressed against the baseline at the given path, using the supplied configuration.
      Parameters:
      candidate - the candidate experiment result
      baseline - the baseline file path
      config - the gate configuration
      Throws:
      AssertionError - on a regression, or on the first local baseline write when bootstrapPasses is false