Package dev.dokimos.junit

Annotation Interface DatasetSource

@Target(METHOD) @Retention(RUNTIME) @ArgumentsSource(DatasetArgumentsProvider.class) @ExtendWith(DatasetRunExtension.class) public @interface DatasetSource
Provides Examples from a Dataset as arguments to a parameterized test.

Supports multiple source types:

  • classpath:datasets/dataset.json - classpath JSON resource
  • classpath:datasets/dataset.jsonl - classpath JSONL resource
  • file:path/to/dataset.json - file path
  • path/to/dataset.json - file path (default)
  • Inline JSON via json()
  • Inline JSONL via jsonl()

Example usage: 


 @ParameterizedTest
 @DatasetSource("classpath:datasets/qa.json")
 void testQa(Example example) {
     String answer = aiService.generate(example.input());
     Assertions.assertEval(example.toTestCase(answer), evaluators);
 }

  • Optional Element Summary

    Optional Elements
    Modifier and Type
    Optional Element
    Description
    MetadataEntry[]
    entries
    Run metadata as typed key-value entries (for example {@MetadataEntry(key = "model", value = "gpt-4")}).
    String
    experimentName
    Name of the run opened when a DatasetReporter field is present.
    String
    json
    Inline JSON dataset.
    String
    jsonl
    Inline JSONL dataset.
    String[]
    metadata
    Run metadata as alternating key-value pairs (for example {"model", "gpt-4", "temperature", "0"}).
    String
    value
    URI of the dataset to load.

  • Element Details

    • value

      String value
      URI of the dataset to load. Supports: classpath:, file:, or plain file paths.
      Default:
      ""

    • json

      String json
      Inline JSON dataset. Use this for small/quick tests only.
      Default:
      ""

    • jsonl

      String jsonl
      Inline JSONL dataset. Each line is a JSON object. Use this for small/quick tests only.
      Default:
      ""

    • experimentName

      String experimentName
      Name of the run opened when a DatasetReporter field is present. Defaults to the test method name when left empty.
      Default:
      ""

    • metadata

      String[] metadata
      Run metadata as alternating key-value pairs (for example {"model", "gpt-4", "temperature", "0"}). Forwarded to the reporter when a DatasetReporter field is present. Defaults to no metadata.
      Default:
      {}

    • entries

      MetadataEntry[] entries
      Run metadata as typed key-value entries (for example {@MetadataEntry(key = "model", value = "gpt-4")}). Forwarded to the reporter when a DatasetReporter field is present. When non-empty this takes precedence over metadata(). Defaults to no metadata.
      Default:
      {}