Package dev.dokimos.core.conversation

Class GoldenGenerator

java.lang.Object
dev.dokimos.core.conversation.GoldenGenerator
public class GoldenGenerator extends Object
Turns ScenarioSeeds into a Dataset of synthetic multi-turn goldens by running each seed through ConversationSimulator against the application under test.

Each seed yields one example: the rendered transcript under inputs["input"], the scenario and turn count in metadata, and a golden-<index> dataset item id. The result is serialized with toJson() or toJsonl() into the shape Dataset.fromJson and Dataset.fromJsonl read, so generated goldens feed @DatasetSource directly.

Generation is stateless: every call to generate() re-runs the seeds.

Example usage: 


 GoldenGenerator generator = GoldenGenerator.builder()
         .application(myApp)
         .name("support-goldens")
         .seed(ScenarioSeed.scripted("Return request", List.of("I want a refund", "Order #123")))
         .build();

 generator.write(Path.of("src/test/resources/datasets/support-goldens.json"));

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    GoldenGenerator.Builder
    Builder for constructing golden generators.

  • Method Summary

    Modifier and Type
    Method
    Description
    static GoldenGenerator.Builder
    builder()
    Creates a new builder for constructing a golden generator.
    Dataset
    generate()
    Runs every seed and collects the resulting conversations into a dataset.
    String
    toJson()
    Generates the goldens and serializes them as pretty-printed JSON in the dataset format.
    String
    toJsonl()
    Generates the goldens and serializes them as JSONL, one example per line.
    void
    write(Path path)
    Generates the goldens and writes them to a JSON file, creating parent directories as needed.
    void
    writeJsonl(Path path)
    Generates the goldens and writes them to a JSONL file, creating parent directories as needed.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • builder

      public static GoldenGenerator.Builder builder()
      Creates a new builder for constructing a golden generator.
      Returns:
      a new builder

    • generate

      public Dataset generate()
      Runs every seed and collects the resulting conversations into a dataset.

      A seed that fails still yields an example, carrying error and errorSource in its metadata. A failure during the conversation keeps the turns gathered so far; a failure before it starts yields an example with no turns.

      Returns:
      the generated dataset, one example per seed, in seed order
      Throws:
      IllegalStateException - if a persona-driven seed is present and no judge was supplied

    • toJson

      public String toJson()
      Generates the goldens and serializes them as pretty-printed JSON in the dataset format.
      Returns:
      the dataset as JSON

    • toJsonl

      public String toJsonl()
      Generates the goldens and serializes them as JSONL, one example per line.
      Returns:
      the examples as JSONL

    • write

      public void write(Path path)
      Generates the goldens and writes them to a JSON file, creating parent directories as needed.
      Parameters:
      path - the file to write
      Throws:
      UncheckedIOException - if writing fails

    • writeJsonl

      public void writeJsonl(Path path)
      Generates the goldens and writes them to a JSONL file, creating parent directories as needed.
      Parameters:
      path - the file to write
      Throws:
      UncheckedIOException - if writing fails