Package dev.dokimos.embabel

Class EmbabelSupport

java.lang.Object
dev.dokimos.embabel.EmbabelSupport
public final class EmbabelSupport extends Object
Entry point for evaluating Embabel agent runs with Dokimos.

Embabel reports tool calls only through event callbacks during a run, so the integration is built around a stateful EmbabelTraceCollector (an AgenticEventListener) plus the installers on this class. The flow is: attach a collector to the run's options, execute the agent, then read the trace. 


 EmbabelTraceCollector collector = new EmbabelTraceCollector();
 ProcessOptions options = EmbabelSupport.attach(new ProcessOptions(), collector);

 AgentInvocation<MyResult> inv = AgentInvocation.builder(platform)
         .options(options)
         .build(MyResult.class);
 inv.invoke(userInput);

 AgentTrace trace = collector.trace();
 EvalTestCase tc = trace.toTestCase(userInput, EmbabelSupport.toToolDefinitions(collector));

  • Method Details

    • attach

      public static com.embabel.agent.core.ProcessOptions attach(com.embabel.agent.core.ProcessOptions options, EmbabelTraceCollector collector)
      Registers the given collector as a listener on a copy of the supplied ProcessOptions.

      ProcessOptions is immutable: ProcessOptions.withListener(com.embabel.agent.api.event.AgenticEventListener) returns a new instance. Use the returned options when building the AgentInvocation so the collector receives the run's events.

      Parameters:
      options - the base process options to copy, never null
      collector - the collector to register, never null
      Returns:
      a new ProcessOptions with the collector attached
      Throws:
      IllegalArgumentException - if options or collector is null

    • attach

      public static EmbabelTraceCollector attach(com.embabel.agent.api.invocation.AgentInvocation.Builder invocationBuilder)
      Creates a fresh collector and registers it on a default ProcessOptions.

      Read the resulting options off the returned collector is not possible directly; prefer attach(ProcessOptions, EmbabelTraceCollector) when you need to keep the options. This convenience overload registers the collector on the AgentInvocation.Builder instead and returns the collector for EmbabelTraceCollector.trace() after the run.

      Parameters:
      invocationBuilder - the invocation builder to configure, never null
      Returns:
      the registered collector
      Throws:
      IllegalArgumentException - if invocationBuilder is null

    • toToolDefinitions

      public static List<ToolDefinition> toToolDefinitions(EmbabelTraceCollector collector)
      Synthesizes ToolDefinitions from the tool names observed during a run.

      Embabel's event stream does not carry the tools' JSON schemas, so each synthesized definition has an empty input schema and an empty description. This is sufficient for evaluators that key off tool names (for example ToolCallValidityEvaluator) but weakens ToolDescriptionReliabilityEvaluator, which has no description to assess. If you have the original tool contracts, build ToolDefinitions from those instead. (Decision O2.)

      Parameters:
      collector - the collector that observed the run, never null
      Returns:
      one tool definition per observed tool name, with an empty schema
      Throws:
      IllegalArgumentException - if collector is null