Package dev.dokimos.core.conversation

Record Class ConversationTrajectory

java.lang.Object
java.lang.Record
dev.dokimos.core.conversation.ConversationTrajectory
Record Components:
messages - the list of messages in chronological order
scenario - a description of the test scenario
metadata - additional metadata about the conversation
public record ConversationTrajectory(List<Message> messages, String scenario, Map<String,Object> metadata) extends Record
Represents a complete conversation trajectory between a simulated user and an application.

A trajectory captures the full history of messages exchanged during a multi-turn conversation, along with scenario information and metadata for evaluation.

  • Constructor Details

    • ConversationTrajectory

      public ConversationTrajectory(List<Message> messages, String scenario, Map<String,Object> metadata)
      Compact constructor ensuring immutability.

  • Method Details

    • empty

      public static ConversationTrajectory empty()
      Creates an empty trajectory with no messages.
      Returns:
      an empty trajectory

    • builder

      public static ConversationTrajectory.Builder builder()
      Creates a new builder for constructing trajectories.
      Returns:
      a new builder

    • turnCount

      public int turnCount()
      Returns the number of conversation turns (user-assistant message pairs).

      A turn is counted as a user message followed by an assistant response.

      Returns:
      the number of complete turns

    • userMessages

      public List<Message> userMessages()
      Returns only the user messages from the conversation.
      Returns:
      list of user messages

    • assistantMessages

      public List<Message> assistantMessages()
      Returns only the assistant messages from the conversation.
      Returns:
      list of assistant messages

    • systemMessages

      public List<Message> systemMessages()
      Returns only the system messages from the conversation.
      Returns:
      list of system messages

    • lastMessage

      public Message lastMessage()
      Returns the last message in the conversation, if any.
      Returns:
      the last message or null if empty

    • lastUserMessage

      public Message lastUserMessage()
      Returns the last user message in the conversation, if any.
      Returns:
      the last user message or null if none

    • lastAssistantMessage

      public Message lastAssistantMessage()
      Returns the last assistant message in the conversation, if any.
      Returns:
      the last assistant message or null if none

    • toolCalls

      public List<ToolCall> toolCalls()
      Flattens every assistant turn's tool calls into one list, in chronological order. This is the flat List<ToolCall> the agent tool-call evaluators read.
      Returns:
      all tool calls across the conversation, never null

    • toolCallsByTurn

      public List<List<ToolCall>> toolCallsByTurn()
      Groups tool calls per assistant turn, in order: one inner list per assistant message (each possibly empty). This is the per-turn view for scoring each turn against its own expected calls. The unit of grouping is an assistant message, which can differ from turnCount() (which counts user/assistant pairs) when a conversation has consecutive or trailing assistant messages.
      Returns:
      tool calls grouped by assistant message, never null

    • toAgentTrace

      public AgentTrace toAgentTrace()
      Collapses the conversation into a single AgentTrace: the final response is the last assistant message's content (or empty), the tool calls are toolCalls(), reasoning is empty, and metadata carries the scenario and turn count. Intended for deterministic, scripted conversations; for non-deterministic simulated runs, prefer per-turn evaluation over toolCallsByTurn().
      Returns:
      the collapsed agent trace

    • toAgentOutputs

      public Map<String,Object> toAgentOutputs()
      Builds the actual-outputs map the agent evaluators read, identical in shape to AgentTrace.toOutputMap().
      Returns:
      the actual outputs map

    • toTestCase

      public EvalTestCase toTestCase()
      Builds a test case for the deterministic tool-call evaluators, using the last user message as input.
      Returns:
      a new test case

    • toTestCase

      public EvalTestCase toTestCase(List<ToolDefinition> tools)
      Builds a test case for the deterministic tool-call and tool-definition evaluators, adding the tools the agent could call.
      Parameters:
      tools - the available tool definitions
      Returns:
      a new test case

    • toTestCase

      public EvalTestCase toTestCase(List<ToolDefinition> tools, List<String> tasks)
      Builds a test case for the judge-based evaluators (TaskCompletionEvaluator, ToolArgumentHallucinationEvaluator). The input is the full rendered transcript so the judge reasons over the whole conversation, but tool calls are rendered name-only ([tool: name], not [tool: name(args)]): the hallucination evaluator grounds against this input, so the argument values under test must not appear in it. The arguments stay available through actualOutputs["toolCalls"].
      Parameters:
      tools - the available tool definitions
      tasks - the tasks the user asked the agent to complete
      Returns:
      a new test case

    • withMessage

      public ConversationTrajectory withMessage(Message message)
      Creates a new trajectory with an additional message appended.
      Parameters:
      message - the message to append
      Returns:
      a new trajectory with the message added

    • isEmpty

      public boolean isEmpty()
      Checks if the conversation is empty.
      Returns:
      true if there are no messages

    • toText

      public String toText()
      Formats the conversation as a simple text transcript.
      Returns:
      the conversation as text

    • toJson

      public String toJson()
      Serializes the trajectory to JSON for debugging and logging.
      Returns:
      JSON representation of the trajectory

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with Objects::equals(Object,Object).
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • messages

      public List<Message> messages()
      Returns the value of the messages record component.
      Returns:
      the value of the messages record component

    • scenario

      public String scenario()
      Returns the value of the scenario record component.
      Returns:
      the value of the scenario record component

    • metadata

      public Map<String,Object> metadata()
      Returns the value of the metadata record component.
      Returns:
      the value of the metadata record component