Package dev.dokimos.mcp

Class ToolHandlers

java.lang.Object
dev.dokimos.mcp.ToolHandlers
public class ToolHandlers extends Object
Implements the four MCP tool handlers for the dokimos evaluation framework.

  • Constructor Details

    • ToolHandlers

      public ToolHandlers(ResultStore store, com.fasterxml.jackson.databind.ObjectMapper json)

  • Method Details

    • handleRunEvaluation

      public io.modelcontextprotocol.spec.McpSchema.CallToolResult handleRunEvaluation(Map<String,Object> arguments)
      Runs an evaluation: loads a dataset, calls an LLM, evaluates results, and persists the run.

    • handleListExperiments

      public io.modelcontextprotocol.spec.McpSchema.CallToolResult handleListExperiments(Map<String,Object> arguments)
      Lists past evaluation runs with optional filtering.

    • handleCompareRuns

      public io.modelcontextprotocol.spec.McpSchema.CallToolResult handleCompareRuns(Map<String,Object> arguments)
      Compares two runs via RunComparison: per-evaluator deltas, pass-rate test, per-item diffs.

    • handleGetFailingQueries

      public io.modelcontextprotocol.spec.McpSchema.CallToolResult handleGetFailingQueries(Map<String,Object> arguments)
      Returns failing queries from a run, filtered by score threshold.