Package dev.dokimos.langchain4j

Class LangChain4jSupport

java.lang.Object
dev.dokimos.langchain4j.LangChain4jSupport
public final class LangChain4jSupport extends Object
Utilities for integrating with LangChain4j.

This class provides factory methods to create Tasks and JudgeLMs from LangChain4j components.

RAG Evaluation


 // 1. Define your AiService to return Result<String>
 interface Assistant {
     Result<String> chat(String userMessage);
 }

 // 2. Build your assistant
 Assistant assistant = AiServices.builder(Assistant.class)
     .chatModel(chatModel)
     .retrievalAugmentor(DefaultRetrievalAugmentor.builder()
         .queryTransformer(compressingQueryTransformer)
         .contentRetriever(retriever)
         .contentAggregator(reRankingAggregator)
         .build())
     .build();

 // 3. Create a Task for evaluation
 Task task = LangChain4jSupport.ragTask(assistant::chat);

 // 4. Run evaluation with some metrics
 Experiment.builder()
     .task(task)
     .evaluators(List.of(faithfulness, contextRelevancy))
     .build()
     .run();

  • Field Details

    • OUTPUT_KEY

      public static final String OUTPUT_KEY
      Default key for the model output in evaluation results.
      See Also:

    • CONTEXT_KEY

      public static final String CONTEXT_KEY
      Default key for additional context in evaluation results.
      See Also:

    • INPUT_KEY

      public static final String INPUT_KEY
      Default key for reading input from dataset examples.
      See Also:

  • Method Details

    • asJudge

      public static JudgeLM asJudge(dev.langchain4j.model.chat.ChatModel model)
      Creates a JudgeLM from a LangChain4j ChatModel.

      Use this to create judges for LLM-based evaluators like LLMJudgeEvaluator, FaithfulnessEvaluator, etc.

      Example: 

      
 ChatModel gemini = VertexAiGeminiChatModel.builder()...build();
 JudgeLM judge = LangChain4jSupport.asJudge(gemini);

 var evaluator = LLMJudgeEvaluator.builder()
     .judge(judge)
     .criteria("Is the response helpful?")
     .build();
      Parameters:
      model - the ChatModel to use as judge
      Returns:
      a JudgeLM that delegates to the ChatModel

    • simpleTask

      public static Task simpleTask(dev.langchain4j.model.chat.ChatModel model)
      Creates a simple Task for Q&A evaluation.

      The task reads "input" from the example and returns a Map with "output".

      Example: 

      
 ChatModel model = OpenAiChatModel.builder()...build();
 Task task = LangChain4jSupport.simpleTask(model);

 // Dataset examples just need "input"
 Example example = Example.of("What is 2+2?", "4");
      Parameters:
      model - the ChatModel to evaluate
      Returns:
      a Task suitable for the Experiment

    • ragTask

      public static Task ragTask(Function<String,dev.langchain4j.service.Result<String>> assistantCall)
      Creates a RAG evaluation Task from a function that returns Result.

      This is the primary integration point for RAG evaluation. LangChain4j's Result class already contains the retrieved sources via result.sources().

      Example: 

      
 interface Assistant {
     Result<String> chat(String userMessage);
 }

 Assistant assistant = AiServices.builder(Assistant.class)
     .chatModel(chatModel)
     .retrievalAugmentor(retrievalAugmentor)
     .build();

 Task task = LangChain4jSupport.ragTask(assistant::chat);
      Parameters:
      assistantCall - a function that takes the input string and returns a Result
      Returns:
      a Task suitable for evaluation

    • ragTask

      public static Task ragTask(Function<String,dev.langchain4j.service.Result<String>> assistantCall, String inputKey, String outputKey, String contextKey)
      Creates a RAG evaluation Task with custom key names.

      Use this when your dataset or evaluators expect different keys.

      Example: 

      
 // Dataset uses "question" instead of "input"
 Task task = LangChain4jSupport.ragTask(
     assistant::chat,
     "question",        // input key
     "answer",          // output key
     "retrievalContext" // context key
 );
      Parameters:
      assistantCall - a function that takes the input string and returns a Result
      inputKey - the key to read from example inputs
      outputKey - the key for the output in the result map
      contextKey - the key for the retrieval context in the result map
      Returns:
      a Task suitable for RAG evaluation

    • customTask

      public static Task customTask(Task taskFunction)
      Creates a flexible Task that allows full control over output mapping.

      Use this for complex scenarios where you want to capture additional data beyond what the standard RAG task implementation provides.

      Example: 

      
 Task task = LangChain4jSupport.customTask(example -> {
     String query = example.input();

     // Track the latency
     long start = System.currentTimeMillis();
     Result<String> result = assistant.chat(query);
     long duration = System.currentTimeMillis() - start;

     return Map.of(
         "output", result.content(),
         "context", LangChain4jSupport.extractTexts(result.sources()),
         "latencyMs", duration,
         "sourceCount", result.sources().size()
     );
 });
      Parameters:
      taskFunction - a function that takes an Example and returns outputs
      Returns:
      a Task suitable for Experiment

    • extractTexts

      public static List<String> extractTexts(List<dev.langchain4j.rag.content.Content> contents)
      Extracts text content from a list of LangChain4j Content objects.

      This is useful when building custom Tasks.

      Parameters:
      contents - the list of Content from result.sources()
      Returns:
      list of text strings, empty list if contents is null

    • extractTextsWithMetadata

      public static List<Map<String,Object>> extractTextsWithMetadata(List<dev.langchain4j.rag.content.Content> contents)
      Extracts text content with metadata from a list of LangChain4j Content objects.

      Returns a list of maps, where each map contains:

      • text - the segment text
      • metadata - the segment metadata as a map

      This is useful when you need source attribution in evaluations.

      Parameters:
      contents - the list of Content from result.sources()
      Returns:
      list of maps containing text and metadata