Interface LLMClient

All Superinterfaces:
Client
All Known Implementing Classes:
AbstractLLMClient, AbstractOpenAIChatClient, AnthropicBedrockMantleSDKClient, AnthropicBedrockSDKClient, AnthropicSDKClient, AzureFoundryOpenAIClient, BedrockConverseClient, OpenAICompatibleClient, OpenAISDKClient
public interface LLMClient extends Client
Interface for LLM client implementations across different providers. Provides unified API for text generation with structured output support.
Since:
0.1 (2025-07-18)
Author:
Andrey Santrosyan

  • Method Details

    • singleResponse

      <T> LLMResponse<T> singleResponse(LLMRequest<T> request) throws TokenEstimateExceedsLimitException
      Gets a structured response from the LLM with retry logic and validation. Returns an LLMResponse with successful=false if the request fails.
      Parameters:
      request - The LLM request with prompt, retry settings, and optional JSON validation
      Returns:
      The LLM response with metadata, tokens used, and parsed response
      Throws:
      TokenEstimateExceedsLimitException - if pre-flight estimate exceeds model limits

    • getTemperature

      Double getTemperature()
      The caller-set sampling temperature, or null when none was set and the provider's own default applies.

    • setTemperature

      void setTemperature(Double temperature)
      Sets the sampling temperature this client's requests carry. Unset (null, the default), no temperature reaches the wire and the provider's own default applies - which is also what keeps the models that accept no sampling parameter callable. A temperature set on a model that rejects it comes back as the provider's own typed refusal.

    • getModelIdentifier

      String getModelIdentifier()

    • createOutgoingMessage

      <T> OutgoingMessage<T> createOutgoingMessage(ResponseHandler<T> responseHandler)
      Creates a new conversation message appropriate for this client. Different providers may return different message implementations (e.g., AnthropicMessage, OpenAIMessage) with provider-specific validation.
      Returns:
      a new empty conversation message

    • streamResponse

      default <T> LLMResponse<T> streamResponse(LLMRequest<T> request, Consumer<StreamChunk> chunkHandler) throws TokenEstimateExceedsLimitException
      Streams response chunks from the LLM and returns the full response record. The chunkHandler receives each delta as it arrives; the returned LLMResponse carries token usage, cache counters, stop reason, provider, and other fields populated by the client.
      Returns:
      the populated response (check LLMResponse.isSuccessful())
      Throws:
      TokenEstimateExceedsLimitException - if pre-flight estimate exceeds model limits

    • close

      default void close()
      Releases resources held by this client (e.g., HTTP connection pools). Called during application shutdown for shared clients.

    • getFormatter

      ContentFormatter getFormatter()
      Gets the content formatter for this client. Formatters handle provider-specific content encoding.
      Returns:
      the content formatter

    • getDialect

      APIDialect getDialect()
      Gets the API dialect for this client. Used to determine encoding strategies and compatibility.
      Returns:
      the API dialect