Skip to content

LanguageModel

The LanguageModel module provides AI text generation capabilities with tool calling support.

This module offers a comprehensive interface for interacting with large language models, supporting both streaming and non-streaming text generation, structured output generation, and tool calling functionality. It provides a unified API that can be implemented by different AI providers while maintaining type safety and effect management.

15 exports Added in v1.0.0 Source

Constructors

make

Added in v1.0.0 Source

Creates a LanguageModel service from provider-specific implementations.

This constructor takes provider-specific implementations for text generation and streaming text generation and returns a LanguageModel service.

Signature

declare const make: (params: ConstructorParams) => Effect.Effect<Service>;

Context

LanguageModel

Added in v1.0.0 Source

The LanguageModel service tag for dependency injection.

This tag provides access to language model functionality throughout your application, enabling text generation, streaming, and structured output capabilities.

Signature

declare class LanguageModel extends any {
  constructor();
}

Example

import { LanguageModel } from "@effect/ai"
import * as Effect from "effect/Effect"

const useLanguageModel = Effect.gen(function* () {
  const model = yield* LanguageModel.LanguageModel
  const response = yield* model.generateText({
    prompt: "What is machine learning?",
  })
  return response.text
})

Functions

generateObject

Added in v1.0.0 Source

Generate a structured object from a schema using a language model.

Signature

declare const generateObject: <
  A,
  I extends Record<string, unknown>,
  R,
  Options extends NoExcessProperties<GenerateObjectOptions<any, A, I, R>, Options>,
  Tools extends Record<string, Tool.Any> = {},
>(
  options: Options & GenerateObjectOptions<Tools, A, I, R>,
) => Effect.Effect<
  GenerateObjectResponse<Tools, A>,
  ExtractError<Options>,
  LanguageModel | R | ExtractContext<Options>
>;

Example

import { LanguageModel } from "@effect/ai"
import { Effect, Schema } from "effect"

const EventSchema = Schema.Struct({
  title: Schema.String,
  date: Schema.String,
  location: Schema.String,
})

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateObject({
    prompt: "Extract event info: Tech Conference on March 15th in San Francisco",
    schema: EventSchema,
    objectName: "event",
  })

  console.log(response.value)
  // { title: "Tech Conference", date: "March 15th", location: "San Francisco" }

  return response.value
})

generateText

Added in v1.0.0 Source

Generate text using a language model.

Signature

declare const generateText: <
  Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
  Tools extends Record<string, Tool.Any> = {},
>(
  options: Options & GenerateTextOptions<Tools>,
) => Effect.Effect<
  GenerateTextResponse<Tools>,
  ExtractError<Options>,
  LanguageModel | ExtractContext<Options>
>;

Example

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateText({
    prompt: "Write a haiku about programming",
    toolChoice: "none",
  })

  console.log(response.text)
  console.log(response.usage.totalTokens)

  return response
})

streamText

Added in v1.0.0 Source

Generate text using a language model with streaming output.

Returns a stream of response parts that are emitted as soon as they are available from the model, enabling real-time text generation experiences.

Signature

declare function streamText<
  Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
  Tools extends Record<string, Any> = {},
>(
  options: Options & GenerateTextOptions<Tools>,
): Stream<StreamPart<Tools>, ExtractError<Options>, LanguageModel | ExtractContext<Options>>;

Models

ConstructorParams interface

Added in v1.0.0 Source

Parameters required to construct a LanguageModel service.

Signature

interface ConstructorParams {
  readonly generateText: (
    options: ProviderOptions,
  ) => Effect<Array<PartEncoded>, AiError, IdGenerator>;
  readonly streamText: (
    options: ProviderOptions,
  ) => Stream<StreamPartEncoded, AiError, IdGenerator>;
}

GenerateObjectOptions interface

Added in v1.0.0 Source

Configuration options for structured object generation.

Signature

interface GenerateObjectOptions<
  Tools extends Record<string, Tool.Any>,
  A,
  I extends Record<string, unknown>,
  R,
> extends GenerateTextOptions<Tools> {
  readonly objectName?: string;
  readonly schema: any;
}

GenerateObjectResponse

Added in v1.0.0 Source

Response class for structured object generation operations.

Signature

declare class GenerateObjectResponse<Tools extends Record<string, Tool.Any>, A> extends GenerateTextResponse<Tools> {
  constructor<Tools extends Record<string, Any>, A>(value: A, content: Array<Part<Tools>>);
  readonly value: A;
}

Example

import { LanguageModel } from "@effect/ai"
import { Effect, Schema } from "effect"

const UserSchema = Schema.Struct({
  name: Schema.String,
  email: Schema.String,
})

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateObject({
    prompt: "Create user: John Doe, john@example.com",
    schema: UserSchema,
  })

  console.log(response.value) // { name: "John Doe", email: "john@example.com" }
  console.log(response.text) // Raw generated text

  return response.value
})

GenerateTextOptions interface

Added in v1.0.0 Source

Configuration options for text generation.

Signature

interface GenerateTextOptions<Tools extends Record<string, Tool.Any>> {
  readonly concurrency?: Concurrency;
  readonly disableToolCallResolution?: boolean;
  readonly prompt: RawInput;
  readonly toolChoice?: ToolChoice<
    { [Name in string | number | symbol]: Tools[Name]["name"] }[keyof Tools]
  >;
  readonly toolkit?: WithHandler<Tools> | Effect<WithHandler<Tools>, any, any>;
}

GenerateTextResponse

Added in v1.0.0 Source

Response class for text generation operations.

Contains the generated content and provides convenient accessors for extracting different types of response parts like text, tool calls, and usage information.

Signature

declare class GenerateTextResponse<Tools extends Record<string, Tool.Any>> {
  constructor<Tools extends Record<string, Any>>(content: Array<Part<Tools>>);
  readonly content: Array<Part<Tools>>;
  finishReason: ["stop", "length", "content-filter", "tool-calls", "error", "pause", "other", "unknown"];
  reasoning: Array<ReasoningPart>;
  reasoningText: string | undefined;
  text: string;
  toolCalls: Array<ToolCallParts<Tools>>;
  toolResults: Array<ToolResultParts<Tools>>;
  usage: Usage;
}

Example

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateText({
    prompt: "Explain photosynthesis",
  })

  console.log(response.text) // Generated text content
  console.log(response.finishReason) // "stop", "length", etc.
  console.log(response.usage) // Usage information

  return response
})

ProviderOptions interface

Added in v1.0.0 Source

Configuration options passed along to language model provider implementations.

This interface defines the normalized options that are passed to the underlying provider implementation, regardless of the specific provider being used.

Signature

interface ProviderOptions {
  readonly prompt: Prompt;
  readonly responseFormat: {
    readonly type: "text";
  } | {
    readonly objectName: string;
    readonly schema: Any;
    readonly type: "json";
  };
  readonly span: Span;
  readonly toolChoice: ToolChoice<any>;
  readonly tools: readonly Array<Any>;
}

Service interface

Added in v1.0.0 Source

The service interface for language model operations.

Defines the contract that all language model implementations must fulfill, providing text generation, structured output, and streaming capabilities.

Signature

interface Service {
  readonly generateObject: <
    A,
    I extends Record<string, unknown>,
    R,
    Options extends NoExcessProperties<GenerateObjectOptions<any, A, I, R>, Options>,
    Tools extends Record<string, Any> = {},
  >(
    options: Options & GenerateObjectOptions<Tools, A, I, R>,
  ) => Effect<GenerateObjectResponse<Tools, A>, ExtractError<Options>, R | ExtractContext<Options>>;
  readonly generateText: <
    Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
    Tools extends Record<string, Any> = {},
  >(
    options: Options & GenerateTextOptions<Tools>,
  ) => Effect<GenerateTextResponse<Tools>, ExtractError<Options>, ExtractContext<Options>>;
  readonly streamText: <
    Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
    Tools extends Record<string, Any> = {},
  >(
    options: Options & GenerateTextOptions<Tools>,
  ) => Stream<StreamPart<Tools>, ExtractError<Options>, ExtractContext<Options>>;
}

ToolChoice type

Added in v1.0.0 Source

The tool choice mode for the language model. - auto (default): The model can decide whether or not to call tools, as well as which tools to call. - required: The model must call a tool but can decide which tool will be called. - none: The model must not call a tool. - { tool: <tool_name> }: The model must call the specified tool. - { mode?: "auto" (default) | "required", "oneOf": [<tool-names>] }: The model is restricted to the subset of tools specified by oneOf. When mode is "auto" or omitted, the model can decide whether or not a tool from the allowed subset of tools can be called. When mode is "required", the model must call one tool from the allowed subset of tools.

Signature

type ToolChoice<Tools extends string> =
  | "auto"
  | "none"
  | "required"
  | {
      readonly tool: Tools;
    }
  | {
      readonly mode?: "auto" | "required";
      readonly oneOf: ReadonlyArray<Tools>;
    };

Utility Types

ExtractContext type

Added in v1.0.0 Source

Utility type that extracts the context requirements from LanguageModel options.

Automatically infers the required services based on the toolkit configuration.

Signature

type ExtractContext<Options> = Options extends {
  readonly toolkit: Toolkit.WithHandler<infer _Tools>;
}
  ? Tool.Requirements<_Tools[keyof _Tools]>
  : Options extends {
        readonly toolkit: Effect.Effect<Toolkit.WithHandler<infer _Tools>, infer _E, infer _R>;
      }
    ? Tool.Requirements<_Tools[keyof _Tools]> | _R
    : never;

ExtractError type

Added in v1.0.0 Source

Utility type that extracts the error type from LanguageModel options.

Automatically infers the possible error types based on toolkit configuration and tool call resolution settings.

Signature

type ExtractError<Options> = Options extends {
  readonly disableToolCallResolution: true;
  readonly toolkit: Toolkit.WithHandler<infer _Tools>;
}
  ? AiError.AiError
  : Options extends {
        readonly disableToolCallResolution: true;
        readonly toolkit: Effect.Effect<Toolkit.WithHandler<infer _Tools>, infer _E, infer _R>;
      }
    ? AiError.AiError | _E
    : Options extends {
          readonly toolkit: Toolkit.WithHandler<infer _Tools>;
        }
      ? AiError.AiError | Tool.HandlerError<_Tools[keyof _Tools]>
      : Options extends {
            readonly toolkit: Effect.Effect<Toolkit.WithHandler<infer _Tools>, infer _E, infer _R>;
          }
        ? AiError.AiError | Tool.HandlerError<_Tools[keyof _Tools]> | _E
        : AiError.AiError;