Execution Planning

Imagine that we’ve refactored our generateDadJoke program from our Getting Started guide. Now, instead of handling all errors internally, the code can fail with domain-specific issues like network interruptions or provider outages:

1
import type { LanguageModel } from "@effect/ai"
2
import { OpenAiLanguageModel } from "@effect/ai-openai"
3
import { Data, Effect } from "effect"
4

5
class NetworkError extends Data.TaggedError("NetworkError") {}
6

7
class ProviderOutage extends Data.TaggedError("ProviderOutage") {}
8

9
declare const generateDadJoke: Effect.Effect<
10
  LanguageModel.GenerateTextResponse<{}>,
11
  NetworkError | ProviderOutage,
12
  LanguageModel.LanguageModel
13
>
14

15
const main = Effect.gen(function* () {
16
  const response = yield* generateDadJoke
17
  console.log(response.text)
18
}).pipe(Effect.provide(OpenAiLanguageModel.model("gpt-4o")))

This is fine, but what if we want to:

Retry the program a fixed number of times on NetworkErrors
Add some backoff delay between retries
Fallback to a different model provider if OpenAi is down

How can we accomplish such logic?

Planning LLM Interactions

The ExecutionPlan module from Effect provides a robust method for creating structured execution plans for your Effect programs. Rather than making a single model call and hoping that it succeeds, you can use ExecutionPlan to describe how to handle errors, retries, and fallbacks in a clear, declarative way.

This is especially useful when:

You want to fall back to a secondary model if the primary one is unavailable
You want to retry on transient errors (e.g. network failures)
You want to control timing between retry attempts

Creating Execution Plans

To create an ExecutionPlan, we can use the ExecutionPlan.make constructor.

Example (Creating an ExecutionPlan for LLM Interactions)

1
import type { LanguageModel } from "@effect/ai"
2
import { OpenAiLanguageModel } from "@effect/ai-openai"
3
import { Data, Effect, ExecutionPlan, Schedule } from "effect"
4

9 collapsed lines
5
class NetworkError extends Data.TaggedError("NetworkError") {}
6

7
class ProviderOutage extends Data.TaggedError("ProviderOutage") {}
8

9
declare const generateDadJoke: Effect.Effect<
10
  LanguageModel.GenerateTextResponse<{}>,
11
  NetworkError | ProviderOutage,
12
  LanguageModel.LanguageModel
13
>
14

15
const DadJokePlan = ExecutionPlan.make({
16
  provide: OpenAiLanguageModel.model("gpt-4o"),
17
  attempts: 3,
18
  schedule: Schedule.exponential("100 millis", 1.5),
19
  while: (error: NetworkError | ProviderOutage) => error._tag === "NetworkError",
20
})
21

22
//     ┌─── Effect<void, NetworkError | ProviderOutage, OpenAiClient>
23
//     ▼
24
const main = Effect.gen(function* () {
25
  const response = yield* generateDadJoke
26
  console.log(response.text)
27
}).pipe(Effect.withExecutionPlan(DadJokePlan))

This plan contains a single step which will:

Provide OpenAi’s "gpt-4o" model as a LanguageModel for the program
Attempt to call OpenAi up to 3 times
Wait with an exponential backoff between attempts (starting at 100ms)
Only re-attempt the call to OpenAi if the error is a NetworkError

Adding Fallback Models

To make your interactions with large language models resilient to provider outages, you can define a fallback models to use. This will allow the plan to automatically fallback to another model if the previous step in the execution plan fails.

Use this when:

You want to make your model interactions resilient to provider outages
You want to potentially have multiple fallback models

Example (Adding a Fallback to Anthropic from OpenAi)

1
import type { LanguageModel } from "@effect/ai"
2
import { AnthropicLanguageModel } from "@effect/ai-anthropic"
3
import { OpenAiLanguageModel } from "@effect/ai-openai"
4
import { Data, Effect, ExecutionPlan, Schedule } from "effect"
5

9 collapsed lines
6
class NetworkError extends Data.TaggedError("NetworkError") {}
7

8
class ProviderOutage extends Data.TaggedError("ProviderOutage") {}
9

10
declare const generateDadJoke: Effect.Effect<
11
  LanguageModel.GenerateTextResponse<{}>,
12
  NetworkError | ProviderOutage,
13
  LanguageModel.LanguageModel
14
>
15

16
const DadJokePlan = ExecutionPlan.make(
17
  {
18
    provide: OpenAiLanguageModel.model("gpt-4o"),
19
    attempts: 3,
20
    schedule: Schedule.exponential("100 millis", 1.5),
21
    while: (error: NetworkError | ProviderOutage) => error._tag === "NetworkError",
22
  },
23
  {
24
    provide: AnthropicLanguageModel.model("claude-4-sonnet-20250514"),
25
    attempts: 2,
26
    schedule: Schedule.exponential("100 millis", 1.5),
27
    while: (error: NetworkError | ProviderOutage) => error._tag === "ProviderOutage",
28
  },
29
)
30

31
//     ┌─── Effect<..., ..., AnthropicClient | OpenAiClient>
32
//     ▼
33
const main = Effect.gen(function* () {
34
  const response = yield* generateDadJoke
35
  console.log(response.text)
36
}).pipe(Effect.withExecutionPlan(DadJokePlan))

This plan contains two steps.

Step 1

The first step will:

Provide OpenAi’s "gpt-4o" model as a LanguageModel for the program
Attempt to call OpenAi up to 3 times
Wait with an exponential backoff between attempts (starting at 100ms)
Only attempt the call to OpenAi if the error is a NetworkError

If all of the above logic fails to run the program successfully, the plan will try to run the program using the second step.

Step 2

The second step will:

Provide Anthropic’s "claude-4-sonnet-20250514" model as a LanguageModel for the program
Attempt to call Anthropic up to 2 times
Wait with an exponential backoff between attempts (starting at 100ms)
Only attempt the fallback if the error is a ProviderOutage

End-to-End Usage

The following is the complete program with the desired execution plan fully implemented:

1
import type { LanguageModel } from "@effect/ai"
2
import { AnthropicClient, AnthropicLanguageModel } from "@effect/ai-anthropic"
3
import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
4
import { NodeHttpClient } from "@effect/platform-node"
5
import { Config, Data, Effect, ExecutionPlan, Layer, Schedule } from "effect"
6

7
class NetworkError extends Data.TaggedError("NetworkError") {}
8

9
class ProviderOutage extends Data.TaggedError("ProviderOutage") {}
10

11
declare const generateDadJoke: Effect.Effect<
12
  LanguageModel.GenerateTextResponse<{}>,
13
  NetworkError | ProviderOutage,
14
  LanguageModel.LanguageModel
15
>
16

17
const DadJokePlan = ExecutionPlan.make(
18
  {
19
    provide: OpenAiLanguageModel.model("gpt-4o"),
20
    attempts: 3,
21
    schedule: Schedule.exponential("100 millis", 1.5),
22
    while: (error: NetworkError | ProviderOutage) => error._tag === "NetworkError",
23
  },
24
  {
25
    provide: AnthropicLanguageModel.model("claude-4-sonnet-20250514"),
26
    attempts: 2,
27
    schedule: Schedule.exponential("100 millis", 1.5),
28
    while: (error: NetworkError | ProviderOutage) => error._tag === "ProviderOutage",
29
  },
30
)
31

32
const main = Effect.gen(function* () {
33
  const response = yield* generateDadJoke
34
  console.log(response.text)
35
}).pipe(Effect.withExecutionPlan(DadJokePlan))
36

37
const Anthropic = AnthropicClient.layerConfig({
38
  apiKey: Config.redacted("ANTHROPIC_API_KEY"),
39
}).pipe(Layer.provide(NodeHttpClient.layerUndici))
40

41
const OpenAi = OpenAiClient.layerConfig({
42
  apiKey: Config.redacted("OPENAI_API_KEY"),
43
}).pipe(Layer.provide(NodeHttpClient.layerUndici))
44

45
main.pipe(Effect.provide([Anthropic, OpenAi]), Effect.runPromise)