Configuration

Configuration is an essential aspect of any cloud-native application. Effect simplifies the process of managing configuration by offering a convenient interface for configuration providers.

The configuration front-end in Effect enables ecosystem libraries and applications to specify their configuration requirements in a declarative manner. It offloads the complex tasks to a ConfigProvider, which can be supplied by third-party libraries.

Effect comes bundled with a straightforward default ConfigProvider that retrieves configuration data from environment variables. This default provider can be used during development or as a starting point before transitioning to more advanced configuration providers.

To make our application configurable, we need to understand three essential elements:

• Config Description: We describe the configuration data using an instance of Config<A>. If the configuration data is simple, such as a string, number, or boolean, we can use the built-in functions provided by the Config module. For more complex data types like HostPort, we can combine primitive configs to create a custom configuration description.

• Config Frontend: We utilize the instance of Config<A> to load the configuration data described by the instance (a Config is, in itself, an effect). This process leverages the current ConfigProvider to retrieve the configuration.

• Config Backend: The ConfigProvider serves as the underlying engine that manages the configuration loading process. Effect comes with a default config provider as part of its default services. This default provider reads the configuration data from environment variables. If we want to use a custom config provider, we can utilize the Effect.withConfigProvider API to configure the Effect runtime accordingly.

Basic Configuration Types

Effect provides several built-in types for configuration values, which you can use right out of the box:

(*) string | number | boolean | null | bigint

Example (Loading Environment Variables)

Here’s an example of loading a basic configuration using environment variables for HOST and PORT:

1
import { Effect, Config } from "effect"
2

3
// Define a program that loads HOST and PORT configuration
4
const program = Effect.gen(function* () {
5
  const host = yield* Config.string("HOST") // Read as a string
6
  const port = yield* Config.number("PORT") // Read as a number
7

8
  console.log(`Application started: ${host}:${port}`)
9
})
10

11
Effect.runPromise(program)

If you run this without setting the required environment variables:

npx tsx primitives.ts

you’ll see an error indicating the missing configuration:

[Error: (Missing data at HOST: "Expected HOST to exist in the process context")] {
  name: '(FiberFailure) Error',
  [Symbol(effect/Runtime/FiberFailure)]: Symbol(effect/Runtime/FiberFailure),
  [Symbol(effect/Runtime/FiberFailure/Cause)]: {
    _tag: 'Fail',
    error: {
      _op: 'MissingData',
      path: [ 'HOST' ],
      message: 'Expected HOST to exist in the process context'
    }
  }
}

To run the program successfully, set the environment variables as shown below:

HOST=localhost PORT=8080 npx tsx primitives.ts

Output:

Application started: localhost:8080

Using Config with Schema

You can define and decode configuration values using a schema.

Example (Decoding a Configuration Value)

1
import { Effect, Schema } from "effect"
2

3
// Define a config that expects a string with at least 4 characters
4
const myConfig = Schema.Config("Foo", Schema.String.pipe(Schema.minLength(4)))

For more information, see the Schema.Config documentation.

Providing Default Values

Sometimes, you may encounter situations where an environment variable is missing, leading to an incomplete configuration. To address this, Effect provides the Config.withDefault function, which allows you to specify a default value. This fallback ensures that your application continues to function even if a required environment variable is not set.

Example (Using Default Values)

1
import { Effect, Config } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const host = yield* Config.string("HOST")
5
  // Use default 8080 if PORT is not set
6
  const port = yield* Config.number("PORT").pipe(Config.withDefault(8080))
7
  console.log(`Application started: ${host}:${port}`)
8
})
9

10
Effect.runPromise(program)

Running this program with only the HOST environment variable set:

HOST=localhost npx tsx defaults.ts

produces the following output:

Application started: localhost:8080

In this case, even though the PORT environment variable is not set, the program continues to run, using the default value of 8080 for the port. This ensures that the application remains functional without requiring every configuration to be explicitly provided.

Handling Sensitive Values

Some configuration values, like API keys, should not be printed in logs.

The Config.redacted function is used to handle sensitive information safely. It parses the configuration value and wraps it in a Redacted<string>, a specialized data type designed to protect secrets.

When you log a Redacted value using console.log, the actual content remains hidden, providing an extra layer of security. To access the real value, you must explicitly use Redacted.value.

Example (Protecting Sensitive Data)

1
import { Effect, Config, Redacted } from "effect"
2

3
const program = Effect.gen(function* () {
4
  //      ┌─── Redacted<string>
5
  //      ▼
6
  const redacted = yield* Config.redacted("API_KEY")
7

8
  // Log the redacted value, which won't reveal the actual secret
9
  console.log(`Console output: ${redacted}`)
10

11
  // Access the real value using Redacted.value and log it
12
  console.log(`Actual value: ${Redacted.value(redacted)}`)
13
})
14

15
Effect.runPromise(program)

When this program is executed:

API_KEY=my-api-key tsx redacted.ts

The output will look like this:

Console output: <redacted>
Actual value: my-api-key

As shown, when logging the Redacted value using console.log, the output is <redacted>, ensuring that sensitive data remains concealed. However, by using Redacted.value, the true value ("my-api-key") can be accessed and displayed, providing controlled access to the secret.

Wrapping a Config with Redacted

By default, when you pass a string to Config.redacted, it returns a Redacted<string>. You can also pass a Config (such as Config.number) to ensure that only validated values are accepted. This adds an extra layer of security by ensuring that sensitive data is properly validated before being redacted.

Example (Redacting and Validating a Number)

1
import { Effect, Config, Redacted } from "effect"
2

3
const program = Effect.gen(function* () {
4
  // Wrap the validated number configuration with redaction
5
  //
6
  //      ┌─── Redacted<number>
7
  //      ▼
8
  const redacted = yield* Config.redacted(Config.number("SECRET"))
9

10
  console.log(`Console output: ${redacted}`)
11
  console.log(`Actual value: ${Redacted.value(redacted)}`)
12
})
13

14
Effect.runPromise(program)

Combining Configurations

Effect provides several built-in combinators that allow you to define and manipulate configurations. These combinators take a Config as input and produce another Config, enabling more complex configuration structures.

Additionally, there are three special combinators for specific use cases:

Example (Using the array combinator)

The following example demonstrates how to load an environment variable as an array of strings using the Config.array constructor.

1
import { Config, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const config = yield* Config.array(Config.string(), "MYARRAY")
5
  console.log(config)
6
})
7

8
Effect.runPromise(program)
9
// Run:
10
// MYARRAY=a,b,c,a npx tsx index.ts
11
// Output:
12
// [ 'a', 'b', 'c', 'a' ]

Example (Using the hashSet combinator)

1
import { Config, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const config = yield* Config.hashSet(Config.string(), "MYSET")
5
  console.log(config)
6
})
7

8
Effect.runPromise(program)
9
// Run:
10
// MYSET=a,"b c",d,a npx tsx index.ts
11
// Output:
12
// { _id: 'HashSet', values: [ 'd', 'a', 'b c' ] }

Example (Using the hashMap combinator)

1
import { Config, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const config = yield* Config.hashMap(Config.string(), "MYMAP")
5
  console.log(config)
6
})
7

8
Effect.runPromise(program)
9
// Run:
10
// MYMAP_A=a MYMAP_B=b npx tsx index.ts
11
// Output:
12
// { _id: 'HashMap', values: [ [ 'A', 'a' ], [ 'B', 'b' ] ] }

Operators

Effect provides several built-in operators to work with configurations, allowing you to manipulate and transform them according to your needs.

Transforming Operators

These operators enable you to modify configurations or validate their values:

Example (Using validate Operator)

1
import { Effect, Config } from "effect"
2

3
const program = Effect.gen(function* () {
4
  // Load the NAME environment variable and validate its length
5
  const config = yield* Config.string("NAME").pipe(
6
    Config.validate({
7
      message: "Expected a string at least 4 characters long",
8
      validation: (s) => s.length >= 4,
9
    }),
10
  )
11
  console.log(config)
12
})
13

14
Effect.runPromise(program)

If we run this program with an invalid NAME value:

NAME=foo npx tsx validate.ts

The output will be:

[Error: (Invalid data at NAME: "Expected a string at least 4 characters long")] {
  name: '(FiberFailure) Error',
  [Symbol(effect/Runtime/FiberFailure)]: Symbol(effect/Runtime/FiberFailure),
  [Symbol(effect/Runtime/FiberFailure/Cause)]: {
    _tag: 'Fail',
    error: {
      _op: 'InvalidData',
      path: [ 'NAME' ],
      message: 'Expected a string at least 4 characters long'
    }
  }
}

Fallback Operators

Fallback operators are useful when you want to provide alternative configurations in case of errors or missing data. These operators ensure that your program can still run even if some configuration values are unavailable.

Example (Using orElse for Fallback)

In this example, the program requires two configuration values: A and B. We set up two configuration providers, each containing only one of the required values. Using the orElse operator, we combine these providers so the program can retrieve both A and B.

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
// A program that requires two configurations: A and B
4
const program = Effect.gen(function* () {
5
  const A = yield* Config.string("A") // Retrieve config A
6
  const B = yield* Config.string("B") // Retrieve config B
7
  console.log(`A: ${A}, B: ${B}`)
8
})
9

10
// First provider has A but is missing B
11
const provider1 = ConfigProvider.fromMap(new Map([["A", "A"]]))
12

13
// Second provider has B but is missing A
14
const provider2 = ConfigProvider.fromMap(new Map([["B", "B"]]))
15

16
// Use `orElse` to fall back from provider1 to provider2
17
const provider = provider1.pipe(ConfigProvider.orElse(() => provider2))
18

19
Effect.runPromise(Effect.withConfigProvider(program, provider))

If we run this program:

npx tsx orElse.ts

The output will be:

A: A, B: B

Custom Configuration Types

Effect allows you to define configurations for custom types by combining primitive configurations using combinators and operators.

For example, let’s create a HostPort class, which has two fields: host and port.

1
class HostPort {
2
  constructor(
3
    readonly host: string,
4
    readonly port: number,
5
  ) {}
6
  get url() {
7
    return `${this.host}:${this.port}`
8
  }
9
}

To define a configuration for this custom type, we can combine primitive configs for string and number:

Example (Defining a Custom Configuration)

1
import { Config } from "effect"
2

3
class HostPort {
4
  constructor(
5
    readonly host: string,
6
    readonly port: number,
7
  ) {}
8
  get url() {
9
    return `${this.host}:${this.port}`
10
  }
11
}
12

13
// Combine the configuration for 'HOST' and 'PORT'
14
const both = Config.all([Config.string("HOST"), Config.number("PORT")])
15

16
// Map the configuration values into a HostPort instance
17
const config = Config.map(both, ([host, port]) => new HostPort(host, port))

In this example, Config.all(configs) combines two primitive configurations, Config<string> and Config<number>, into a Config<[string, number]>. The Config.map operator is then used to transform these values into an instance of the HostPort class.

Example (Using Custom Configuration)

1
import { Effect, Config } from "effect"
2

3
class HostPort {
4
  constructor(
5
    readonly host: string,
6
    readonly port: number,
7
  ) {}
8
  get url() {
9
    return `${this.host}:${this.port}`
10
  }
11
}
12

13
// Combine the configuration for 'HOST' and 'PORT'
14
const both = Config.all([Config.string("HOST"), Config.number("PORT")])
15

16
// Map the configuration values into a HostPort instance
17
const config = Config.map(both, ([host, port]) => new HostPort(host, port))
18

19
// Main program that reads configuration and starts the application
20
const program = Effect.gen(function* () {
21
  const hostPort = yield* config
22
  console.log(`Application started: ${hostPort.url}`)
23
})
24

25
Effect.runPromise(program)

When you run this program, it will try to retrieve the values for HOST and PORT from your environment variables:

HOST=localhost PORT=8080 npx tsx App.ts

If successful, it will print:

Application started: localhost:8080

Nested Configurations

We’ve seen how to define configurations at the top level, whether for primitive or custom types. In some cases, though, you might want to structure your configurations in a more nested way, organizing them under common namespaces for clarity and manageability.

For instance, consider the following ServiceConfig type:

1
class ServiceConfig {
2
  constructor(
3
    readonly host: string,
4
    readonly port: number,
5
    readonly timeout: number,
6
  ) {}
7
  get url() {
8
    return `${this.host}:${this.port}`
9
  }
10
}

If you were to use this configuration in your application, it would expect the HOST, PORT, and TIMEOUT environment variables at the top level. But in many cases, you may want to organize configurations under a shared namespace—for example, grouping HOST and PORT under a SERVER namespace, while keeping TIMEOUT at the root.

To do this, you can use the Config.nested operator, which allows you to nest configuration values under a specific namespace. Let’s update the previous example to reflect this:

1
import { Config } from "effect"
2

3
class ServiceConfig {
4
  constructor(
5
    readonly host: string,
6
    readonly port: number,
7
    readonly timeout: number,
8
  ) {}
9
  get url() {
10
    return `${this.host}:${this.port}`
11
  }
12
}
13

14
const serverConfig = Config.all([Config.string("HOST"), Config.number("PORT")])
15

16
const serviceConfig = Config.map(
17
  Config.all([
18
    // Read 'HOST' and 'PORT' from 'SERVER' namespace
19
    Config.nested(serverConfig, "SERVER"),
20
    // Read 'TIMEOUT' from the root namespace
21
    Config.number("TIMEOUT"),
22
  ]),
23
  ([[host, port], timeout]) => new ServiceConfig(host, port, timeout),
24
)

Now, if you run your application with this configuration setup, it will look for the following environment variables:

• SERVER_HOST for the host value
• SERVER_PORT for the port value
• TIMEOUT for the timeout value

This structured approach keeps your configuration more organized, especially when dealing with multiple services or complex applications.

Mocking Configurations in Tests

When testing services, there are times when you need to provide specific configurations for your tests. To simulate this, it’s useful to mock the configuration backend that reads these values.

You can achieve this using the ConfigProvider.fromMap constructor. This method allows you to create a configuration provider from a Map<string, string>, where the map represents the configuration data. You can then use this mock provider in place of the default one by calling Effect.withConfigProvider.

Example (Mocking a Config Provider for Testing)

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
class HostPort {
4
  constructor(
5
    readonly host: string,
6
    readonly port: number,
7
  ) {}
8
  get url() {
9
    return `${this.host}:${this.port}`
10
  }
11
}
12

13
const config = Config.map(
14
  Config.all([Config.string("HOST"), Config.number("PORT")]),
15
  ([host, port]) => new HostPort(host, port),
16
)
17

18
const program = Effect.gen(function* () {
19
  const hostPort = yield* config
20
  console.log(`Application started: ${hostPort.url}`)
21
})
22

23
// Create a mock config provider using a map with test data
24
const mockConfigProvider = ConfigProvider.fromMap(
25
  new Map([
26
    ["HOST", "localhost"],
27
    ["PORT", "8080"],
28
  ]),
29
)
30

31
// Run the program using the mock config provider
32
Effect.runPromise(Effect.withConfigProvider(program, mockConfigProvider))
33
// Output: Application started: localhost:8080

This approach helps you create isolated tests that don’t rely on external environment variables, ensuring your tests run consistently with mock configurations.

Handling Nested Configuration Values

For more complex setups, configurations often include nested keys. By default, ConfigProvider.fromMap uses . as the separator for nested keys.

Example (Providing Nested Configuration Values)

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
const config = Config.nested(Config.number("PORT"), "SERVER")
4

5
const program = Effect.gen(function* () {
6
  const port = yield* config
7
  console.log(`Server is running on port ${port}`)
8
})
9

10
// Mock configuration using '.' as the separator for nested keys
11
const mockConfigProvider = ConfigProvider.fromMap(new Map([["SERVER.PORT", "8080"]]))
12

13
Effect.runPromise(Effect.withConfigProvider(program, mockConfigProvider))
14
// Output: Server is running on port 8080

Customizing the Path Delimiter

If your configuration data uses a different separator (such as _), you can change the delimiter using the pathDelim option in ConfigProvider.fromMap.

Example (Using a Custom Path Delimiter)

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
const config = Config.nested(Config.number("PORT"), "SERVER")
4

5
const program = Effect.gen(function* () {
6
  const port = yield* config
7
  console.log(`Server is running on port ${port}`)
8
})
9

10
// Mock configuration using '_' as the separator
11
const mockConfigProvider = ConfigProvider.fromMap(new Map([["SERVER_PORT", "8080"]]), {
12
  pathDelim: "_",
13
})
14

15
Effect.runPromise(Effect.withConfigProvider(program, mockConfigProvider))
16
// Output: Server is running on port 8080

ConfigProvider

The ConfigProvider module in Effect allows applications to load configuration values from different sources. The default provider reads from environment variables, but you can customize its behavior when needed.

Loading Configuration from Environment Variables

The ConfigProvider.fromEnv function creates a ConfigProvider that loads values from environment variables. This is the default provider used by Effect unless another is specified.

If your application requires a custom delimiter for nested configuration keys, you can configure ConfigProvider.fromEnv accordingly.

Example (Changing the Path Delimiter)

The following example modifies the path delimiter ("__") and sequence delimiter ("|") for environment variables.

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  // Read SERVER_HOST and SERVER_PORT as nested configuration values
5
  const port = yield* Config.nested(Config.number("PORT"), "SERVER")
6
  const host = yield* Config.nested(Config.string("HOST"), "SERVER")
7
  console.log(`Application started: ${host}:${port}`)
8
})
9

10
Effect.runPromise(
11
  Effect.withConfigProvider(
12
    program,
13
    // Custom delimiters
14
    ConfigProvider.fromEnv({ pathDelim: "__", seqDelim: "|" }),
15
  ),
16
)

To match the custom delimiter ("__"), set environment variables like this:

SERVER__HOST=localhost SERVER__PORT=8080 npx tsx index.ts

Output:

Application started: localhost:8080

Loading Configuration from JSON

The ConfigProvider.fromJson function creates a ConfigProvider that loads values from a JSON object.

Example (Reading Nested Configuration from JSON)

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  // Read SERVER_HOST and SERVER_PORT as nested configuration values
5
  const port = yield* Config.nested(Config.number("PORT"), "SERVER")
6
  const host = yield* Config.nested(Config.string("HOST"), "SERVER")
7
  console.log(`Application started: ${host}:${port}`)
8
})
9

10
Effect.runPromise(
11
  Effect.withConfigProvider(
12
    program,
13
    ConfigProvider.fromJson(JSON.parse(`{"SERVER":{"PORT":8080,"HOST":"localhost"}}`)),
14
  ),
15
)
16
// Output: Application started: localhost:8080

Using Nested Configuration Namespaces

The ConfigProvider.nested function allows grouping configuration values under a namespace. This is helpful when structuring settings logically, such as grouping SERVER-related values.

Example (Using a Nested Namespace)

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const port = yield* Config.number("PORT") // Reads SERVER_PORT
5
  const host = yield* Config.string("HOST") // Reads SERVER_HOST
6
  console.log(`Application started: ${host}:${port}`)
7
})
8

9
Effect.runPromise(
10
  Effect.withConfigProvider(
11
    program,
12
    ConfigProvider.fromEnv().pipe(
13
      // Uses SERVER as a namespace
14
      ConfigProvider.nested("SERVER"),
15
    ),
16
  ),
17
)

Since we defined "SERVER" as the namespace, the environment variables must follow this pattern:

SERVER_HOST=localhost SERVER_PORT=8080 npx tsx index.ts

Output:

Application started: localhost:8080

Converting Configuration Keys to Constant Case

The ConfigProvider.constantCase function transforms all configuration keys into constant case (uppercase with underscores). This is useful when adapting environment variables to match different naming conventions.

Example (Using constantCase for Environment Variables)

1
import { Config, ConfigProvider, Effect } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const port = yield* Config.number("Port") // Reads PORT
5
  const host = yield* Config.string("Host") // Reads HOST
6
  console.log(`Application started: ${host}:${port}`)
7
})
8

9
Effect.runPromise(
10
  Effect.withConfigProvider(
11
    program,
12
    // Convert keys to constant case
13
    ConfigProvider.fromEnv().pipe(ConfigProvider.constantCase),
14
  ),
15
)

Since constantCase converts "Port" → "PORT" and "Host" → "HOST", the environment variables must be set as follows:

HOST=localhost PORT=8080 npx tsx index.ts

Output:

Application started: localhost:8080

Deprecations

Secret Deprecated

Deprecated since version 3.3.0: Please use Config.redacted for handling sensitive information going forward.

The Config.secret function was previously used to secure sensitive information in a similar way to Config.redacted. It wraps configuration values in a Secret type, which also conceals details when logged but allows access via Secret.value.

Example (Using Deprecated Config.secret)

1
import { Effect, Config, Secret } from "effect"
2

3
const program = Effect.gen(function* () {
4
  const secret = yield* Config.secret("API_KEY")
5

6
  // Log the secret value, which won't reveal the actual secret
7
  console.log(`Console output: ${secret}`)
8

9
  // Access the real value using Secret.value and log it
10
  console.log(`Actual value: ${Secret.value(secret)}`)
11
})
12

13
Effect.runPromise(program)

When this program is executed:

API_KEY=my-api-key tsx secret.ts

The output will look like this:

Console output: Secret(<redacted>)
Actual value: my-api-key