Getting Started

In this getting started guide, we will demonstrate how to generate a simple text completion using an LLM provider (OpenAi) using the Effect AI integration packages.

We’ll walk through:

• Writing provider-agnostic logic to interact with an LLM
• Declaring the specific LLM model to use for the interaction
• Using a provider integration to make the program executable

Installation

First, we will need to install the base @effect/ai package to gain access to the core AI abstractions. In addition, we will need to install at least one provider integration package (in this case @effect/ai-openai):

npm

# Install the base package for the core abstractions (always required)
npm install @effect/ai

# Install one (or more) provider integrations
npm install @effect/ai-openai

# Also add the core Effect package (if not already installed)
npm install effect

pnpm

# Install the base package for the core abstractions (always required)
pnpm add @effect/ai

# Install one (or more) provider integrations
pnpm add @effect/ai-openai

# Also add the core Effect package (if not already installed)
pnpm add effect

Yarn

# Install the base package for the core abstractions (always required)
yarn add @effect/ai

# Install one (or more) provider integrations
yarn add @effect/ai-openai

# Also add the core Effect package (if not already installed)
yarn add effect

Bun

# Install the base package for the core abstractions (always required)
bun add @effect/ai

# Install one (or more) provider integrations
bun add @effect/ai-openai

# Also add the core Effect package (if not already installed)
bun add effect

Define an Interaction with a Language Model

First let’s define a simple interaction with a large language model (LLM):

Example (Using the LanguageModel Service to Generate a Dad Joke)

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

// Using `LanguageModel` will add it to your program's requirements
//
//          ┌─── Effect<GenerateTextResponse<{}>, AiError, LanguageModel>
//          ▼
const generateDadJoke = Effect.gen(function*() {
  // Use the `LanguageModel` to generate some text
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  // Log the generated text to the console
  console.log(response.text)
  // Return the response
  return response
})

Declarative LLM Interactions

Notice that the above code does not know or care which LLM provider (OpenAi, Anthropic, etc.) will be used. Instead, we focus on what we want to accomplish (i.e. our business logic), not how to accomplish it.

Select a Provider

Next, we need to select which model provider we want to use:

Example (Using a Model Provider to Satisfy the LanguageModel Requirement)

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

// Create a `Model` which provides a concrete implementation of
// `LanguageModel` and requires an `OpenAiClient`
//
//      ┌─── Model<"openai", LanguageModel | ProviderName, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

// Provide the `Model` to the program
//
//     ┌─── Effect<GenerateTextResponse<{}>, AiError, OpenAiClient>
//     ▼
const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

Before moving on, it is important that we understand the purpose of the Model data type.

Understanding Model

The Model data type represents a provider-specific implementation of one or more services, such as LanguageModel or EmbeddingsModel. It is the primary way that you can plug a real large language model into your program.

export interface Model<ProviderName, Provides, Requires> {}

An Model has three generic type parameters:

• ProviderName - the name of the large language model provider that will be used
• Provides - the services this model will provide when built
• Requires - the services this model will require to be built

This allows Effect to track which services the Model requires as well as which services the Model will provide.

Creating n Model

To create a Model, you can use the model-specific factory from one of Effect’s provider integration packages.

Example (Defining a Model to Interact with OpenAI)

import { OpenAiLanguageModel } from "@effect/ai-openai"

//      ┌─── Model<"openai", LanguageModel | ProviderName, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

This creates a Model that:

• Provides the ProviderName service, which allows introspection of the current provider in use by the program
• Provides an OpenAI-specific implementation of the LanguageModel service using "gpt-4o"
• Requires an OpenAiClient to be built

Providing a Model

Once you’ve created a Model, you can directly Effect.provide it to your Effect programs just like any other service:

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"

//      ┌─── Model<"openai", LanguageModel | ProviderName, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

//       ┌─── Effect<GenerateTextResponse<{}>, AiError, OpenAiClient>
//       ▼
const program = LanguageModel.generateText({
  prompt: "Generate a dad joke"
}).pipe(Effect.provide(Gpt4o))

Benefits of Model

There are several benefits to this approach:

Reusability

You can provide the same Model to as many programs as you like.

Example (Providing a Model to Multiple Programs)

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = Effect.gen(function*() {
  // You can provide the `Model` individually to each
  // program, or to all of them at once (as we do here)
  const res1 = yield* generateDadJoke
  const res2 = yield* generateDadJoke
  const res3 = yield* generateDadJoke
}).pipe(Effect.provide(Gpt4o))

Flexibility

If we know that one model or provider performs better at a given task than another, we can freely mix and match models and providers together.

For example, if we know Anthropic’s Claude generates some really great dad jokes, we can mix it into our existing program with just a few lines of code:

Example (Mixing Multiple Providers and Models)

import { AnthropicLanguageModel } from "@effect/ai-anthropic"
import { OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
const Claude37 = AnthropicLanguageModel.model("claude-3-7-sonnet-latest")

//      ┌─── Effect<void, AiError, AnthropicClient | OpenAiClient>
//      ▼
const main = Effect.gen(function*() {
  const res1 = yield* generateDadJoke
  const res2 = yield* generateDadJoke
  const res3 = yield* Effect.provide(generateDadJoke, Claude37)
}).pipe(Effect.provide(Gpt4o))

Because Effect performs type-level dependency tracking, we can see that an AnthropicClient is now required to make our program runnable.

Abstractability

An Model can also be yield*’ed to lift its dependencies into the calling Effect. This is particularly useful when creating services that depend on AI interactions, where you want to avoid leaking service-level dependencies into the service interface.

For example, in the code below the main program is only dependent upon the DadJokes service. All AI requirements are abstracted away into Layer composition.

Example (Abstracting LLM Interactions into a Service)

import { AnthropicLanguageModel } from "@effect/ai-anthropic"
import { OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
const Claude37 = AnthropicLanguageModel.model("claude-3-7-sonnet-latest")

class DadJokes extends Effect.Service<DadJokes>()("app/DadJokes", {
  effect: Effect.gen(function*() {
    // Yielding the model will return a layer with no requirements
    //
    //     ┌─── Layer<LanguageModel | ProviderName>
    //     ▼
    const gpt = yield* Gpt4o
    const claude = yield* Claude37

    const generateDadJoke = Effect.gen(function*() {
      const response = yield* LanguageModel.generateText({
        prompt: "Generate a dad joke"
      })
      console.log(response.text)
      return response
    })

    return {
      generateDadJoke: Effect.provide(generateDadJoke, gpt),
      generateBetterDadJoke: Effect.provide(generateDadJoke, claude)
    }
  })
}) {}

// Programs which utilize the `DadJokes` service have no knowledge of
// any AI requirements
//
//     ┌─── Effect<void, AiError, DadJokes>
//     ▼
const main = Effect.gen(function*() {
  const dadJokes = yield* DadJokes
  const res1 = yield* dadJokes.generateDadJoke
  const res2 = yield* dadJokes.generateBetterDadJoke
})

// The AI requirements are abstracted away into `Layer` composition
//
//         ┌─── Layer<DadJokes, never, AnthropicClient | OpenAiClient>
//         ▼
DadJokes.Default

Create a Provider Client

To make our code executable, we must finish satisfying our program’s requirements.

Let’s take another look at our program from earlier:

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

//     ┌─── Effect<GenerateTextResponse<{}>, AiError, OpenAiClient>
//     ▼
const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

We can see that our main program still requires us to provide an OpenAiClient.

Each of our provider integration packages exports a client module that can be used to construct a client for that provider.

Example (Creating a Client Layer for a Model Provider)

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { Config, Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

// Create a `Layer` which produces an `OpenAiClient` and requires
// an `HttpClient`
//
//      ┌─── Layer<OpenAiClient, ConfigError, HttpClient>
//      ▼
const OpenAi = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
})

In the code above, we use the layerConfig constructor from the OpenAiClient module to create a Layer which will produce an OpenAiClient. The layerConfig constructor allows us to read in configuration variables using Effect’s configuration system.

The provider clients also have a dependency on an HttpClient implementation to avoid any platform dependencies. This way, you can provide whichever HttpClient implementation is most appropriate for the platform your code is running upon.

For example, if we know we are going to run this code in NodeJS, we can utilize the NodeHttpClient module from @effect/platform-node to provide an HttpClient implementation:

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { NodeHttpClient } from "@effect/platform-node"
import { Config, Effect, Layer } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

// Create a `Layer` which produces an `OpenAiClient` and requires
// an `HttpClient`
//
//      ┌─── Layer<OpenAiClient, ConfigError, HttpClient>
//      ▼
const OpenAi = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
})

// Provide a platform-specific implementation of `HttpClient` to our
// OpenAi layer
//
//        ┌─── Layer<OpenAiClient, ConfigError, never>
//        ▼
const OpenAiWithHttp = Layer.provide(OpenAi, NodeHttpClient.layerUndici)

Running the Program

Now that we have a Layer which provides us with an OpenAiClient, we’re ready to make our main program runnable.

Our final program looks like the following:

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { LanguageModel } from "@effect/ai"
import { NodeHttpClient } from "@effect/platform-node"
import { Config, Effect, Layer } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

const OpenAi = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
})

const OpenAiWithHttp = Layer.provide(OpenAi, NodeHttpClient.layerUndici)

main.pipe(
  Effect.provide(OpenAiWithHttp),
  Effect.runPromise
)