A type of Large Language Model (LLM) that interacts with the Bedrock service. It extends the base LLM class and implements the BaseBedrockInput interface. The class is designed to authenticate and interact with the Bedrock service, which is a part of Amazon Web Services (AWS). It uses AWS credentials for authentication and can be configured with various parameters such as the model to use, the AWS region, and the maximum number of tokens to generate.

The BedrockChat class supports both synchronous and asynchronous interactions with the model, allowing for streaming responses and handling new token callbacks. It can be configured with optional parameters like temperature, stop sequences, and guardrail settings for enhanced control over the generated responses.

Example

import { BedrockChat } from 'path-to-your-bedrock-chat-module';
import { HumanMessage } from '@langchain/core/messages';

async function run() {
// Instantiate the BedrockChat model with the desired configuration
const model = new BedrockChat({
model: "anthropic.claude-v2",
region: "us-east-1",
credentials: {
accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
},
maxTokens: 150,
temperature: 0.7,
stopSequences: ["\n", " Human:", " Assistant:"],
streaming: false,
trace: "ENABLED",
guardrailIdentifier: "your-guardrail-id",
guardrailVersion: "1.0",
guardrailConfig: {
tagSuffix: "example",
streamProcessingMode: "SYNCHRONOUS",
},
});

// Prepare the message to be sent to the model
const message = new HumanMessage("Tell me a joke");

// Invoke the model with the message
const res = await model.invoke([message]);

// Output the response from the model
console.log(res);
}

run().catch(console.error);

For streaming responses, use the following example:

Example

import { BedrockChat } from 'path-to-your-bedrock-chat-module';
import { HumanMessage } from '@langchain/core/messages';

async function runStreaming() {
// Instantiate the BedrockChat model with the desired configuration
const model = new BedrockChat({
model: "anthropic.claude-3-sonnet-20240229-v1:0",
region: "us-east-1",
credentials: {
accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
},
maxTokens: 150,
temperature: 0.7,
stopSequences: ["\n", " Human:", " Assistant:"],
streaming: true,
trace: "ENABLED",
guardrailIdentifier: "your-guardrail-id",
guardrailVersion: "1.0",
guardrailConfig: {
tagSuffix: "example",
streamProcessingMode: "SYNCHRONOUS",
},
});

// Prepare the message to be sent to the model
const message = new HumanMessage("Tell me a joke");

// Stream the response from the model
const stream = await model.stream([message]);
for await (const chunk of stream) {
// Output each chunk of the response
console.log(chunk);
}
}

runStreaming().catch(console.error);

Hierarchy (view full)

Implements

  • BaseBedrockInput

Constructors

Properties

codec: EventStreamCodec = ...
credentials: CredentialType
fetchFn: {
    (input, init?): Promise<Response>;
    (input, init?): Promise<Response>;
}

Type declaration

    • (input, init?): Promise<Response>
    • Parameters

      • input: string | URL | Request
      • Optional init: RequestInit

      Returns Promise<Response>

    • (input, init?): Promise<Response>
    • Parameters

      • input: RequestInfo
      • Optional init: RequestInit

      Returns Promise<Response>

guardrailIdentifier: string = ""
guardrailVersion: string = ""
model: string = "amazon.titan-tg1-large"
region: string
streaming: boolean = false
usesMessagesApi: boolean = false
endpointHost?: string
guardrailConfig?: {
    streamProcessingMode: "SYNCHRONOUS" | "ASYNCHRONOUS";
    tagSuffix: string;
}

Type declaration

  • streamProcessingMode: "SYNCHRONOUS" | "ASYNCHRONOUS"
  • tagSuffix: string
maxTokens?: number = undefined
modelKwargs?: Record<string, unknown>
stopSequences?: string[]

⚠️ Deprecated ⚠️

Use as a call option using .bind() instead.

This feature is deprecated and will be removed in the future.

It is not recommended for use.

temperature?: number = undefined
trace?: "ENABLED" | "DISABLED"

Methods

  • Parameters

    • tools: any[]
    • Optional _kwargs: any

    Returns Runnable<BaseLanguageModelInput, BaseMessageChunk, BaseChatModelCallOptions>

  • Parameters

    • Optional options: unknown

    Returns {
        max_tokens: undefined | number;
        stop: any;
        temperature: undefined | number;
        tools: undefined | Record<string, unknown>[];
    }

    • max_tokens: undefined | number
    • stop: any
    • temperature: undefined | number
    • tools: undefined | Record<string, unknown>[]

Generated using TypeDoc