OpenAIRealtimeWebSocket

Transport layer that’s handling the connection between the client and OpenAI’s Realtime API via WebSockets. While this transport layer is designed to be used within a RealtimeSession, it can also be used standalone if you want to have a direct connection to the Realtime API.

Extends

OpenAIRealtimeBase

Implements

RealtimeTransportLayer

Constructors

Constructor

new OpenAIRealtimeWebSocket(options?): OpenAIRealtimeWebSocket

Parameters

Parameter	Type
`options`?	`OpenAIRealtimeWebSocketOptions`

Returns

OpenAIRealtimeWebSocket

Overrides

OpenAIRealtimeBase.constructor

Accessors

_tracingConfig

Set Signature

set _tracingConfig(tracingConfig): void

Sets the internal tracing config. This is used to track the tracing config that has been set during the session.create event.

Parameters

Parameter	Type
`tracingConfig`	`null` \| `RealtimeTracingConfig`

Returns

void

Inherited from

OpenAIRealtimeBase._tracingConfig

connectionState

Get Signature

get connectionState(): WebSocketState

The current connection state of the WebSocket connection.

Returns

WebSocketState

currentModel

Get Signature

get currentModel(): OpenAIRealtimeModels

The current model that is being used by the transport layer.

Returns

OpenAIRealtimeModels

Set Signature

set currentModel(model): void

The current model that is being used by the transport layer. Note: The model cannot be changed mid conversation.

Parameters

Parameter	Type
`model`	`OpenAIRealtimeModels`

Returns

void

Inherited from

OpenAIRealtimeBase.currentModel

muted

Get Signature

get muted(): null

Always returns null as the WebSocket transport layer does not handle muting. Instead, this should be handled by the client by not triggering the sendAudio method.

Returns

null

Whether the input audio track is currently muted null if the muting is not handled by the transport layer

Implementation of

RealtimeTransportLayer.muted

Overrides

OpenAIRealtimeBase.muted

status

Get Signature

get status(): "connecting" | "connected" | "disconnected"

The current status of the WebSocket connection.

Returns

"connecting" | "connected" | "disconnected"

Implementation of

RealtimeTransportLayer.status

Overrides

OpenAIRealtimeBase.status

Methods

_cancelResponse()

_cancelResponse(): void

Send a cancel response event to the Realtime API. This is used to cancel an ongoing response that the model is currently generating.

Returns

void

_interrupt()

_interrupt(elapsedTime): void

Do NOT call this method directly. Call interrupt() instead for proper interruption handling.

This method is used to send the right events to the API to inform the model that the user has interrupted the response. It might be overridden/extended by an extended transport layer. See the TwilioRealtimeTransportLayer for an example.

Parameters

Parameter	Type	Description
`elapsedTime`	`number`	The elapsed time since the response started.

Returns

void

close()

close(): void

Close the WebSocket connection.

This will also reset any internal connection tracking used for interruption handling.

Returns

void

Implementation of

RealtimeTransportLayer.close

Overrides

OpenAIRealtimeBase.close

connect()

connect(options): Promise<void>

Establishes the connection to the model and keeps the connection alive

Parameters

Parameter	Type	Description
`options`	`RealtimeTransportLayerConnectOptions`	The options for the connection

Returns

Promise<void>

Implementation of

RealtimeTransportLayer.connect

Overrides

OpenAIRealtimeBase.connect

emit()

emit<K>(type, ...args): boolean

Type Parameters

Type Parameter
`K` extends keyof `RealtimeTranportEventTypes`

Parameters

Parameter	Type
`type`	`K`
…`args`	`OpenAIRealtimeEventTypes`[`K`]

Returns

boolean

Implementation of

RealtimeTransportLayer.emit

Inherited from

OpenAIRealtimeBase.emit

interrupt()

interrupt(): void

Interrupt the ongoing response. This method is triggered automatically by the client when voice activity detection (VAD) is enabled (default) as well as when an output guardrail got triggered.

You can also call this method directly if you want to interrupt the conversation for example based on an event in the client.

Returns

void

Implementation of

RealtimeTransportLayer.interrupt

Overrides

OpenAIRealtimeBase.interrupt

mute()

mute(_muted): never

Will throw an error as the WebSocket transport layer does not support muting.

Parameters

Parameter	Type
`_muted`	`boolean`

Returns

never

Implementation of

RealtimeTransportLayer.mute

Overrides

OpenAIRealtimeBase.mute

off()

off<K>(type, listener): EventEmitter<EventTypes>

Type Parameters

Type Parameter
`K` extends keyof `RealtimeTranportEventTypes`

Parameters

Parameter	Type
`type`	`K`
`listener`	(…`args`) => `void`

Returns

EventEmitter<EventTypes>

Implementation of

RealtimeTransportLayer.off

Inherited from

OpenAIRealtimeBase.off

on()

on<K>(type, listener): EventEmitter<EventTypes>

Type Parameters

Type Parameter
`K` extends keyof `RealtimeTranportEventTypes`

Parameters

Parameter	Type
`type`	`K`
`listener`	(…`args`) => `void`

Returns

EventEmitter<EventTypes>

Implementation of

RealtimeTransportLayer.on

Inherited from

OpenAIRealtimeBase.on

once()

once<K>(type, listener): EventEmitter<EventTypes>

Type Parameters

Type Parameter
`K` extends keyof `RealtimeTranportEventTypes`

Parameters

Parameter	Type
`type`	`K`
`listener`	(…`args`) => `void`

Returns

EventEmitter<EventTypes>

Implementation of

RealtimeTransportLayer.once

Inherited from

OpenAIRealtimeBase.once

resetHistory()

resetHistory(oldHistory, newHistory): void

Reset the history of the conversation. This will create a diff between the old and new history and send the necessary events to the Realtime API to update the history.

Parameters

Parameter	Type	Description
`oldHistory`	`RealtimeItem`[]	The old history of the conversation.
`newHistory`	`RealtimeItem`[]	The new history of the conversation.

Returns

void

Implementation of

RealtimeTransportLayer.resetHistory

Inherited from

OpenAIRealtimeBase.resetHistory

sendAudio()

sendAudio(audio, options?): void

Send an audio buffer to the Realtime API. This is used for your client to send audio to the model to respond.

Parameters

Parameter	Type	Description
`audio`	`ArrayBuffer`	The audio buffer to send.
`options`?	{ `commit`: `boolean`; }	The options for the audio buffer.
`options.commit`?	`boolean`	‐

Returns

void

Implementation of

RealtimeTransportLayer.sendAudio

Overrides

OpenAIRealtimeBase.sendAudio

sendEvent()

sendEvent(event): void

Send an event to the Realtime API. This will stringify the event and send it directly to the API. This can be used if you want to take control over the connection and send events manually.

Parameters

Parameter	Type	Description
`event`	`RealtimeClientMessage`	The event to send.

Returns

void

Implementation of

RealtimeTransportLayer.sendEvent

Overrides

OpenAIRealtimeBase.sendEvent

sendFunctionCallOutput()

sendFunctionCallOutput(
   toolCall,
   output,
   startResponse?): void

Send the output of a function call to the Realtime API.

Parameters

Parameter	Type	Description
`toolCall`	`TransportToolCallEvent`	The tool call to send the output for.
`output`	`string`	The output of the function call.
`startResponse`?	`boolean`	Whether to start a new response after sending the output.

Returns

void

Implementation of

RealtimeTransportLayer.sendFunctionCallOutput

Inherited from

OpenAIRealtimeBase.sendFunctionCallOutput

sendMessage()

sendMessage(message, otherEventData): void

Send a message to the Realtime API. This will create a new item in the conversation and trigger a response.

Parameters

Parameter	Type	Description
`message`	`RealtimeUserInput`	The message to send.
`otherEventData`	`Record`<`string`, `any`>	Additional event data to send.

Returns

void

Implementation of

RealtimeTransportLayer.sendMessage

Inherited from

OpenAIRealtimeBase.sendMessage

updateSessionConfig()

updateSessionConfig(config): void

Updates the session config. This will merge it with the current session config with the default values and send it to the Realtime API.

Parameters

Parameter	Type	Description
`config`	`Partial`<`RealtimeSessionConfig`>	The session config to update.

Returns

void

Implementation of

RealtimeTransportLayer.updateSessionConfig

Inherited from

OpenAIRealtimeBase.updateSessionConfig