OpenAIRealtimeWebRTC
Transport layer that’s handling the connection between the client and OpenAI’s Realtime API via WebRTC. While this transport layer is designed to be used within a RealtimeSession, it can also be used standalone if you want to have a direct connection to the Realtime API.
Unless you specify a mediaStream
or audioElement
option, the transport layer will
automatically configure the microphone and audio output to be used by the session.
Extends
Section titled “Extends”Implements
Section titled “Implements”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new OpenAIRealtimeWebRTC(options?): OpenAIRealtimeWebRTC
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
Returns
Section titled “Returns”OpenAIRealtimeWebRTC
Overrides
Section titled “Overrides”OpenAIRealtimeBase
.constructor
Accessors
Section titled “Accessors”_tracingConfig
Section titled “_tracingConfig”Set Signature
Section titled “Set Signature”set _tracingConfig(tracingConfig): void
Sets the internal tracing config. This is used to track the tracing config that has been set during the session.create event.
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
|
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”OpenAIRealtimeBase
._tracingConfig
connectionState
Section titled “connectionState”Get Signature
Section titled “Get Signature”get connectionState(): WebRTCState
The current connection state of the WebRTC connection including the peer connection and data channel.
Returns
Section titled “Returns”currentModel
Section titled “currentModel”Get Signature
Section titled “Get Signature”get currentModel(): OpenAIRealtimeModels
The current model that is being used by the transport layer.
Returns
Section titled “Returns”Set Signature
Section titled “Set Signature”set currentModel(model): void
The current model that is being used by the transport layer. Note: The model cannot be changed mid conversation.
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”OpenAIRealtimeBase
.currentModel
Get Signature
Section titled “Get Signature”get muted(): boolean
Whether the session is muted.
Returns
Section titled “Returns”boolean
Whether the input audio track is currently muted null if the muting is not handled by the transport layer
Implementation of
Section titled “Implementation of”Overrides
Section titled “Overrides”status
Section titled “status”Get Signature
Section titled “Get Signature”get status(): "connecting" | "connected" | "disconnected"
The current status of the WebRTC connection.
Returns
Section titled “Returns”"connecting"
| "connected"
| "disconnected"
Implementation of
Section titled “Implementation of”Overrides
Section titled “Overrides”Methods
Section titled “Methods”close()
Section titled “close()”close(): void
Close the connection to the Realtime API and disconnects the underlying WebRTC connection.
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”Overrides
Section titled “Overrides”connect()
Section titled “connect()”connect(options): Promise<void>
Connect to the Realtime API. This will establish the connection to the OpenAI Realtime API via WebRTC.
If you are using a browser, the transport layer will also automatically configure the microphone and audio output to be used by the session.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
The options for the connection. |
Returns
Section titled “Returns”Promise
<void
>
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.connect
Overrides
Section titled “Overrides”emit()
Section titled “emit()”emit<K>(type, ...args): boolean
Type Parameters
Section titled “Type Parameters”Type Parameter |
---|
|
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
|
… |
Returns
Section titled “Returns”boolean
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”interrupt()
Section titled “interrupt()”interrupt(): void
Interrupt the current response if one is ongoing and clear the audio buffer so that the agent stops talking.
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.interrupt
Overrides
Section titled “Overrides”mute()
Section titled “mute()”mute(muted): void
Mute or unmute the session.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
|
Whether to mute the session. |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”Overrides
Section titled “Overrides”off<K>(type, listener): EventEmitter<EventTypes>
Type Parameters
Section titled “Type Parameters”Type Parameter |
---|
|
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
|
|
(… |
Returns
Section titled “Returns”EventEmitter
<EventTypes
>
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”on<K>(type, listener): EventEmitter<EventTypes>
Type Parameters
Section titled “Type Parameters”Type Parameter |
---|
|
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
|
|
(… |
Returns
Section titled “Returns”EventEmitter
<EventTypes
>
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”once()
Section titled “once()”once<K>(type, listener): EventEmitter<EventTypes>
Type Parameters
Section titled “Type Parameters”Type Parameter |
---|
|
Parameters
Section titled “Parameters”Parameter | Type |
---|---|
|
|
|
(… |
Returns
Section titled “Returns”EventEmitter
<EventTypes
>
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”resetHistory()
Section titled “resetHistory()”resetHistory(oldHistory, newHistory): void
Reset the history of the conversation. This will create a diff between the old and new history and send the necessary events to the Realtime API to update the history.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
The old history of the conversation. |
|
|
The new history of the conversation. |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.resetHistory
Inherited from
Section titled “Inherited from”OpenAIRealtimeBase
.resetHistory
sendAudio()
Section titled “sendAudio()”sendAudio(audio, options?): void
Send an audio buffer to the Realtime API. If { commit: true }
is passed, the audio buffer
will be committed and the model will start processing it. This is necessary if you have
disabled turn detection / voice activity detection (VAD).
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
|
The audio buffer to send. |
|
{ |
The options for the audio buffer. |
|
|
‐ |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.sendAudio
Inherited from
Section titled “Inherited from”sendEvent()
Section titled “sendEvent()”sendEvent(event): void
Send an event to the Realtime API. This will stringify the event and send it directly to the API. This can be used if you want to take control over the connection and send events manually.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
The event to send. |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.sendEvent
Overrides
Section titled “Overrides”sendFunctionCallOutput()
Section titled “sendFunctionCallOutput()”sendFunctionCallOutput( toolCall, output, startResponse?): void
Send the output of a function call to the Realtime API.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
The tool call to send the output for. |
|
|
|
The output of the function call. |
|
|
Whether to start a new response after sending the output. |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.sendFunctionCallOutput
Inherited from
Section titled “Inherited from”OpenAIRealtimeBase
.sendFunctionCallOutput
sendMessage()
Section titled “sendMessage()”sendMessage(message, otherEventData): void
Send a message to the Realtime API. This will create a new item in the conversation and trigger a response.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
|
The message to send. |
|
|
Additional event data to send. |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.sendMessage
Inherited from
Section titled “Inherited from”OpenAIRealtimeBase
.sendMessage
updateSessionConfig()
Section titled “updateSessionConfig()”updateSessionConfig(config): void
Updates the session config. This will merge it with the current session config with the default values and send it to the Realtime API.
Parameters
Section titled “Parameters”Parameter | Type | Description |
---|---|---|
|
|
The session config to update. |
Returns
Section titled “Returns”void
Implementation of
Section titled “Implementation of”RealtimeTransportLayer
.updateSessionConfig