Constructor Detail

Method Detail

• visitConversationCreated

   abstract T visitConversationCreated(ConversationCreatedEvent conversationCreated)

  Returned when a conversation is created. Emitted right after session creation.

• visitConversationItemCreated

   abstract T visitConversationItemCreated(ConversationItemCreatedEvent conversationItemCreated)

  Returned when a conversation item is created. There are several scenarios that produce this event:

  • The server is generating a Response, which if successful will produce either one or two Items, which will be of type message (role assistant) or type function_call.

  • The input audio buffer has been committed, either by the client or the server (in server_vad mode). The server will take the content of the input audio buffer and add it to a new user message Item.

  • The client has sent a conversation.item.create event to add a new Item to the Conversation.

• visitConversationItemDeleted

   abstract T visitConversationItemDeleted(ConversationItemDeletedEvent conversationItemDeleted)

  Returned when an item in the conversation is deleted by the client with a conversation.item.delete event. This event is used to synchronize the server's understanding of the conversation history with the client's view.

• visitConversationItemInputAudioTranscriptionCompleted

   abstract T visitConversationItemInputAudioTranscriptionCompleted(ConversationItemInputAudioTranscriptionCompletedEvent conversationItemInputAudioTranscriptionCompleted)

  This event is the output of audio transcription for user audio written to the user audio buffer. Transcription begins when the input audio buffer is committed by the client or server (in server_vad mode). Transcription runs asynchronously with Response creation, so this event may come before or after the Response events.

  Realtime API models accept audio natively, and thus input transcription is a separate process run on a separate ASR (Automatic Speech Recognition) model. The transcript may diverge somewhat from the model's interpretation, and should be treated as a rough guide.

• visitConversationItemInputAudioTranscriptionDelta

   abstract T visitConversationItemInputAudioTranscriptionDelta(ConversationItemInputAudioTranscriptionDeltaEvent conversationItemInputAudioTranscriptionDelta)

  Returned when the text value of an input audio transcription content part is updated.

• visitConversationItemInputAudioTranscriptionFailed

   abstract T visitConversationItemInputAudioTranscriptionFailed(ConversationItemInputAudioTranscriptionFailedEvent conversationItemInputAudioTranscriptionFailed)

  Returned when input audio transcription is configured, and a transcription request for a user message failed. These events are separate from other error events so that the client can identify the related Item.

• visitConversationItemRetrieved

   abstract T visitConversationItemRetrieved(RealtimeServerEvent.ConversationItemRetrieved conversationItemRetrieved)

  Returned when a conversation item is retrieved with conversation.item.retrieve.

• visitConversationItemTruncated

   abstract T visitConversationItemTruncated(ConversationItemTruncatedEvent conversationItemTruncated)

  Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event. This event is used to synchronize the server's understanding of the audio with the client's playback.

  This action will truncate the audio and remove the server-side text transcript to ensure there is no text in the context that hasn't been heard by the user.

• visitError

   abstract T visitError(RealtimeErrorEvent error)

  Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementors to monitor and log error messages by default.

• visitInputAudioBufferCleared

   abstract T visitInputAudioBufferCleared(InputAudioBufferClearedEvent inputAudioBufferCleared)

  Returned when the input audio buffer is cleared by the client with a input_audio_buffer.clear event.

• visitInputAudioBufferCommitted

   abstract T visitInputAudioBufferCommitted(InputAudioBufferCommittedEvent inputAudioBufferCommitted)

  Returned when an input audio buffer is committed, either by the client or automatically in server VAD mode. The item_id property is the ID of the user message item that will be created, thus a conversation.item.created event will also be sent to the client.

• visitInputAudioBufferSpeechStarted

   abstract T visitInputAudioBufferSpeechStarted(InputAudioBufferSpeechStartedEvent inputAudioBufferSpeechStarted)

  Sent by the server when in server_vad mode to indicate that speech has been detected in the audio buffer. This can happen any time audio is added to the buffer (unless speech is already detected). The client may want to use this event to interrupt audio playback or provide visual feedback to the user.

  The client should expect to receive a input_audio_buffer.speech_stopped event when speech stops. The item_id property is the ID of the user message item that will be created when speech stops and will also be included in the input_audio_buffer.speech_stopped event (unless the client manually commits the audio buffer during VAD activation).

• visitInputAudioBufferSpeechStopped

   abstract T visitInputAudioBufferSpeechStopped(InputAudioBufferSpeechStoppedEvent inputAudioBufferSpeechStopped)

  Returned in server_vad mode when the server detects the end of speech in the audio buffer. The server will also send an conversation.item.created event with the user message item that is created from the audio buffer.

• visitRateLimitsUpdated

   abstract T visitRateLimitsUpdated(RateLimitsUpdatedEvent rateLimitsUpdated)

  Emitted at the beginning of a Response to indicate the updated rate limits. When a Response is created some tokens will be "reserved" for the output tokens, the rate limits shown here reflect that reservation, which is then adjusted accordingly once the Response is completed.

• visitResponseOutputAudioDelta

   abstract T visitResponseOutputAudioDelta(ResponseAudioDeltaEvent responseOutputAudioDelta)

  Returned when the model-generated audio is updated.

• visitResponseOutputAudioDone

   abstract T visitResponseOutputAudioDone(ResponseAudioDoneEvent responseOutputAudioDone)

  Returned when the model-generated audio is done. Also emitted when a Response is interrupted, incomplete, or cancelled.

• visitResponseOutputAudioTranscriptDelta

   abstract T visitResponseOutputAudioTranscriptDelta(ResponseAudioTranscriptDeltaEvent responseOutputAudioTranscriptDelta)

  Returned when the model-generated transcription of audio output is updated.

• visitResponseOutputAudioTranscriptDone

   abstract T visitResponseOutputAudioTranscriptDone(ResponseAudioTranscriptDoneEvent responseOutputAudioTranscriptDone)

  Returned when the model-generated transcription of audio output is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

• visitResponseContentPartAdded

   abstract T visitResponseContentPartAdded(ResponseContentPartAddedEvent responseContentPartAdded)

  Returned when a new content part is added to an assistant message item during response generation.

• visitResponseContentPartDone

   abstract T visitResponseContentPartDone(ResponseContentPartDoneEvent responseContentPartDone)

  Returned when a content part is done streaming in an assistant message item. Also emitted when a Response is interrupted, incomplete, or cancelled.

• visitResponseCreated

   abstract T visitResponseCreated(ResponseCreatedEvent responseCreated)

  Returned when a new Response is created. The first event of response creation, where the response is in an initial state of in_progress.

• visitResponseDone

   abstract T visitResponseDone(ResponseDoneEvent responseDone)

  Returned when a Response is done streaming. Always emitted, no matter the final state. The Response object included in the response.done event will include all output Items in the Response but will omit the raw audio data.

• visitResponseFunctionCallArgumentsDelta

   abstract T visitResponseFunctionCallArgumentsDelta(ResponseFunctionCallArgumentsDeltaEvent responseFunctionCallArgumentsDelta)

  Returned when the model-generated function call arguments are updated.

• visitResponseFunctionCallArgumentsDone

   abstract T visitResponseFunctionCallArgumentsDone(ResponseFunctionCallArgumentsDoneEvent responseFunctionCallArgumentsDone)

  Returned when the model-generated function call arguments are done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

• visitResponseOutputItemAdded

   abstract T visitResponseOutputItemAdded(ResponseOutputItemAddedEvent responseOutputItemAdded)

  Returned when a new Item is created during Response generation.

• visitResponseOutputItemDone

   abstract T visitResponseOutputItemDone(ResponseOutputItemDoneEvent responseOutputItemDone)

  Returned when an Item is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

• visitResponseOutputTextDelta

   abstract T visitResponseOutputTextDelta(ResponseTextDeltaEvent responseOutputTextDelta)

  Returned when the text value of an "output_text" content part is updated.

• visitResponseOutputTextDone

   abstract T visitResponseOutputTextDone(ResponseTextDoneEvent responseOutputTextDone)

  Returned when the text value of an "output_text" content part is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

• visitSessionCreated

   abstract T visitSessionCreated(SessionCreatedEvent sessionCreated)

  Returned when a Session is created. Emitted automatically when a new connection is established as the first server event. This event will contain the default Session configuration.

• visitSessionUpdated

   abstract T visitSessionUpdated(SessionUpdatedEvent sessionUpdated)

  Returned when a session is updated with a session.update event, unless there is an error.

• visitTranscriptionSessionUpdated

   abstract T visitTranscriptionSessionUpdated(TranscriptionSessionUpdatedEvent transcriptionSessionUpdated)

  Returned when a transcription session is updated with a transcription_session.update event, unless there is an error.

• visitTranscriptionSessionCreated

   abstract T visitTranscriptionSessionCreated(TranscriptionSessionCreated transcriptionSessionCreated)

  Returned when a transcription session is created.

• visitOutputAudioBufferStarted

   abstract T visitOutputAudioBufferStarted(RealtimeServerEvent.OutputAudioBufferStarted outputAudioBufferStarted)

  WebRTC Only: Emitted when the server begins streaming audio to the client. This event is emitted after an audio content part has been added (response.content_part.added) to the response. Learn more.

• visitOutputAudioBufferStopped

   abstract T visitOutputAudioBufferStopped(RealtimeServerEvent.OutputAudioBufferStopped outputAudioBufferStopped)

  WebRTC Only: Emitted when the output audio buffer has been completely drained on the server, and no more audio is forthcoming. This event is emitted after the full response data has been sent to the client (response.done). Learn more.

• visitOutputAudioBufferCleared

   abstract T visitOutputAudioBufferCleared(RealtimeServerEvent.OutputAudioBufferCleared outputAudioBufferCleared)

  WebRTC Only: Emitted when the output audio buffer is cleared. This happens either in VAD mode when the user has interrupted (input_audio_buffer.speech_started), or when the client has emitted the output_audio_buffer.clear event to manually cut off the current audio response. Learn more.

• visitConversationItemAdded

   abstract T visitConversationItemAdded(ConversationItemAdded conversationItemAdded)

  Returned when a conversation item is added.

• visitConversationItemDone

   abstract T visitConversationItemDone(ConversationItemDone conversationItemDone)

  Returned when a conversation item is finalized.

• visitInputAudioBufferTimeoutTriggered

   abstract T visitInputAudioBufferTimeoutTriggered(InputAudioBufferTimeoutTriggered inputAudioBufferTimeoutTriggered)

  Returned when the server VAD timeout is triggered for the input audio buffer.

• visitConversationItemInputAudioTranscriptionSegment

   abstract T visitConversationItemInputAudioTranscriptionSegment(ConversationItemInputAudioTranscriptionSegment conversationItemInputAudioTranscriptionSegment)

  Returned when an input audio transcription segment is identified for an item.

• visitMcpListToolsInProgress

   abstract T visitMcpListToolsInProgress(McpListToolsInProgress mcpListToolsInProgress)

  Returned when listing MCP tools is in progress for an item.

• visitMcpListToolsCompleted

   abstract T visitMcpListToolsCompleted(McpListToolsCompleted mcpListToolsCompleted)

  Returned when listing MCP tools has completed for an item.

• visitMcpListToolsFailed

   abstract T visitMcpListToolsFailed(McpListToolsFailed mcpListToolsFailed)

  Returned when listing MCP tools has failed for an item.

• visitResponseMcpCallArgumentsDelta

   abstract T visitResponseMcpCallArgumentsDelta(ResponseMcpCallArgumentsDelta responseMcpCallArgumentsDelta)

  Returned when MCP tool call arguments are updated during response generation.

• visitResponseMcpCallArgumentsDone

   abstract T visitResponseMcpCallArgumentsDone(ResponseMcpCallArgumentsDone responseMcpCallArgumentsDone)

  Returned when MCP tool call arguments are finalized during response generation.

• visitResponseMcpCallInProgress

   abstract T visitResponseMcpCallInProgress(ResponseMcpCallInProgress responseMcpCallInProgress)

  Returned when an MCP tool call has started and is in progress.

• visitResponseMcpCallCompleted

   abstract T visitResponseMcpCallCompleted(ResponseMcpCallCompleted responseMcpCallCompleted)

  Returned when an MCP tool call has completed successfully.

• visitResponseMcpCallFailed

   abstract T visitResponseMcpCallFailed(ResponseMcpCallFailed responseMcpCallFailed)

  Returned when an MCP tool call has failed.

• unknown

   T unknown(JsonValue json)

  Maps an unknown variant of RealtimeServerEvent to a value of type T.

  An instance of RealtimeServerEvent can contain an unknown variant if it was deserialized from data that doesn't match any known variant. For example, if the SDK is on an older version than the API, then the API may respond with new variants that the SDK is unaware of.