Streaming

Meeting BaaS v2 supports real-time audio streaming over WebSocket, allowing you to receive meeting audio as it happens and optionally send audio back into the meeting. This enables use cases like live transcription, real-time translation, AI-powered meeting assistants, and speaking bots.

Overview

Streaming provides:

• Output Streaming: Receive the meeting's mixed audio in real time via WebSocket
• Input Streaming: Send audio into the meeting so participants can hear it (for speaking bots, AI agents, etc.)
• Bidirectional Streaming: Combine both - receive meeting audio and speak back - using a single or two separate WebSocket connections
• Managed Real-Time Transcription: Let Meeting BaaS run real-time speech-to-text and stream transcript events to your WebSocket endpoint, with a choice of providers
• Speaker Diarization: Receive real-time speaker state updates as JSON messages alongside the audio stream
• Configurable Sample Rate: Choose from 16,000 Hz, 24,000 Hz (default), 32,000 Hz, or 48,000 Hz
• Works on All Platforms: Google Meet, Microsoft Teams, and Zoom

Enabling Streaming

To enable streaming, include streaming_enabled and streaming_config in your bot creation request:

{
  "meeting_url": "https://meet.google.com/abc-defg-hij",
  "bot_name": "AI Assistant",
  "streaming_enabled": true,
  "streaming_config": {
    "output_url": "wss://your-server.com/audio-stream",
    "input_url": null,
    "audio_frequency": 16000
  }
}

Configuration Fields

Streaming Modes

Output Only (Receive Meeting Audio)

Use this mode when you want to process meeting audio in real time - for example, to feed it into your own transcription engine, AI model, or analytics pipeline.

{
  "streaming_enabled": true,
  "streaming_config": {
    "output_url": "wss://your-server.com/audio-stream",
    "input_url": null,
    "audio_frequency": 24000
  }
}

Your WebSocket server receives:

• A handshake message (JSON) when the connection opens
• Binary audio chunks (raw PCM) every 100ms
• Speaker state updates (JSON) when speakers change

Input Only (Send Audio into the Meeting)

Use this mode when you want to inject audio into the meeting without processing the output - for example, playing pre-recorded announcements or TTS audio.

{
  "streaming_enabled": true,
  "streaming_config": {
    "output_url": null,
    "input_url": "wss://your-server.com/audio-input",
    "audio_frequency": 24000
  }
}

Your WebSocket server sends binary audio chunks to the bot, and participants in the meeting hear the audio.

Bidirectional (Receive and Send Audio)

Use this mode for interactive AI agents and speaking bots. The bot receives meeting audio, you process it (e.g., speech-to-text → LLM → text-to-speech), and send audio back.

Option A: Same URL for both directions

When input_url and output_url are the same, the bot uses a single bidirectional WebSocket connection:

{
  "streaming_enabled": true,
  "streaming_config": {
    "output_url": "wss://your-server.com/audio",
    "input_url": "wss://your-server.com/audio",
    "audio_frequency": 24000
  }
}

Option B: Separate URLs

When the URLs differ, the bot opens two separate WebSocket connections - one for sending audio to your server, and one for receiving audio from your server:

{
  "streaming_enabled": true,
  "streaming_config": {
    "output_url": "wss://your-server.com/audio-out",
    "input_url": "wss://your-server.com/audio-in",
    "audio_frequency": 24000
  }
}

Managed Real-Time Transcription

Set mode to "transcription" to have Meeting BaaS run real-time speech-to-text for you and stream transcript events to your output_url over WebSocket as the meeting happens - no need to run your own STT engine on the audio stream.

{
  "streaming_enabled": true,
  "streaming_config": {
    "mode": "transcription",
    "output_url": "wss://your-server.com/transcripts",
    "transcription": {
      "provider": "gladia",
      "api_key": null,
      "custom_params": null,
      "region": null
    }
  }
}

The streaming_config.transcription object configures the real-time STT provider:

Custom Parameters (live vs. batch)

custom_params is forwarded to the provider's live session API - for Gladia, that is POST /v2/live, not the pre-recorded API used for batch transcription. The two APIs accept different shapes, and reusing batch-shaped params is the most common mistake: for example, Gladia's live API nests translation under realtime_processing, while the batch API takes translation_config at the top level.

{
  "streaming_config": {
    "mode": "transcription",
    "output_url": "wss://your-server.com/transcripts",
    "transcription": {
      "provider": "gladia",
      "custom_params": {
        "language_config": { "languages": ["ru"] },
        "realtime_processing": {
          "translation": true,
          "translation_config": { "target_languages": ["en", "de", "it"] }
        }
      }
    }
  }
}

custom_params is validated against the provider's live schema when you create the bot - unknown fields are rejected with a 400 that points at the correct live-API location where one exists (e.g. translation_config → realtime_processing.translation_config).

The same batch-vs-live distinction applies to every provider - always use the provider's real-time/streaming parameter reference, not the pre-recorded one:

Transcription Session Events

In transcription mode, the bot sends three event types to output_url, all sharing the { "event", "bot_id", "data" } envelope:

If you receive error, live transcription is down for the rest of the meeting - the message field carries the provider's reason (e.g. rejected parameters). Recording and batch transcription are unaffected. Treat a connection that never receives session.started as not yet live rather than silent.

The bot also sends standard WebSocket ping frames roughly every 30 seconds so intermediaries (e.g. Cloudflare tunnels, which drop idle connections after ~100s) keep the connection open through quiet stretches of the meeting. Most WebSocket libraries answer pings automatically - no action needed.

Transcript Events

Your WebSocket server receives JSON text messages, one per transcript segment. Every message has the same envelope:

{
  "event": "transcript.segment",
  "bot_id": "123e4567-e89b-12d3-a456-426614174000",
  "data": {
    "text": "Hello everyone, let's get started.",
    "isFinal": true,
    "utteranceStart": 12.34,
    "utteranceEnd": 15.02,
    "confidence": 0.97,
    "words": [
      { "text": "Hello", "start": 12.34, "end": 12.61, "confidence": 0.98 }
    ],
    "speaker": { "name": "John Doe", "id": 1 }
  }
}

The event and bot_id envelope fields are always present. Fields under data are provider-dependent - treat all of them as optional.

Partial vs. Final Segments

Messages carry no explicit segment or utterance identifier. Partial segments (isFinal: false) are progressive snapshots of the utterance currently being spoken - each new partial for that utterance replaces the previous one, and the final segment (isFinal: true) supersedes all partials for it. Correlate them by time: partials and their final cover overlapping utteranceStart/utteranceEnd ranges. The simplest robust approach is to use partials for live display only (always replacing the last partial shown) and build your stored transcript exclusively from isFinal: true segments.

If the WebSocket connection drops, the bot reconnects with exponential backoff (1s doubling up to 60s) and buffers up to 100 transcript events while disconnected, flushing them on reconnect. Events beyond the buffer limit are dropped.

WebSocket Protocol

Connection Lifecycle

1. The bot joins the meeting and establishes WebSocket connection(s) to your server
2. Immediately sends a handshake message (JSON text) on the output connection
3. Begins streaming binary audio chunks every 100ms
4. Sends speaker state updates (JSON text) whenever the active speakers change
5. On the input connection, the bot listens for binary audio chunks from your server
6. When the meeting ends or the bot leaves, the WebSocket connections close

Handshake Message

When the output WebSocket connection opens, the bot sends a JSON text message:

{
  "protocol_version": 2,
  "bot_id": "123e4567-e89b-12d3-a456-426614174000",
  "offset": 0.0,
  "sample_rate": 24000,
  "start_time": null
}

Use this message to initialize your audio processing pipeline with the correct sample rate and to associate the stream with a specific bot.

To timestamp the audio stream: audio_time_ms = handshake_receipt_time_ms + (offset + cumulative_samples / sample_rate) * 1000. Audio capture starts when the bot opens the meeting page, which is later than joined_at in the bot details. On reconnection the bot sends a new handshake with offset reset to 0.0, so restart the clock; audio during the disconnection is dropped.

Output Audio Chunks (Bot → Your Server)

After the handshake, the bot sends binary WebSocket messages containing raw audio data:

The audio is the mixed meeting audio - all participants' audio combined into a single mono stream. Each chunk represents exactly 100 milliseconds of audio.

Speaker State Updates (Bot → Your Server)

Alongside audio chunks, the bot sends JSON text messages with real-time speaker information whenever the active speakers change:

[
  {
    "name": "John Doe",
    "id": 1,
    "timestamp": 1788284782279,
    "isSpeaking": true
  },
  {
    "name": "Jane Smith",
    "id": 2,
    "timestamp": 1788284782279,
    "isSpeaking": false
  }
]

These updates are sent on the output WebSocket as JSON text messages. Your server can distinguish them from audio chunks by checking the WebSocket message type: text messages are speaker state, binary messages are audio.

Input Audio Chunks (Your Server → Bot)

To send audio into the meeting, your server sends binary WebSocket messages on the input connection:

The bot receives these chunks and plays them into the meeting - all participants will hear the audio. There is no strict chunk size requirement for input audio, but sending in consistent intervals (e.g., every 20-100ms) produces the smoothest playback.

Reconnection

The bot automatically reconnects to your WebSocket server if the connection drops:

• Uses exponential backoff: 1s, 2s, 4s, 8s, ... up to 60s maximum
• Resends the handshake message after reconnecting
• Audio chunks sent during disconnection are not buffered - they are dropped

Your WebSocket server should be prepared to receive a new handshake message at any time, indicating a reconnection.

Examples

Creating a Bot with Output Streaming

curl -X POST "https://api.meetingbaas.com/v2/bots" \
     -H "Content-Type: application/json" \
     -H "x-meeting-baas-api-key: YOUR-API-KEY" \
     -d '{
           "meeting_url": "https://meet.google.com/abc-defg-hij",
           "bot_name": "Audio Listener",
           "streaming_enabled": true,
           "streaming_config": {
             "output_url": "wss://your-server.com/audio-stream",
             "input_url": null,
             "audio_frequency": 24000
           }
         }'

import requests

response = requests.post(
    "https://api.meetingbaas.com/v2/bots",
    headers={
        "Content-Type": "application/json",
        "x-meeting-baas-api-key": "YOUR-API-KEY",
    },
    json={
        "meeting_url": "https://meet.google.com/abc-defg-hij",
        "bot_name": "Audio Listener",
        "streaming_enabled": True,
        "streaming_config": {
            "output_url": "wss://your-server.com/audio-stream",
            "input_url": None,
            "audio_frequency": 24000,
        },
    },
)
print(response.json())

fetch("https://api.meetingbaas.com/v2/bots", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-meeting-baas-api-key": "YOUR-API-KEY",
  },
  body: JSON.stringify({
    meeting_url: "https://meet.google.com/abc-defg-hij",
    bot_name: "Audio Listener",
    streaming_enabled: true,
    streaming_config: {
      output_url: "wss://your-server.com/audio-stream",
      input_url: null,
      audio_frequency: 24000,
    },
  }),
})
  .then((response) => response.json())
  .then((data) => console.log(data.data.bot_id));

Creating a Bidirectional Speaking Bot

curl -X POST "https://api.meetingbaas.com/v2/bots" \
     -H "Content-Type: application/json" \
     -H "x-meeting-baas-api-key: YOUR-API-KEY" \
     -d '{
           "meeting_url": "https://meet.google.com/abc-defg-hij",
           "bot_name": "AI Meeting Assistant",
           "streaming_enabled": true,
           "streaming_config": {
             "output_url": "wss://your-server.com/audio",
             "input_url": "wss://your-server.com/audio",
             "audio_frequency": 24000
           }
         }'

import requests

response = requests.post(
    "https://api.meetingbaas.com/v2/bots",
    headers={
        "Content-Type": "application/json",
        "x-meeting-baas-api-key": "YOUR-API-KEY",
    },
    json={
        "meeting_url": "https://meet.google.com/abc-defg-hij",
        "bot_name": "AI Meeting Assistant",
        "streaming_enabled": True,
        "streaming_config": {
            "output_url": "wss://your-server.com/audio",
            "input_url": "wss://your-server.com/audio",
            "audio_frequency": 24000,
        },
    },
)
print(response.json())

fetch("https://api.meetingbaas.com/v2/bots", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-meeting-baas-api-key": "YOUR-API-KEY",
  },
  body: JSON.stringify({
    meeting_url: "https://meet.google.com/abc-defg-hij",
    bot_name: "AI Meeting Assistant",
    streaming_enabled: true,
    streaming_config: {
      output_url: "wss://your-server.com/audio",
      input_url: "wss://your-server.com/audio",
      audio_frequency: 24000,
    },
  }),
})
  .then((response) => response.json())
  .then((data) => console.log(data.data.bot_id));

WebSocket Server (Receiving Audio)

Here's how to build a WebSocket server that receives and processes the stream:

import asyncio
import json
import numpy as np
import websockets

async def handle_stream(websocket):
    async for message in websocket:
        if isinstance(message, str):
            # JSON message - either handshake or speaker state
            data = json.loads(message)

            if "protocol_version" in data:
                # Handshake message
                print(f"Bot connected: {data['bot_id']}")
                print(f"Sample rate: {data['sample_rate']} Hz")
            else:
                # Speaker state update
                for speaker in data:
                    status = "speaking" if speaker["isSpeaking"] else "silent"
                    print(f"{speaker['name']}: {status}")

        elif isinstance(message, bytes):
            # Binary message - raw Int16 PCM audio
            audio = np.frombuffer(message, dtype=np.int16)
            print(f"Audio chunk: {len(audio)} samples, "
                  f"duration: {len(audio) / 24000 * 1000:.0f}ms")

            # Process the audio (e.g., feed to STT, analyze, store)
            # audio is a numpy array of signed 16-bit integers

async def main():
    async with websockets.serve(handle_stream, "0.0.0.0", 8765):
        print("WebSocket server running on ws://0.0.0.0:8765")
        await asyncio.Future()  # Run forever

asyncio.run(main())

const { WebSocketServer } = require("ws");

const wss = new WebSocketServer({ port: 8765 });

wss.on("connection", (ws) => {
  console.log("Bot connected");

  ws.on("message", (message, isBinary) => {
    if (!isBinary) {
      // JSON message - either handshake or speaker state
      const data = JSON.parse(message.toString());

      if (data.protocol_version) {
        // Handshake message
        console.log(`Bot ID: ${data.bot_id}`);
        console.log(`Sample rate: ${data.sample_rate} Hz`);
      } else {
        // Speaker state update
        data.forEach((speaker) => {
          const status = speaker.isSpeaking ? "speaking" : "silent";
          console.log(`${speaker.name}: ${status}`);
        });
      }
    } else {
      // Binary message - raw Int16 PCM audio
      const audio = new Int16Array(
        message.buffer,
        message.byteOffset,
        message.byteLength / 2
      );
      console.log(
        `Audio chunk: ${audio.length} samples, ` +
        `duration: ${(audio.length / 24000) * 1000}ms`
      );

      // Process the audio (e.g., feed to STT, analyze, store)
    }
  });

  ws.on("close", () => console.log("Bot disconnected"));
});

console.log("WebSocket server running on ws://0.0.0.0:8765");

WebSocket Server (Bidirectional)

For a bidirectional setup where you receive audio, process it, and send audio back:

import asyncio
import json
import numpy as np
import websockets

SAMPLE_RATE = 24000

async def handle_bidirectional(websocket):
    async for message in websocket:
        if isinstance(message, str):
            data = json.loads(message)
            if "protocol_version" in data:
                print(f"Bot connected: {data['bot_id']}")
                continue
            # Speaker state update
            for speaker in data:
                if speaker["isSpeaking"]:
                    print(f"Now speaking: {speaker['name']}")
            continue

        # Binary audio from the meeting
        audio_in = np.frombuffer(message, dtype=np.int16)

        # --- Your processing pipeline here ---
        # Example: speech-to-text → LLM → text-to-speech
        # audio_out = your_pipeline(audio_in)

        # Send audio back into the meeting (Int16 PCM)
        # await websocket.send(audio_out.tobytes())

async def main():
    async with websockets.serve(handle_bidirectional, "0.0.0.0", 8765):
        print("Bidirectional WebSocket server on ws://0.0.0.0:8765")
        await asyncio.Future()

asyncio.run(main())

const { WebSocketServer } = require("ws");

const SAMPLE_RATE = 24000;
const wss = new WebSocketServer({ port: 8765 });

wss.on("connection", (ws) => {
  console.log("Bot connected");

  ws.on("message", (message, isBinary) => {
    if (!isBinary) {
      const data = JSON.parse(message.toString());
      if (data.protocol_version) {
        console.log(`Bot ID: ${data.bot_id}`);
        return;
      }
      // Speaker state update
      data.forEach((s) => {
        if (s.isSpeaking) console.log(`Now speaking: ${s.name}`);
      });
      return;
    }

    // Binary audio from the meeting
    const audioIn = new Int16Array(
      message.buffer,
      message.byteOffset,
      message.byteLength / 2
    );

    // --- Your processing pipeline here ---
    // Example: speech-to-text → LLM → text-to-speech
    // const audioOut = yourPipeline(audioIn);

    // Send audio back into the meeting (Int16 PCM)
    // ws.send(Buffer.from(audioOut.buffer));
  });

  ws.on("close", () => console.log("Bot disconnected"));
});

console.log("Bidirectional WebSocket server on ws://0.0.0.0:8765");

Combining Streaming with Recording and Transcription

Streaming works independently from recording and transcription. You can enable all three at once:

{
  "meeting_url": "https://meet.google.com/abc-defg-hij",
  "bot_name": "Full-Featured Bot",
  "recording_mode": "speaker_view",
  "transcription_enabled": true,
  "transcription_config": {
    "provider": "gladia"
  },
  "streaming_enabled": true,
  "streaming_config": {
    "output_url": "wss://your-server.com/audio-stream",
    "input_url": null,
    "audio_frequency": 24000
  }
}

The recording, transcription, and streaming pipelines operate independently - enabling streaming does not affect recording quality or transcription accuracy.

Error Handling

Currently, we don't give any feedback on errors with the websocket connection or invalid message formats. We plan to improve this in the future.

Troubleshooting:

• Verify your WebSocket server is running and accessible from the internet
• Ensure the URL uses wss:// for secure WebSocket connections
• Check that your server accepts WebSocket upgrade requests
• Verify there are no firewall rules blocking the connection

Connection Drops

If the WebSocket connection drops during a meeting, the bot will automatically attempt to reconnect with exponential backoff. Audio chunks during the disconnection period are lost and not buffered.

Your server should handle reconnection gracefully - when the bot reconnects, it sends a fresh handshake message.

Best Practices

1. Use wss:// endpoints: We recommend using secure WebSocket connections with valid TLS certificates.
2. Handle reconnections: Your server should accept new handshake messages at any time, as the bot reconnects automatically on connection drops.
3. Process audio asynchronously: Audio chunks arrive every 100ms. Ensure your processing pipeline can keep up to avoid backpressure.
4. Match sample rates: When sending audio back (input streaming), always use the same sample rate configured in audio_frequency. Mismatched rates cause distorted audio.
5. Distinguish message types: Use the WebSocket message type to differentiate - binary for audio, text for JSON (handshake and speaker state).
6. Keep connections alive: The bot expects the WebSocket connection to remain open. Avoid closing the connection from your server while the meeting is active.
7. Monitor speaker state: Use speaker state updates to know who is talking - this is useful for building real-time diarization or triggering AI responses to specific speakers.

Frequently Asked Questions

Next Steps

• Send a Bot to get started with the API
• Set up Webhooks to receive bot status notifications
• Explore Speaking Bots for a ready-made AI meeting agent framework
• Check the API Reference for complete parameter documentation