WebRTC.ventures

Up next! WebRTC Live Episode 115: Real lessons on multi-agent orchestration, streaming avatars, and the design tradeoffs behind a live training simulator.

How can AI scale clinical roleplay training beyond what live sessions with human trainers can deliver? For CETA Global, a nonprofit that trains frontline mental health providers around the world, the answer was an AI “flight simulator”: a real-time platform where psychologists practice difficult client sessions with an AI-generated patient, complete with a lip-synced streaming avatar, live coaching, and replayable feedback. Behind the scenes, a team of six AI agents works in concert to play the patient, coach the trainee, and evaluate the session in real time. In this episode of WebRTC Live, host Arin Sime sits down with Andres Rincon, the WebRTC.ventures lead developer on the project, for an inside look at EBTSim, which went from concept to working platform in just eight weeks for a Google.org Accelerator: Generative AI demo. They’ll dig into what it takes to make concurrent AI feel like a single conversation: a WebSocket-based real-time communication layer, purpose-built Gemini agents orchestrated through Google ADK, low-latency speech-to-text and text-to-speech, and HeyGen streaming avatars. Andrés will also share the design decisions that shaped the experience, including why the team chose push-to-talk over automatic turn detection, and how they made the AI client’s resistance adaptive rather than scripted. If you’re building voice AI applications, multi-agent experiences, or training simulators, this episode is full of practical lessons on video avatars, agent orchestration, live coaching, and automated assessment that help professionals build and demonstrate competency in realistic scenarios. Can't attend? Register at https://lnkd.in/e3eaEH5q and we'll send you the recording and highlights.

Our WebRTC Developer Advocate, Hector Zelaya, makes his debut on ClueCon Weekly! Check out his episode: Voice AI in Practice, From Tutorial to Production.

WebRTC.ventures recently completed a Kurento to LiveKit migration for a client running an extraordinarily complex real-time communication application, without downtime. The platform included conferences, webinars, inbound and outbound telephony, recordings, and live transcription and translation in English and Spanish. All in production, serving real users every day throughout the migration. We solved the main challenges we set out to address: scale limitations, cost, fragility, and unlocking of AI capabilities that would have required significant rework on the old stack. The full technical case study by Alfred Gonzalez Trastoy covers the evaluation of Kurento alternatives, the before-and-after architecture, the phased rollout, and the production pitfalls to avoid in your own migration. It's now available to download. https://lnkd.in/embxmaeP

In the Janus WebRTC Server, a room lives in the memory of exactly one instance. That single constraint shapes everything about scaling Janus. In an SFU, each room’s media paths converge on its host, and ICE candidates pin each browser to that specific instance. Rooms don’t migrate. That means something external has to decide where a room lives and remember it. Janus won’t do this for you. There’s no built-in clustering, and coordination is intentionally left to the integrator. The solution is a Media Resource Broker: a thin control layer that turns a pool of independent Janus instances into a single elastic service. The broker has three parts: - A stateless router exposing an HTTP API - A directory store (Redis) that maps rooms to instances - A pool of Janus instances, where each instance is individually addressable (VMs, bare compute instances, or Kubernetes) - When a room is created, the router selects the least-loaded instance and persists the mapping. If multiple router replicas race to assign the same room, the system resolves conflicts and cleans up orphaned sessions. One configuration trap can block everything: the Janus admin API is disabled by default. Until admin_http is enabled, every create fails—with symptoms that point everywhere except the root cause. Scaling is asymmetric. Adding capacity is immediate; removing it is not. A live room dies with its instance, so nodes must be drained before termination. In practice, that means marking instances, letting rooms expire, and enforcing safe shutdown with OnDelete strategies, preStop hooks, and autoscaling guardrails. Full build details in this post by WebRTC.ventures DevOps Engineer, Andere Collins. https://lnkd.in/eQu-Qrqv

As our presence in Spain continues to grow, with an expanding local team and clients in the region, we're looking forward to connecting with fellow real-time communications and AI professionals at the AI Tinkerers Madrid Meetup on Tuesday, July 7. If you're working on AI-powered voice, video, or real-time communication technologies, we'd love to meet you. WebRTC.ventures is proud to co-sponsor this event alongside our new friends at BTS. A few spots are still available if you'd like to join us: https://lnkd.in/evGSnAU2

Vonage recently released the Video Connector SDK and a companion Pipecat Transport. These new tools let a server-side AI agent join a Vonage Video session as a full participant, with real-time audio and video. Together they make it possible to build an AI Avatar that doesn't just chat, but actually listens, thinks, and speaks inside a live video call. It's ideal for customer support, live commerce, virtual tutors, telehealth, and more. We paired the Vonage Video Connector SDK and Pipecat Transport with: - Real-time speech-to-text using Deepgram - LLM-powered reasoning through Amazon Bedrock - Natural text-to-speech from ElevenLabs - A lip-synced talking avatar from Simli All of it orchestrated with Pipecat, an open-source framework by Daily for building voice and multimodal AI agents. The building blocks are modular. Swap STT providers, replace the LLM, or use a different avatar service. The pipeline structure stays the same. We wrote up the full technical build, with code: https://lnkd.in/eq-Swq9j. From here, you could extend this in several directions: - Multi-agent sessions. Multiple AI participants with different roles in the same call - Tool use. Give the agent function-calling capabilities to look up data, book appointments, or control external systems - Vision input. Process the participant’s video feed so the agent can see what’s happening (screen sharing, product demos, whiteboarding) - Multilingual support. Detect the participant’s language and respond in kind

"Real-time phone translation conversational agent" (Agente conversacional para traducción telefónica en tiempo real) and "Real-time noise reduction under strict latency constraints" (Reducción de ruido en tiempo real bajo restricciones estrictas de latencia) are the two demos that will be presented by our AI Tinkerers Madrid co-sponsor BTS on July 7 - one week from today! If you are in Madrid and interested in the intersection of AI and real-time communications, don't miss these presentation by Dayana Ribas and Aitor del Val Allueva. Only a few seats remain. Register at https://lnkd.in/evGSnAU2.