--- name: realtime-voice description: Real-time bidirectional voice conversations (<1s latency) via OpenAI Realtime GA API (gpt-realtime / gpt-realtime-1.5 / gpt-realtime-2) with Gemini Live fallback. Use for live two-way voice (not just TTS). Full duplex audio in/out, Arabic supported. Verified live 2026-06-01. ---
Realtime Voice Skill (bidirectional)
Providers (auto-fallback order):
1. OpenAI Realtime (PRIMARY) — GA API. Models:gpt-realtime (best), gpt-realtime-mini (cheap), gpt-realtime-1.5, gpt-realtime-2 (newest), gpt-realtime-translate, gpt-realtime-whisper. Full duplex.
2. Gemini Live — gemini-2.0-flash-live-001 (cheapest fallback).⚠️ xAI Realtime — DOES NOT EXIST (verified 2026-06-01)
xAI models list has NO realtime/audio model (only grok-4.x text + grok-imagine image/video). "Merge with xAI realtime" is NOT possible — xAI offers no realtime voice endpoint. PRIMARY realtime engine = OpenAI GA (SOTA + verified working). Gemini Live = fallback.🔑 GA fix (2026-06-01)
The legacy headerOpenAI-Beta: realtime=v1 is now REJECTED (beta_api_shape_disabled).
GA shape = bearer auth ONLY, no beta header. openai_realtime.py patched accordingly. Verified: Arabic reply received live.Usage
bash
python3 skills/realtime-voice/openai_realtime.py \
--prompt "System instructions" --input-audio in.wav --output-audio out.wav \
--model gpt-realtime # or gpt-realtime-mini / gpt-realtime-1.5 / gpt-realtime-2
Audio specs
- Input: WAV 16kHz mono PCM16
- Output: WAV 24kHz mono PCM16
- For live mic-to-speaker on Samsung S24: needs mobile client (Telegram voice-note proxy works server-side).
Notes
- Both providers use WebSocket streaming under the hood.
- Telegram voice-note bridge: accept user voice note -> realtime -> reply as voice note.