--- name: realtime-voice description: Real-time bidirectional voice conversations (<1s latency) via OpenAI Realtime GA API (gpt-realtime / gpt-realtime-1.5 / gpt-realtime-2) with Gemini Live fallback. Use for live two-way voice (not just TTS). Full duplex audio in/out, Arabic supported. Verified live 2026-06-01. ---

Realtime Voice Skill (bidirectional)

Providers (auto-fallback order):

1. OpenAI Realtime (PRIMARY) — GA API. Models: gpt-realtime (best), gpt-realtime-mini (cheap), gpt-realtime-1.5, gpt-realtime-2 (newest), gpt-realtime-translate, gpt-realtime-whisper. Full duplex. 2. Gemini Live — gemini-2.0-flash-live-001 (cheapest fallback).

⚠️ xAI Realtime — DOES NOT EXIST (verified 2026-06-01)

xAI models list has NO realtime/audio model (only grok-4.x text + grok-imagine image/video). "Merge with xAI realtime" is NOT possible — xAI offers no realtime voice endpoint. PRIMARY realtime engine = OpenAI GA (SOTA + verified working). Gemini Live = fallback.

🔑 GA fix (2026-06-01)

The legacy header OpenAI-Beta: realtime=v1 is now REJECTED (beta_api_shape_disabled). GA shape = bearer auth ONLY, no beta header. openai_realtime.py patched accordingly. Verified: Arabic reply received live.

Usage

bash
python3 skills/realtime-voice/openai_realtime.py \
  --prompt "System instructions" --input-audio in.wav --output-audio out.wav \
  --model gpt-realtime   # or gpt-realtime-mini / gpt-realtime-1.5 / gpt-realtime-2

Audio specs

• Input: WAV 16kHz mono PCM16
• Output: WAV 24kHz mono PCM16
• For live mic-to-speaker on Samsung S24: needs mobile client (Telegram voice-note proxy works server-side).

Notes

• Both providers use WebSocket streaming under the hood.
• Telegram voice-note bridge: accept user voice note -> realtime -> reply as voice note.

🔴 Gemini 3.8 Live (رُقّي 2026-09-16 بأمر د. وائل — جاهز للتفعيل)

• primary = gemini-3.8-live (native S2S، GA مُثبَت حيّاً 2026-09-17: HTTP 200 + WebSocket setup handshake نجح setupComplete على KW، صوت عربي حقيقي 180KB/75KB).
• التفكير المتوازي: gemini-3.8-live-extended-thinking (GA مُثبَت HTTP 200 2026-09-17 — يفكّر ويتكلّم بالتوازي، يحتاج thinking_config). التفعيل عند طلب «Live/محادثة مباشرة». قيد: WebSocket فقط (generateContent يرفضه بـ400).
• التفعيل: عند «محادثة مباشرة / live / صوت مباشر» → يُشغّل بكامل قدرات الترسانة (الأدوات + المنهجية الذهبية).
• التشغيل: python3 gemini_live.py --input-audio in.wav --output-audio out.wav [--model gemini-3.8-live].
• 🔴 قيد: WebSocket (bidiGenerateContent) فقط، لا REST. response_modalities=["AUDIO"]. العربية ضمن 97 لغة.
• fallback: OpenAI Realtime (openai_realtime.py) يبقى بديلاً.