podcast-generation

已同步 v1.1.0 授權: MIT 來源: microsoft/skills/.github/skills/podcast-generation 提交: cde32f9 收錄的最新變更: 2026-01-28

Generate AI-powered podcast-style audio narratives using Azure OpenAI's GPT Realtime Mini model via WebSocket. Use when building text-to-speech features, audio narrative generation, podcast creation from content, or integrating with Azure OpenAI Realtime API for real audio output. Covers full-stack implementation from React frontend to Python FastAPI backend with WebSocket streaming.

摘要

用途: 說明如何使用 Azure OpenAI Realtime 將文字轉為語音敘述，並準備供播放的音訊。
使用時機: 當服務需要透過 WebSocket 將文字串流轉為旁白，並提供可在網頁前端播放的音訊時使用。
輸出結果: 會收集 24 kHz、16 位元、單聲道的 PCM 音訊與逐字稿文字，再將音訊轉為 WAV，供 Base64 編碼後在瀏覽器播放。

上游變更（2 筆，最新收錄 2026-01-28）

a2003b6 2026-01-28 docs(skills): standardize all skills to skill-creator guidelines 移除了 podcast-generation 的 SKILL.md 檔案前後的 ```skill 圍欄。

14224cc 2026-01-16 feat: Add skill creator scripts and templates for initializing and packaging skills 新增 podcast-generation 技能，說明透過 Azure OpenAI GPT Realtime Mini WebSocket 產生音訊、PCM 轉 WAV、瀏覽器播放、語音選項與 API 事件。

npx skills add "lettucebo/Skills#v2.2.0@podcast-generation" --full-depth

⚠ v2.2.0 發布後即可使用

Podcast Generation with GPT Realtime Mini

Generate real audio narratives from text content using Azure OpenAI's Realtime API.

Quick Start

Configure environment variables for Realtime API
Connect via WebSocket to Azure OpenAI Realtime endpoint
Send text prompt, collect PCM audio chunks + transcript
Convert PCM to WAV format
Return base64-encoded audio to frontend for playback

Environment Configuration

AZURE_OPENAI_AUDIO_API_KEY=your_realtime_api_key
AZURE_OPENAI_AUDIO_ENDPOINT=https://your-resource.cognitiveservices.azure.com
AZURE_OPENAI_AUDIO_DEPLOYMENT=gpt-realtime-mini

Note: Endpoint should NOT include /openai/v1/ - just the base URL.

Core Workflow

Backend Audio Generation

from openai import AsyncOpenAI
import base64

# Convert HTTPS endpoint to WebSocket URL
ws_url = endpoint.replace("https://", "wss://") + "/openai/v1"

client = AsyncOpenAI(
    websocket_base_url=ws_url,
    api_key=api_key
)

audio_chunks = []
transcript_parts = []

async with client.realtime.connect(model="gpt-realtime-mini") as conn:
    # Configure for audio-only output
    await conn.session.update(session={
        "output_modalities": ["audio"],
        "instructions": "You are a narrator. Speak naturally."
    })
    
    # Send text to narrate
    await conn.conversation.item.create(item={
        "type": "message",
        "role": "user",
        "content": [{"type": "input_text", "text": prompt}]
    })
    
    await conn.response.create()
    
    # Collect streaming events
    async for event in conn:
        if event.type == "response.output_audio.delta":
            audio_chunks.append(base64.b64decode(event.delta))
        elif event.type == "response.output_audio_transcript.delta":
            transcript_parts.append(event.delta)
        elif event.type == "response.done":
            break

# Convert PCM to WAV (see scripts/pcm_to_wav.py)
pcm_audio = b''.join(audio_chunks)
wav_audio = pcm_to_wav(pcm_audio, sample_rate=24000)

Frontend Audio Playback

// Convert base64 WAV to playable blob
const base64ToBlob = (base64, mimeType) => {
  const bytes = atob(base64);
  const arr = new Uint8Array(bytes.length);
  for (let i = 0; i < bytes.length; i++) arr[i] = bytes.charCodeAt(i);
  return new Blob([arr], { type: mimeType });
};

const audioBlob = base64ToBlob(response.audio_data, 'audio/wav');
const audioUrl = URL.createObjectURL(audioBlob);
new Audio(audioUrl).play();

Voice Options

Voice	Character
alloy	Neutral
echo	Warm
fable	Expressive
onyx	Deep
nova	Friendly
shimmer	Clear

Realtime API Events

response.output_audio.delta - Base64 audio chunk
response.output_audio_transcript.delta - Transcript text
response.done - Generation complete
error - Handle with event.error.message

Audio Format

Input: Text prompt
Output: PCM audio (24kHz, 16-bit, mono)
Storage: Base64-encoded WAV

References

Full architecture: See references/architecture.md for complete stack design
Code examples: See references/code-examples.md for production patterns
PCM conversion: Use scripts/pcm_to_wav.py for audio format conversion