From 9c77f2a7bef485fa137f123cbf55b42cacb2b285 Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Tue, 15 Jul 2025 21:12:18 +0300 Subject: feat: add OpenAI gpt-4o-mini-tts support with voice instructions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add support for OpenAI's new gpt-4o-mini-tts model with customizable voice instructions - Add OpenAIInstruction field to audio configuration for natural language voice control - Update CLI with --openai-instruction flag for runtime voice customization - Enhanced cache key generation to include voice instructions - Update default model to gpt-4o-mini-tts with Bulgarian-optimized instructions - Add support for new voices: ash, ballad, coral, sage, verse - Improve error handling for models requiring special API access - Update documentation with examples and model information - Create .totalrecall.yaml.example with comprehensive configuration options Note: The gpt-4o-mini-tts model requires special API access and may not be available to all accounts yet. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- GPT4O_AUDIO_NOTE.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 GPT4O_AUDIO_NOTE.md (limited to 'GPT4O_AUDIO_NOTE.md') diff --git a/GPT4O_AUDIO_NOTE.md b/GPT4O_AUDIO_NOTE.md new file mode 100644 index 0000000..a227c60 --- /dev/null +++ b/GPT4O_AUDIO_NOTE.md @@ -0,0 +1,35 @@ +# GPT-4o Audio Support Note + +## Current Status + +The standard OpenAI Text-to-Speech API (`/v1/audio/speech`) currently supports only: +- `tts-1` - Standard quality +- `tts-1-hd` - High definition quality + +## GPT-4o Audio Capabilities + +According to OpenAI documentation, GPT-4o models have audio capabilities, but these work differently: + +1. **Realtime API**: GPT-4o audio generation might be part of the new Realtime API, which uses WebSockets for bidirectional audio streaming. + +2. **Chat Completions with Audio**: GPT-4o might support audio output through the chat completions API with special modality parameters, but this requires different request/response handling than the standard TTS API. + +3. **Model Names**: Models like `gpt-4o-audio-preview` or `gpt-4o-mini` with audio capabilities might not be compatible with the standard TTS endpoint. + +## Experimental Usage + +You can try experimental model names with the `--openai-model` flag: +```bash +./totalrecall "word" --openai-model gpt-4o-audio-preview +``` + +However, this will likely result in a 404 error as these models require different API endpoints. + +## Future Implementation + +To properly support GPT-4o audio generation, we would need to: +1. Implement support for the Realtime API (WebSocket-based) +2. Or implement the chat completions API with audio modalities +3. Handle different request/response formats for audio data + +For now, stick with `tts-1` or `tts-1-hd` for reliable audio generation. \ No newline at end of file -- cgit v1.2.3