Unified Device Config & Multi-Transport Sync Protocol (opengnd.device.config.v1)
Authoritative cross-client specification (V1.4.0) for opengnd.device.config.v1 across AgentHUD v2.4.1+ firmware (Stop completion voice trigger fix & 4 built-in neural voices/lang), OpenGND Cloud Relay, OpenGND Hub v3.0.4+, and Web (/configure & /console): covers 4 built-in neural voice presets (builtin:warm_female default), 3-tier Daily/Advanced/Developer UI grouping, 4-protocol transports, 5 rotary knob modes & 16-session newest-first 6-row windowed THREAD_LIST, ES8311 REG32 hardware dB volume calibration + 300ms live debounced sync, 8-language persona localization, 3-mode sound sources & per-scene rules, Cloud AI Sound/Voice Clone Studio, and OGND 4-bit IMA-ADPCM.
1. Canonical JSON Schema (opengnd.device.config.v1 · <= 16,384 Bytes)
Every OpenGND configuration client—WebSerial Studio, LAN HTTP, OpenGND Cloud Console, and OpenGND Hub v3.0.4+—shares one isomorphic JSON model: opengnd.device.config.v1. Payloads are strictly bounded to 16,384 UTF-8 bytes and merge partially on PATCH/config_set while keeping Wi-Fi passwords, BYOK API keys, and auth tokens strictly isolated outside device_config.
Optional client/cloud fields audio.tts_voice_id (<= 128 B), audio.tts_provider_id (<= 64 B), and audio.sfx_prompt (<= 256 B) track selected cloud/cloned voices and AI sound prompts in canonical JSON while remaining omitted from compact MCU wire frames (toFirmwareWirePayload). Conversely, audio.system_voice_name (default 'builtin:warm_female', <= 128 B canonical / <= 64 B wire) IS emitted by toFirmwareWirePayload() when non-empty so WebSerial and LAN HTTP immediately sync the user's chosen built-in neural voice preset ('builtin:warm_female' | 'builtin:clear_male' | 'builtin:bright_female' | 'builtin:deep_male') or browser voice to device NVS.
{
"schema": "opengnd.device.config.v1",
"device": {
"sku": "SKU-01",
"fw_id": "agent-hud",
"fw_version": "2.4.1",
"device_id": "94A990EE607C"
},
"system": {
"ui_lang": "en",
"brightness_pct": 85,
"time_24h": true,
"tz_offset_min": 480
},
"controls": {
"knob_mode": "auto",
"knob_invert": false,
"auto_expand_on_turn": true
},
"persona": {
"avatar_id": "nova",
"custom_name": "",
"custom_names": {}
},
"audio": {
"muted": false,
"volume_pct": 75,
"sound_mode": "human",
"voice_engine": "system",
"system_voice_name": "builtin:warm_female",
"on_done_mode": "phrase",
"on_confirm_mode": "phrase",
"on_notify_mode": "chime",
"confirm_alert_sound": "double_beep",
"chime_before_voice": true,
"signature_sound": "crystal_chord",
"custom_signature_notes": [523, 659, 784, 988],
"custom_signature_timbre": "bell",
"custom_signature_tempo_ms": 95,
"play_signature_on_done": true,
"play_poetic_voice_on_done": true,
"voice_lang": "follow_ui",
"poetic_custom_text": "",
"confirm_custom_text": "",
"notify_custom_text": ""
}
}Wi-Fi PSKs, BYOK LLM keys, and OpenGND Cloud tokens never enter opengnd.device.config.v1 or cloud D1 devices.config_json. Device identity fields (sku, fw_id, fw_version, device_id) are hardware-derived and ignored on client PATCH writes.
2. Four-Protocol Transport Mapping Matrix
The browser Unified Configurator (/configure) and Studio Tab 04 adapt transparently across four physical and network transports using one canonical payload:
| Operation | webserial (115200 B64_MCP) | lan-http (Port 80 REST) | cloud-relay (OpenGND Cloud) | opengnd-desktop (v3.0.4+) |
|---|---|---|---|---|
| Read Full Config | @@HIL:ACTION:B64_MCP:{"action":"config_get"}@@ -> @@CONFIG_JSON:<b64>@@ | GET /api/v1/device/config | GET /api/v1/devices/:id/config (+ /api/xiaozhi/ota) | invoke('device_config_get') |
| Merge / Save Config | @@HIL:ACTION:B64_MCP:{"action":"config_set","config":...}@@ -> @@CONFIG_JSON:<b64>@@ | POST /api/v1/device/config | PATCH /api/v1/devices/:id/config -> DO config_sync | invoke('device_config_set') |
| Audition Audio Sequence | {"action":"audio_test","mode":"full"} | POST /api/v1/device/audio-test | Browser WebAudio preview + DO config_sync | invoke('device_audio_test') |
| Upload Signature ADPCM | sig_sound_begin -> sig_sound_chunk (<= 512B) -> sig_sound_commit | POST /api/v1/device/signature-sound (octet-stream) | Syncs custom parameters; raw flash blob via USB/LAN/Desktop | invoke('device_upload_signature_sound') |
| Upload Poetic Voice ADPCM | custom_voice_begin -> custom_voice_chunk (<= 512B) -> custom_voice_commit | POST /api/v1/device/custom-voice (octet-stream) | Syncs voice_lang + poetic_custom_text | invoke('synthesize_and_push_poetic_voice') |
When visiting https://opengnd.com over HTTPS, Chromium may block plain HTTP fetches to local .local or RFC1918 IP addresses. Use the USB WebSerial transport directly in Chrome/Edge, Cloud Relay via /console, or open http://opengnd-clock.local/ directly.
3. Rotary Encoder 5 Modes & 16-Session Newest-First THREAD_LIST Overlay
The top EC11 rotary encoder (IO47/IO21) supports five selectable interaction modes (controls.knob_mode) plus knob_invert and auto_expand_on_turn. High-frequency volume rotation updates ES8311 REG32 hardware dB gain in <20ms while trailing-debouncing NVS writes by 1,000ms to protect SPI Flash; Web/Desktop slider/mute changes sync via ~300ms debounced lightweight { audio: { volume, volume_pct, muted } } patches and flush immediately before audio_test or scene audition.
Critical safety rule: knob_mode operates strictly at the user interaction layer—ognd_pwr_sup (0V battery cut on SW4 >= 4s, soft reboot on SW3+IO14 >= 2.5s, 120s-RWDT USB Download on SW3 >= 7s) and CRITICAL permission lockouts can never be bypassed or overridden.
| knob_mode | Rotary CW / CCW Behavior | Short Press IO14 Behavior | Overlay / Notes |
|---|---|---|---|
| auto (Default) | Moves SETTINGS_MENU cursor when open; scrolls content when expanded; opens THREAD_LIST across multiple idle sessions. | Toggles expansion or executes active focus action. | Cooperates with auto_expand_on_turn. |
| scroll_content | Auto-expands card if enabled and scrolls UTF-8 body lines continuously. | Collapses or expands the reading viewport. | Clamped to 0 .. total_lines - visible_lines. |
| switch_sessions | First turn opens newest-first THREAD_LIST overlay (up to 16 active sessions in a 6-row sliding window; WORK green, WAIT amber, CRIT red, DONE orange); turns scroll cursor #1..#N. | Activates highlighted task and closes THREAD_LIST. | SW3 or 2.5s idle timeout dismisses without changing active task. |
| volume | Adjusts audio.volume_pct in ±5% steps (0..100%) mapped to calibrated ES8311 REG32 dB curve with Signal Orange OSD. | Toggles audio.muted and IO45 PA enable. | Unmutes automatically when turning up from 0%. |
| pomodoro | Cycles 15m -> 25m -> 45m -> 60m focus timer presets. | Starts, pauses, or resets Pomodoro timer. | Synced across AgentHUD & Desktop. |
4. 8-Language Localization & 6 Built-in Personas
Core 5 languages (en, zh-CN, zh-TW, ja, ko) ship with full Flash XIP 4bpp antialiased fonts on SKU-01; Extended 3 languages (es, fr, de) are preserved verbatim in NVS and voice_lang while falling back cleanly to Latin-1/English glyphs on-device. Persona names resolve deterministically in four steps: persona.custom_names[lang] -> persona.custom_name -> localized built-in table -> English fallback.
| avatar_id (Aliases) | en | zh-CN | zh-TW | ja | ko | es / fr / de |
|---|---|---|---|---|---|---|
| nova (ognd-spark) | Nova | 星诺 | 星諾 | ノヴァ | 노바 | Nova |
| duck (pixel_duck) | Dr. Quack | 呱博士 | 呱博士 | ダック博士 | 오리박사 | Dr. Quack |
| neko (neko_chan) | Mittens | 奈奈喵 | 奈奈喵 | ミケちゃん | 나비 | Mittens |
| bmo (robo_bit) | Bit-01 | 比特仔 | 比特仔 | ビット君 | 비트군 | Bit-01 |
| fox (ember_spirit) | Sparky | 暖焰狐 | 暖焰狐 | カルシファー | 불꽃요정 | Sparky |
| panda (mochi_bun) | Mochi | 糯米团 | 糯米團 | もち丸 | 모찌 | Mochi |
5. Signature Sound Synthesizer & OGND 4-bit IMA-ADPCM Format
Signature chimes combine 3-harmonic additive ADSR synthesis across five built-in presets (crystal_chord, warm_marimba, zen_bowl, cyber_pulse, retro_arcade) plus custom sequences (1..16 notes, 130..4000 Hz or 0 rest, 6 timbres, 40..500 ms step tempo) and optional seeded heuristic generation.
When uploading custom chimes (ah_sig_adpcm <= 16,384 B, <= 2.0s) or poetic completion voices (ah_poetic_adpcm <= 65,536 B, <= 8.0s), OpenGND encodes 16kHz Mono audio into a headerless OGND 4-bit IMA-ADPCM binary stream (8,000 B/s) followed by a deterministic 120ms silence gap before the poetic voice line plays (unless muted or PTT ES7210 recording is active).
| Byte Offset | Length | Field | Type & Spec |
|---|---|---|---|
| 0x00 .. 0x01 | 2 Bytes | init_pred | int16_le initial signed PCM16 predictor (-32768 .. 32767) |
| 0x02 | 1 Byte | step_idx | uint8 initial IMA step table index (clamped 0 .. 88) |
| 0x03 | 1 Byte | reserved | uint8 reserved flag byte, must be 0x00 |
| 0x04 .. EOF | N Bytes | nibble_payload | uint8[] low nibble first: sample[2k] = byte & 0x0F, sample[2k+1] = (byte >> 4) & 0x0F |
6. Cloud AI Voice Studio, Multi-Vendor Voices & Cloning Bridge (OPENGND-49)
Unified Configurator (/configure) integrates credit-metered cloud audio generation directly with hardware NVS storage without adding any decoder complexity to the ESP32-S3 firmware. Cloud endpoints emit deterministic device bundles containing both wav_b64 (for instant browser preview) and 16kHz Mono 4-bit IMA-ADPCM adpcm_b64 (for direct push via transport.uploadSignatureSound <= 16,384 B or transport.uploadCustomVoice <= 32,768 B).
For zero-shot voice cloning, the browser decodes MediaRecorder streams (Opus/WebM) or uploaded .wav/.mp3/.m4a/.webm audio files locally via encodePcm16OrBufferToWavBytes() into standard 16kHz Mono 16-bit PCM RIFF WAV bytes before uploading to POST /api/v1/cloud/voices/clone—eliminating server-side transcoding overhead on Cloudflare Workers.
| Endpoint | Role | Payload & Parameters | Hardware / Credit Integration |
|---|---|---|---|
| GET /api/v1/cloud/voices | User / Guest | credentials: include | Returns vendor-grouped preset voices, user cloned_voices, and studio_capabilities (toggles, pricing, limits, ready providers). |
| POST /api/v1/cloud/sfx/generate | Signed-in User | { prompt, duration_sec (0.6..2.0), target: 'signature' } | Charges sfx_generation_credits * multiplier; returns wav_b64 + adpcm_b64 (<= 16,384 B) ready for uploadSignatureSound(). |
| POST /api/v1/cloud/tts | Signed-in User | { text, voice, language, format: 'device_bundle', target: 'custom_voice', provider_id? } | Supports preset & cloned voices (+cloned_tts_extra_credits); returns wav_b64 + adpcm_b64 (<= 32,768 B) for uploadCustomVoice(). |
| POST /api/v1/cloud/voices/clone & DELETE /api/v1/cloud/voices/cloned/:id | Signed-in User | 16kHz Mono 16-bit PCM WAV + name, language, reference_text?, provider_id? | Enforces voice_clone_max_per_user and voice_clone_credits * multiplier; creates or removes user-scoped cloned voices. |
| GET / PATCH /api/v1/admin/voice-studio-config | Admin | text_to_sfx_enabled, voice_clone_enabled, sfx_generation_credits, voice_clone_credits, cloned_tts_extra_credits, voice_clone_max_per_user, preferred_sfx_provider_id, preferred_clone_provider_id | Configures global feature gates, credit rates, per-user clone caps, and preferred routing providers. |
| POST /api/v1/admin/providers/:id/sync-voices | Admin | TTS provider :id (elevenlabs, fish_audio, minimax_t2a, azure_speech) | Refreshes upstream vendor voice catalogs into D1; MiniMax channels pass config.group_id. |
HTTP 401 prompts users to sign in at /console; HTTP 402 directs to /console?tab=credits for credit top-up; HTTP 503 indicates the feature is disabled by admin or the target upstream provider key is not yet configured in /console?tab=providers.
7. 3-Mode Audio Source & Per-Scene Broadcast Rules (V1.2.0 · OPENGND-51)
V1.2.0 streamlines device audio setup into two orthogonal steps: (1) choose a sound source mode (audio.sound_mode: mechanical, human [system vs cloud], or cloned), and (2) choose per-scene playback behavior across Task Completed (on_done_mode), Approval Needed (on_confirm_mode + confirm_alert_sound), and Status & Alerts (on_notify_mode).
Each scene selects one of four behaviors—content (speak AI reply/approval/alert summary up to 220 chars), phrase (speak a short 8-language template or inline custom phrase <= 256 B), chime (hardware tone only), or off (visual card only)—alongside chime_before_voice to toggle the ~0.4s lead-in alert.
| Field | Type & Default | Allowed Values / Bounds | Behavior & Firmware Projection |
|---|---|---|---|
| audio.sound_mode | string (default: 'human') | mechanical | human | cloned | Top-level 3-way selector. 'mechanical' shows 5 hardware chords + collapsible composer/AI SFX; 'human' toggles System vs Cloud; 'cloned' manages vc_* voices. |
| audio.voice_engine & audio.system_voice_name | string (default: 'system'), string (default: 'builtin:warm_female') | system | cloud; 'builtin:warm_female' | 'builtin:clear_male' | 'builtin:bright_female' | 'builtin:deep_male' | OS voice <= 128 B | 'system' offers 4 zero-credit built-in neural voices per language (/audio/builtin-voices/<lang>/<shortId>.mp3 + firmware NVS sync <= 64 B) plus OS SpeechSynthesis fallback; 'cloud' routes to vendor-grouped /api/v1/cloud/tts. |
| audio.on_done_mode | string (default: 'phrase') | content | phrase | chime | off | Task completion rule. Automatically derives play_poetic_voice_on_done and play_signature_on_done in toFirmwareWirePayload(). |
| audio.on_confirm_mode & audio.confirm_alert_sound | string (default: 'phrase'), string (default: 'double_beep') | content | phrase | chime | off; double_beep | radar_ping | amber_bell | cyber_ack | Decision/approval rule paired with 4 synthesized alert tones and CONFIRM_PHRASES_8 / confirm_custom_text. |
| audio.on_notify_mode | string (default: 'chime') | content | phrase | chime | off | Background status & alert rule paired with NOTIFY_PHRASES_8 / notify_custom_text. |
| audio.chime_before_voice | boolean (default: true) | true | false | Controls whether content/phrase speech is preceded by the ~0.4s hardware chime/alert + 120ms gap. |
| audio.confirm_custom_text & audio.notify_custom_text | string (default: '') | UTF-8 0 .. 256 B | Single-line custom phrase overrides active when on_confirm_mode or on_notify_mode is set to 'phrase'. |
AgentHUD v2.4.1 fixes the Stop hook task-completion voice celebration trigger so finishing a session via Stop reliably plays signature chime + 120ms + built-in neural / custom ADPCM voice. deriveLegacyAudioFlags() maps on_done_mode ('content' | 'phrase' -> play_poetic_voice_on_done=true; 'chime' | 'off' -> false) and chime_before_voice into legacy booleans; when older firmware echoes legacy booleans on config_set, normalizeUnifiedDeviceConfig preserves explicit scene modes and system_voice_name without clobbering.