割り込み可能な対話型音声AI — デプロイガイド

GitHub でソースを見る
プリセット

Cloud or OpenAI-Compatible LLM

Speech runs on the device; replies are generated by Qwen API or the OpenAI-compatible endpoint you enter. The default is Alibaba Cloud Model Studio's Beijing endpoint with qwen3.5-flash; replace the base URL, key, and model ID for another provider.

  • Microphone: a reSpeaker XVF3800 USB microphone array is required (a plain USB mic causes echo and false interruptions).
  • Network: required for every conversation turn.
  • API key: enter an API Key and a Model ID at deploy time; the model must support streaming Chat Completions.

Optional wake word: select Wake word required in the deployment form and enter a short Chinese or English phrase; a short tone plays when it is detected. Always listening is the default.

1

Deploy the Cloud-Backed Voice Terminal

After deployment, you can interrupt a spoken answer at any time.

デプロイ方法

Run speech on RK3576 and connect to a cloud or LAN model. Requires at least 12 GB free disk.

配線
  1. Connect the reSpeaker XVF3800 and speaker
  2. Fill in the SSH details and the model endpoint settings
入力が必要な項目（SenseCraft Solution アプリで入力）
  • Conversation language*既定値 zh
    2 件の選択肢
    • · Chinese
    • · English

    Spoken language for recognition and for the synthesized reply. The list is the RK runtime's 30-language set (openvoicestream matrix language_catalog_source), the narrower of the two available lists - Qwen3-ASR upstream advertises 52 and Whisper 99. RK3576 serves Chinese and English only: its TTS bundle is Matcha icefall zh-en, and the multilingual Kokoro bundle on this board is TTS-only, so no profile pairs it with an ASR backend yet. The deployment resolves (language, device) to one speech profile before any service starts and stops with an error if this board cannot serve the language.

  • Device IP*
  • SSH Username*既定値 cat
  • SSH Password*
  • OpenAI-compatible Base URL*既定値 https://dashscope.aliyuncs.com/compatible-mode/v1
  • API Key*
  • Model ID*既定値 qwen3.5-flash
  • Assistant personality*既定値 You are a natural, witty voice assistant. Reply in the user's language. Keep responses conversational and brief—usually one or two sentences. Add light humor and personality when appropriate, but never at the expense of accuracy. Avoid rambling, repetition, lectures, Markdown, headings, and lists. If details are missing, ask one short question. For spoken Chinese, write numbers, dates, times, money, and units as natural spoken Chinese.

    Editable system prompt applied to every conversation.

  • RK Performance Mode*既定値 performance
    2 件の選択肢
    • · Maximum performance
    • · System default / balanced

    Lock CPU, NPU, DDR, and GPU (when present) to their highest reported frequencies, or keep the system governor unchanged.

既定値のある残り 4 項目
  • Conversation mode既定値 always_on
    2 件の選択肢
    • · Always listening
    • · Wake word required
  • Wake word既定値 你好小智

    Any short Chinese or English phrase compiled at startup by the open-vocabulary KWS backend.

  • Wake-word sensitivity既定値 0.25
    3 件の選択肢
    • · Strict (0.35)
    • · Balanced (0.25)
    • · Sensitive (0.15)

    Lower threshold is more sensitive and may increase false triggers.

  • Reply after how much silence既定値 0
    4 件の選択肢
    • · Balanced (image default
    • · Snappy (0.9 s)
    • · Tolerant (2 s)
    • · Very tolerant (3 s)

    Server-side speech endpoint: how long the assistant waits after you stop speaking before it replies. The image default (1.5 s) tolerates a natural mid-sentence pause; a smaller value feels snappier but can cut one sentence into two; a larger value tolerates long thinking pauses at the cost of a slower reply. Must stay above the device-side VAD silence (600 ms). Only affects the paths where the device itself decides the end of speech — a separate microphone/agent application keeps its own setting.

トラブルシューティング
SymptomAction
API returns 401Confirm the API key belongs to the endpoint's region
Speech is recognized but no reply playsCheck the agent log for LLM request errors and confirm the model supports streaming
No audio after replugging the microphoneConfirm the new Agent image is running
デプロイ後

Ask a question. Within one second of playback starting, speak again; the current answer should stop immediately.

2

Verify Conversation and Barge-in

Watch the listening, thinking, speaking, and barged-in states on the dashboard.

トラブルシューティング
SymptomAction
State changes but there is no soundCheck that the default playback device is the speaker
Room noise triggers interruptionsConfirm the microphone is the reSpeaker XVF3800, then raise the client VAD threshold slightly; do not mute capture
デプロイ後

Three normal turns plus an immediate stop when you say “wait” during playback pass this check.

3

Verify the Selected Language

Confirm that recognition and spoken replies use the language chosen at deploy time.

トラブルシューティング
SymptomAction
Replies come back in the wrong languageCheck the assistant personality prompt
Deployment never startedThe selected language is not supported on this board; pick a supported one
Transcript language is right but audio is wrongReport the language and the board; do not edit the speech profile yourself
デプロイ後

Acceptance checklist

  1. curl -fsS http://<device-ip>:8621/health returns success.
  2. Ask one question; a spoken reply starts within a few seconds.
  3. Speak again within one second of playback starting; the current answer stops immediately.
  4. Two or three turns are transcribed and answered in the selected language.
  5. On the device, docker compose -p conversational_voice_ai -f ~/conversational_voice_ai/assets/docker/docker-compose.<target>.yml logs --since 10m | grep -i error returns nothing (<target> is rk3576, rk3588, jetson, or rpi5).