Building a Wake-Word Voice Assistant on a $30 Bluetooth Speaker

I wanted the Alexa experience with an agent I could change: say “OpenClaw, do X,” have the machine hear the request, do the work, and speak the answer back. The command could be turning off a television or looking something up. I wanted the long tail of requests to reach a reasoning agent with tools, without having to write a new intent handler for every phrase.

The hardware was deliberately ordinary: an E09 mini Bluetooth speaker, weighing 81 grams, with 5W output and a built-in noise-reduction microphone. I paired it with server14, the always-on Ubuntu mini-PC that already runs my household automations. The speaker was the only new hardware, at about $30.

The clip demonstrates the spoken interaction. It also makes the latency audible: there is a wait before the acknowledgement and another before the answer. The video does not show the agent’s tool trace, so the spoken numbers alone are not evidence that it implemented or ran a generator.

The architecture: four parts and one rule

The audio loop is straightforward once the devices are cooperating:

Microphone → voice-activity detection → Whisper
          → wake-word match → command queue
          → handler (local action or agent turn)
          → text-to-speech → speaker

1. Capture. Read raw audio from the speaker’s microphone and divide it into utterances.
2. Transcribe. Send each detected utterance to OpenAI Whisper and receive text.
3. Detect. Find the wake-word span in the transcript and take the text after it as the command.
4. Act and answer. Handle a few quick commands locally; send everything else to an OpenClaw agent turn, then speak its reply.

The hard rule is that exactly one component owns speech output. That sounds obvious until two independent processes both decide to acknowledge the same request.

Two daemons, one queue

I split the work into two small Python programs connected by an append-only JSON-lines file. The listener captures audio, detects speech, transcribes it, recognizes the wake word, and enqueues a command. It never speaks. The handler reads the queue, does the work, and speaks the result.

That separation keeps an agent round-trip from blocking audio capture while the model is thinking. The listener and handler also have different restart behavior: the handler records a byte offset into the queue so it can resume near where it stopped instead of reading yesterday’s commands from the beginning.

An offset is useful recovery bookkeeping, but it does not make actions exactly-once. A crash after an action and before saving its offset can still repeat that action. For a small assistant, commands such as “set the TV to off” are easier to retry safely than “toggle the TV.” Actions that cannot tolerate a repeat need their own durable acknowledgement or deduplication.

The audio stack is the hard part

Getting words to an agent was easier than persuading Linux and Bluetooth to agree about the microphone. On this classic Bluetooth setup, A2DP gives good playback quality, while HFP makes the microphone available with lower-quality, mono call audio. I put the device into HFP and accepted the tradeoff. It is fine for a short spoken answer; it is not the profile I would choose for music.

PipeWire handles the audio, and WirePlumber manages the Bluetooth profiles and routing. Its Bluetooth documentation is a useful reference when the playback device exists but the capture device does not. The profiles and codecs available still depend on the speaker and adapter.

For voice-activity detection, I kept the implementation simple: compute RMS energy over 30-millisecond frames. A run of loud-enough frames starts an utterance; a run of quiet frames ends it. That gives the transcriber discrete audio clips without adding a neural VAD or a webrtcvad dependency.

The simplicity has limits. A noisy room can look like speech, and a quiet voice can fall below the threshold. This is an energy detector, not a person detector. In my logs, Whisper occasionally turned near-silence into confident little transcripts such as “Bye” or “Thank you.” Those never reached the handler because they did not contain the wake word.

This build uses OpenAI’s speech-to-text API for completed utterances. It is worth being precise about the order: the wake-word check happens after transcription, so audio selected by the energy detector is sent to the API even when nobody said “OpenClaw.” A local wake-word detector before upload would be a different design.

The wake word is messier than “openclaw”

Real transcripts gave me openclaw, open claw, open call, open, claw, and even open open claw. My first parser normalized the two-word spelling and then used the resulting index to slice the original string. The index had shifted. It left a stray letter at the start of the command.

The fix was to match the whole wake-word span on the original transcript and slice from the end of that match. Here is the small parsing helper that expresses the approach:

import re

WAKE = re.compile(
    r"(?:\bopen[\s,\-]*)+(?:claw|call|clause|clog)\b",
    re.IGNORECASE,
)

def command_after_wake(transcript: str) -> str | None:
    match = WAKE.search(transcript)
    if match is None:
        return None
    command = transcript[match.end():].lstrip(" \t\r\n,.;:!?—–-")
    return command or None

command_after_wake("Open open claw. What time is it?")
# 'What time is it?'

Matching and slicing the same string removes the indexing bug. The permissive alternatives help with this particular recognizer, though a phrase like “open call” can also occur in ordinary conversation. The pattern is a convenience trigger, not speaker authentication.

Let the agent handle the long tail

The first version knew about five hardcoded intents. That got a demo working, but it brought back the limitation I wanted to escape: every new request required another special case.

I kept a fast local path for the time, the date, and turning the TV on or off. Everything else goes to a real headless OpenClaw agent turn with its configured tools. A weather question can trigger a lookup; a request that needs a tool can use one. The voice layer passes on the command instead of trying to enumerate all possible tasks.

The agent is told that its answer will be played through a small speaker: one or two natural spoken sentences, no Markdown, no bullet lists, no emoji. The handler then sends that text to the text-to-speech API and plays the resulting audio.

Because an agent turn can take several seconds, the handler says “Let me check” before starting the slower work. It does not make the answer arrive sooner, but it makes the wait understandable. The demo at the top preserves those gaps rather than editing them out.

The device-control side builds on my earlier Home Assistant integration. That gives the agent a way to operate supported household devices. This project adds a microphone and a spoken reply to that existing capability.

The overlapping voices hid an echo problem

At one point, every command produced two voices talking over each other. The listener said its own “On it” acknowledgement while the handler said “Let me check” and then the answer. Two processes were independently trying to be helpful through one speaker.

The first fix was structural: the listener went silent. It only enqueues. The handler owns acknowledgements and answers, so the speech path has one mouth.

The second fix addressed what happened when that mouth was next to the microphone. While speaking, the handler creates a .speaking file. The listener watches for the marker and suspends capture into its utterance buffer during playback. Without that gate, the microphone hears the assistant’s own voice, transcribes it, and can trigger another command if the reply happens to contain “OpenClaw.”

This is a half-duplex design: I cannot interrupt it with another voice command while it is speaking. The marker suppresses feedback by preventing listening during playback; it is not acoustic echo cancellation. A file on its own is also not a process mutex. One handler must still own playback.

For a service left running, I would also make cleanup explicit: remove the speaking marker in a finally block, recover stale markers after crashes, and discard buffered playback audio before accepting the next utterance. Those details keep a useful mute mechanism from becoming a permanently deaf assistant.

What I would keep for the next build

• Separate capture from action. Listening and agent work have different latency and recovery needs.
• Choose the Bluetooth profile deliberately. On this device, microphone access means accepting call-quality output.
• Parse the original transcript. Match the complete wake-word span before slicing out the request.
• Give playback a single owner. Acknowledgements, answers, and microphone suppression belong to the same coordinated speech path.
• Keep a small fast path and an agent fallback. Common commands stay quick while the rest can use the agent’s tools.

The result is a speaker on the desk that lets me talk to the mini-PC already doing useful work around the house. I can inspect the Python, change the queue format, adjust the wake-word pattern, and decide what the agent can do.

The $30 figure covers the new speaker, with the computer reused. Whisper, text-to-speech, and the agent’s model still have usage costs, and the API calls need an internet connection. What I own is the integration: ordinary hardware, readable files, and a voice interface I can keep changing.

Built on server14 with Ubuntu, Python, PipeWire, OpenAI Whisper and text-to-speech, and OpenClaw. Continue with giving the assistant access to Home Assistant, or browse the rest of the build notes.