Home / Work / Jarvis

Jarvis — Offline Voice Coding Assistant

Say "Hey Jarvis", speak a request, and it is transcribed and handed to Aider wired to a local LLM as a one-shot coding command. Wake word, speech-to-text and the model all run on the machine, so nothing leaves it.

Status
Working tool
Repository
Private (code walkthrough available on request)
Tech Stack
Python, openWakeWord, faster-whisper, llama.cpp (llama-server), Aider, sounddevice

Overview

Say "Hey Jarvis", speak a request, and Jarvis transcribes it and hands it to Aider, which edits the chosen project using a local model. Everything runs on the machine; no audio or code leaves it.

The Problem

  • Cloud assistants send your voice and your code to third parties.
  • Typing every instruction to a coding agent breaks focus.

How It Works

  1. A single 16 kHz microphone stream is read in 80 ms frames.
  2. openWakeWord scores each frame; above the threshold, Jarvis starts listening.
  3. A few frames from just before the trigger are kept so the first word isn't clipped.
  4. Recording stops after 1.2 s of silence or 6 s at most.
  5. faster-whisper transcribes the audio locally.
  6. The text is sent to Aider as a one-shot command against the local llama-server model.

Highlights

  • Fully offline and private.
  • Same audio stream reused between wake word and command, so there's no gap.
  • Start scripts for Windows and Linux/macOS.