whisper.cpp: Offline Voice Transcription Guide

whisper.cpp is Georgi Gerganov's C and C++ port of OpenAI's Whisper speech model, built to run offline speech to text on ordinary hardware with no cloud dependency. It is the standard answer in 2026 when someone asks how to transcribe voice recordings privately, for free, at volume.

This guide covers the working setup: install, model sizes and their honest trade-offs, batch-transcribing a folder of voice memos, and the limits that matter for productivity workflows, because transcription is only half of capture. For the cloud-and-app side of the same problem, see voice to text for notes.

What is whisper.cpp and why does it exist?

whisper.cpp is a from-scratch reimplementation of OpenAI's Whisper inference in portable C and C++, released under the MIT license and maintained in the ggml-org/whisper.cpp repository. It exists because the reference Python implementation is heavy: it needs a Python environment, PyTorch, and a GPU to feel fast. The port strips that down to a small binary that runs on CPUs, with Metal acceleration on Apple Silicon, and squeezes the model into ggml format files you download once.

Whisper itself is OpenAI's trained speech recognizer, published in a 2022 paper after training on 680,000 hours of weakly supervised audio. Its key property is robustness across accents, noise, and languages rather than benchmark chasing, which is why it became the substrate for nearly every transcription product of the last three years. whisper.cpp made that substrate free to run anywhere.

The audience in practice splits three ways: developers wiring transcription into pipelines, privacy-sensitive users who cannot send audio to a cloud, and tinkerers who want unlimited volume without per-minute pricing. All three get the same binary.

How do you install whisper.cpp?

Install whisper.cpp by cloning the GitHub repository and running make, or by installing a prebuilt build through Homebrew if your package manager carries it. From source, the shape of the job on macOS is:

git clone https://github.com/ggml-org/whisper.cpp
cd whisper.cpp
bash ./models/download-ggml-model.sh base.en
make

On Apple Silicon this appears as a few minutes of compilation and produces binaries under build/bin, with whisper-cli as the transcription entry point on current checkouts. Older documentation calls the binary main; if your build directory shows that name instead, it is the same tool and the flags are unchanged.

A Homebrew install trades that freshness for convenience, which is usually fine for voice memo use. Verify with brew search before assuming your tap has it. Building from source is what we recommend for anyone who will batch large volumes, because the upstream project moves quickly and Metal optimizations land in the repo before any packaged build.

Which whisper.cpp model size should you pick?

Pick base.en for short, clear English voice memos and small.en or large-v3 when the audio is noisy, accented, or full of jargon. The model size decides three things at once: word error rate, transcription speed, and RAM usage. Bigger models are more accurate and slower, and the accuracy gap concentrates in exactly the places voice memo users care about: names, numbers, and technical vocabulary.

The practical range, from the project's published ggml files:

Model	File size	Approx. RAM	Best for
tiny	~75 MB	~390 MB	Throwaway audio, nearly clean speech
base	~142 MB	~500 MB	Short English memos in quiet rooms
small	~466 MB	~1.0 GB	Noisy commutes, mixed accents
medium	~1.5 GB	~2.6 GB	Podcasts, interviews, harder audio
large-v3	~2.9 GB	~4.7 GB	Maximum accuracy, batch runs

The ".en" variants are English-only and slightly better on English than the multilingual files at the same size. For a voice-memo workflow on a Mac with 16 GB of RAM, starting at base.en and stepping up only when the output annoys you is the cheapest calibration path.

How do you batch-transcribe voice memos offline?

Batch transcription with whisper.cpp is two shell loops: one to convert every audio file into the format the model expects, one to run the transcriber over the converted files. The convert step is where most first-time failures live.

whisper.cpp expects 16 kHz, mono, 16-bit PCM WAV. A voice memo from an iPhone is an .m4a at 44.1 kHz stereo, so every file needs a pass through ffmpeg:

for f in memos/*.m4a; do
  ffmpeg -i "$f" -ar 16000 -ac 1 -c:a pcm_s16le "wav/$(basename "$f" .m4a).wav"
done

Then transcribe each WAV into a text file with timestamps:

for f in wav/*.wav; do
  ./build/bin/whisper-cli -m models/ggml-base.en.bin -f "$f" -otxt -of "txt/$(basename "$f" .wav)"
done

Local batch runs change the economics of transcription volume. Cloud speech APIs price by the minute, around fractions of a cent to a couple cents, so a year of archived voice memos is a real line item. A whisper.cpp batch over the same archive costs electricity and patience. For overnight jobs, large-v3 on an M-series Mac is a defensible accuracy choice because nobody is watching the clock.

What can whisper.cpp not do?

whisper.cpp cannot file, classify, summarize, or structure anything. It is a transcriber: audio in, text out. It does not know what a task is, what a project is, or that "remind me to..." signals something different from "I was thinking about...". It also does not decide when a transcript is finished; that is your shell script's problem.

Those limits matter for capture workflows. A folder of accurate transcripts is materially better than a folder of audio, but it is still a pile. Someone has to read each file, decide which sentences are actions, and move them into a system that tracks them. At memo volume, that someone is you, and at archive volume it is a scripting project.

The other honest limits: no speaker diarization (it cannot label who said what), no real-time streaming in the base CLI, quality that trails the best hosted models on very hard audio, and setup friction that assumes comfort with a terminal. If any of those are dealbreakers, the DIY path stops being cheap in the currency that matters.

Where does a managed tool fit alongside whisper.cpp?

A managed tool fits for the half whisper.cpp never had: turning speech into organized work. The transcript is the commodity, and whisper.cpp prices it at zero. The scarce part is the router that says a fragment is a task, belongs to the Apex project, and deserves a next step. Building that yourself means prompting an LLM over every transcript with project context, maintaining confidence thresholds, and designing the review UI, which is a product, not a weekend script.

For people who live offline-first, the interesting pattern is hybrid: local capture and local transcription, then a structured layer on top when back online. That is roughly the shape of an offline task manager: the queue holds captures without a network, and organization happens when connectivity returns. The permanent record can live as plain files too; our voice to markdown walkthrough covers getting transcripts into portable markdown, which pairs naturally with self-hosted transcription.

If you want the filing half done for you, with confidence thresholds and a review inbox, that is the problem an AI task manager like quik.md exists to solve. Whisper runs underneath many of those products anyway. The question is only who maintains the layer above it.

FAQ

What is whisper.cpp?

whisper.cpp is Georgi Gerganov's C and C++ port of OpenAI's Whisper speech recognition model. It runs offline on ordinary CPUs, with optional acceleration through Metal on Apple Silicon, and is released as open source under the MIT license.

Does whisper.cpp work fully offline?

Yes. After you download a model file once, every transcription happens locally with no network calls. That is the entire point of the port: privacy, zero per-minute cost, and transcription on planes, commutes, and anywhere with no signal.

Which whisper.cpp model size should I use?

For short voice memos in clear English, base.en at 142 MB is the sweet spot. For noisy audio, accents, or technical vocabulary, step up to small.en or large-v3. Tiny models are fast but miss enough words that the correction cost usually exceeds the time saved.

How accurate is offline whisper transcription?

Large models hit roughly 5 to 8 percent word error rate on clean English audio, close to the cloud versions. Smaller models trade accuracy for speed and memory. Noise, accents, and proper nouns degrade every size; large-v3 degrades least.

Can whisper.cpp organize my voice notes into tasks?

No. whisper.cpp outputs transcript text and nothing else. It has no concept of tasks, projects, or filing. Turning a folder of transcripts into organized work requires either your own scripts on top or a managed tool with a routing layer.

Related reads

• Voice to markdown: transcripts as portable files
• Voice to text for notes: the 2026 app guide
• The offline task manager workflow
• AI task managers: the pillar

References

• Georgi Gerganov, whisper.cpp, ggml-org, MIT license.
• Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, OpenAI, 2022.
• Whisper launch post and research summary, OpenAI.
• ffmpeg documentation, FFmpeg project.