Whisper: The Most Reliable Free Voice Input Tool I’ve Kept Running Since I Started Developing with AI

Have you ever used voice input? You speak, and it turns what you say straight into text. No typing required.

 

If you want to build your own voice input tool, Whisper is currently the most reliable free option.

 

Ever since I started developing with AI, it’s been the only program I’ve kept running on my desktop without ever closing it. I dictated the first draft of this article, too.

 

If you want to use AI to develop a program you’ll keep using over time, I’d suggest trying to build your own speech recognition app that stays running in the background, once you’re ready.

 

In the AI era, AI does everything for you, especially digital work like this. Eventually, you don’t even feel like typing anymore. Just talking is the easiest option.

 

Plenty of people already send voice messages when chatting on LINE, the messaging app, because they can’t be bothered to type.

 

Right now, though, the iPhone’s built-in speech recognition leaves a lot to be desired. It gets things wrong so often that it barely feels usable. If you’ve been having that problem, give Whisper a try.

 

So let’s take a look at Whisper!

 

Whisper is a speech-to-text model made by OpenAI. There’s an open-source version you can install and run on your own computer for free. If you don’t want to install it, you can pay to use the API: send audio in and get text back.
(source: GitHub "openai/whisper")

 

It comes in several sizes, from the smallest, tiny, through base, small, and medium, to the largest, large. There’s also a faster version called turbo. The bigger the model, the more accurate it is, and the more it demands from your computer.

 

So you can’t just install it on any machine and expect it to run. I’ve tried installing it on a phone. An iPhone 17 Pro can manage roughly up to medium; anything bigger doesn’t run smoothly.

 

If you can’t run it locally, you’ll have to use a paid API. Groq does currently offer a free allowance, though.

 

Here are a few situations I regularly use it for:

 

 

Use #1: Talk Instead of Typing

The most straightforward use is to stop typing and talk to AI instead. The same goes for writing articles. Just dictate them.

 

Of course, it still mishears things.

 

There are some additional correction methods you can try. But Whisper itself is a generative model, so adding any correction, or running another correction pass after transcription, slows the whole process down. Whether that’s worth it is up to you.

 

Generally, saying more at a time actually improves the recognition rate. You’re more likely to get a result where you don’t need to change anything.

 

These days, though, AI Agents can figure out what I mean about 90% of the time, even with transcription errors. They may have been specifically tuned for this, which would explain why AI is now pretty forgiving of those errors.

 

 

Use #2: Make Subtitles

If you have a video without subtitles, you can use it to generate them.

 

But it can’t actually listen to too much at once, so you need to split the audio up and merge the results.

 

If you’re having trouble getting the splits and joins right, you can display the audio waveform. Generally, where nobody is speaking, the waveform visibly disappears: it’s just empty. That lets you line up what’s being said with the timeline.

 

Matching every individual word perfectly to a timestamp isn’t easy. You can do it to some extent, but I don’t think that level of precision is necessary in most cases.

 

Matching sentences to their timestamps is much easier. Once you can line up the sentences, you have your timeline. Video and audio files already have timing built in, after all. With that timeline, the subtitle file can include timestamps.

 

For our subtitles, we split the audio at pauses into chunks of about 60 seconds, cutting at the last pause between the 50- and 60-second marks. We once tried feeding it 10 minutes at a time. It broke words apart and ran sentences together. Even English words got split up.

 

Another thing to watch out for: if you send it a segment where nobody is speaking, it makes up text on its own, like “Lyrics, music, arrangement: Hatsune Miku,” “Thanks for watching,” or “Subtitles provided by the community.” That’s because much of its training used existing subtitle files, and subtitling groups often put those messages in silent sections. Whisper thinks it needs to produce that text when there’s no sound. So we simply don’t send segments where nobody is talking, and we discard any output containing those phrases.

 

Of course, the transcription will often still have errors. Send it through an AI Agent one more time to correct them, and your subtitle file is basically done.

 

Once everything is set up, you can get a subtitle file in a dozen or so seconds, or at least in under a minute.

 

 

Use #3: Have a Back-and-Forth Conversation with an AI Agent

You’ll also use it if you’re building an AI assistant.

 

That’s because an AI assistant takes text as input. Right now, it doesn’t take audio. Even if it does in the future, it’ll actually be turning that audio into text first and feeding it back in. The AI company is simply doing one extra step for you, and possibly charging you for that extra step, too.

 

So Whisper may be necessary if you want to talk to an AI assistant. You speak, it turns your speech into text, and that text gets sent in.

 

At that point, conversion speed really matters.

 

If you want talking to your AI Agent to feel like talking to a person, the Agent might respond within 0.5 seconds after you finish speaking. But if the conversion in between takes five seconds, the whole conversation becomes stop-and-start.

 

So your computer being able to run it and your computer being able to run it smoothly are two different things. In an interaction, you really feel that difference in timing.

 

 

Use #4: Have It Listen to Generated Speech to Check Whether It’s Right

This is an extension of the second use.

 

Once you can generate subtitles, you can also automatically correct AI-generated speech.

 

After all, AI-generated speech often comes out sounding strange. Automatic correction means having Whisper listen to the audio and check whether the sentence came out right. If there’s a problem, rephrase it and generate it again.

 

Of course, this won’t be 100% reliable. But before you start correcting the speech yourself, Whisper has already done a first pass. At least the parts it can’t make out will have been flagged, which can save you most of the time you’d otherwise spend.

 

But could it mishear something? Yes.

 

So in some cases, you will end up wasting resources. Generating speech might cost money, and Whisper might cost money, too. But the time you save still makes it worthwhile. You can spend that time doing something else.

 

And if, like me, you generate speech locally and run Whisper locally too, it doesn’t cost money, just time. Well, it’s not really free: when I run both at once, my computer gets a bit sluggish because it isn’t very new.

 

 

If It Won’t Run Smoothly, Use a Remote Model—but Your Voice Gets Sent Out

If your computer can’t run it at all, or can run it but not smoothly enough, you’ll need to consider a remote model and call a paid API directly.

 

Remote services generally cost money because they require computing resources. Some offer a free allowance. We usually use Groq, which gives you quite a generous allowance.

 

One thing to keep in mind is that your voice gets sent to a remote service, so it all gets recorded. Exactly how it handles your voice is hard to say.

 

But given the sheer volume it handles, I think it comes down to whether you’re comfortable with that. For people like us who already have a lot of voice recordings online, our voices have actually already been used for training. Generally, though, it won’t specifically target an individual user.

 

Even if it does target you, producing your actual voice isn’t necessarily easy. The sound a microphone picks up is a little different from the sound played back in your videos.

 

Of course, you can also use OpenAI’s services directly. Most people have already sent their data to OpenAI anyway. And since OpenAI made Whisper, it offers voice services for more situations, including more real-time options. Naturally, the pricing differs too.

 

 

A Better Computer Makes a Difference in How Smoothly It Runs

Although I use it every day, we ran an actual test for this article: running Whisper locally on a Mac with an M2 Max, it transcribed a two-minute course recording in 5.5 seconds, including the time it took to load the model.

 

I’ve also used an external graphics card, a 3090. For a single sentence, there wasn’t much difference between the 3090 and my Mac M2. Having more computing power didn’t make it run faster. There may be room to tune the 3090 further, but I haven’t looked into that yet.

 

I think computers from 2027 or 2028 onward, or a higher-end computer you buy today, could run it a bit better.

 

The test used whisper.cpp 1.8.3 with the large-v3-turbo model, run by Claude Code on my Mac with an Apple M2 Max. The source material was a two-minute recording from an old Progress Bar course.

 

 

If You Work with AI, Whisper Is Almost a Must-Have

So those are the four things I use Whisper for: talking directly to AI, making subtitles, having back-and-forth conversations with an AI Agent, and checking generated speech.

 

The first and third are actually the same thing. They just look a little different because the situations are different.

 

AI still only takes text as input. If you want to speak, something has to convert it for you along the way. If your computer runs it smoothly, run it locally for free. If it doesn’t, use Groq’s free allowance. If you want something more immediate and more accurate, pay for OpenAI.

 

Since Whisper is free, there are actually plenty of other providers offering services like this.

 

As for which one works best, or has been tuned more, that’s still hard to say. This technology keeps improving over time.