← kiryl.zip

In-browser ML denoising: porting DeepFilterNet3 and Resemble Enhance to WASM

July 17, 2026

noisles.co is my side project: an online tool that removes noise from audio and video files. It has been running for over two years and serves about 5,000 unique visitors a month — all organic.

The most interesting engineering decision in it: the ML denoising doesn’t run on a server. It runs in your browser.

Why client-side

Server-side ML inference for a free tool means the operating cost scales with usage. Client-side inference means the cost scales with… nothing. The entire infrastructure bill for noisles is roughly $75 a year — a small VPS and the domain.

There are also the boring-but-real wins: no upload latency for large files, and user audio never leaves their machine, which makes the privacy story trivially simple.

But noisles didn’t start client-side. It didn’t even start as a website. It started with tuk-tuks.

The tuk-tuk problem

A while back I went to Sri Lanka with friends, and like any reasonable person in the 2020s I documented the trip in Telegram video circles. Back home I discovered that half of them were unlistenable: tuk-tuks, buses, street chaos — a constant wall of background noise. Some of it just sat on top of my voice; some of it buried the voice completely.

My first idea was beautifully naive: audio is just voice + bus, so I’ll split the waveform in two, keep the voice, discard the bus. Spoiler: that’s not how it works. Modern denoising can peel noise off speech remarkably well — but where the noise completely drowned the voice at recording time, there is nothing left to recover. You can’t un-honk a horn.

Still, “remove most of the noise” was clearly achievable. So I went looking for an engine.

First shipped thing: a Telegram bot

The very first working version was a Telegram bot: forward it a noisy video circle, and it sends the same circle back with the background noise cleaned up. Recorded something and realized it’s drowning in traffic? Forward, wait, re-share. Received an unlistenable circle from a friend? Same move.

It never became a hit, but it worked — and it forced the actually important question: what does the denoising?

The low-hanging fruit: FFmpeg’s RNNoise filters

The first engine I tried was FFmpeg’s arnndn filter — an RNNoise-based neural filter with a default model plus a handful of community models tuned for different scenarios.

Integration was almost embarrassingly easy: ffmpeg.wasm already exists, compiled via Emscripten, complete with a virtual file system, and runs happily inside a web worker. No invention required — just package X into Y. The one piece of license homework in the whole stack lives here, too: the compiled FFmpeg core inherits FFmpeg’s LGPL/GPL terms, which just ask that the build’s source stays available — and an unmodified public build qualifies.

The results? Better, but not breakthrough. These filters leave a solid chunk of the noise behind. Nicer to listen to, nothing to write home about.

DeepFilterNet3, the server era

Then I found DeepFilterNet3, and it genuinely impressed me. Dual-licensed MIT/Apache — I checked before getting attached. One problem: it shipped as precompiled CLI binaries. Perfect for me and my experiments, useless for a normal person with a noisy video.

So the first integration was the obvious one: wrap the CLI in an API on my server. No rocket science, fully working — and entirely dependent on my server resources. DeepFilterNet isn’t particularly heavy, but “free tool whose costs grow with its popularity” is a business model with a built-in self-destruct button.

Formats made it worse. DeepFilterNet natively eats only WAV in a specific flavor, while real users bring mp3, mp4, m4a, and ogg (hello, Telegram). And converting to WAV before upload is a trap: a 1 MB mp3 balloons into 10–30 MB of WAV that you’d have to push through a possibly-mobile connection twice — up and back.

The pipeline I landed on juggled formats on both ends: the client converts whatever you upload into a compact mp3 with ffmpeg.wasm → mp3 goes to the server → server converts to WAV → DeepFilterNet3 does its thing → back to mp3 → back to the client → client converts to your original format. It worked well — compressed formats on the wire, WAV only where DeepFilterNet demanded it.

But the server was still in the loop. And I wanted it out.

The WASM port (with a confession)

Here’s the thing: the DeepFilterNet repo already had a WASM build of the core — merged in PR #452. The engine is written in Rust, and Rust compiles to WASM without drama. Compiling it was not the rocket science. Using it was: nobody had packaged that module to behave like the CLI tool — file in, clean file out.

The unlock was finding grazder’s raw web examples, which wired DeepFilterNet3 up as a real-time processor: you speak into the microphone, your voice comes out denoised. Great starting point — wrong shape for my case.

Now the confession: at that point I knew approximately nothing about Web Workers and browser worker contexts. This was early 2025 — DeepSeek R1 had just landed, Claude Code was still months from release, and “agentic development” meant either Cursor (I had no subscription) or a chat window (free). I took the chat window.

Together with R1, I reshaped that real-time code into an offline file pipeline: decode the WAV into a raw audio stream, feed it chunk by chunk into a Worker running the DeepFilterNet3 WASM module, collect the processed chunks, glue them back into a WAV, and convert that to the user’s original format with ffmpeg.wasm. All of it in the browser. The server’s job shrank to serving static files.

flowchart LR
subgraph browser
  A["noisy file"] --> M["mp3 · ffmpeg.wasm"]
end
subgraph server
  D["wav → DeepFilterNet3 → mp3"]
end
M -- upload --> D
D -- download --> M

flowchart LR
subgraph server
  ST["static files"]
end
subgraph browser
  N["noisy file"] --> DE["decode"] --> W["DeepFilterNet3 · WASM"] --> EN["encode · ffmpeg.wasm"] --> CF["clean file"]
end
ST -. serves .-> N

The whole engine lands as a ~16 MB one-time download: an 8 MB WASM module plus the ONNX model, shipped as a gzipped tarball and unpacked right inside the worker. Input handling is deliberately boring — everything gets normalized to 44.1 kHz mono up front, and the worker asks the WASM module for its preferred frame length at runtime rather than hardcoding it. And all of it runs single-threaded: no WASM threads, no SIMD, no SharedArrayBuffer heroics — the model is light enough to get away with it.

The numbers, measured end-to-end — decode, inference, encode, i.e. what a user actually waits: DeepFilterNet3 cleans audio at 3.26× real-time on my MacBook Pro (M3 Pro, 36 GB) in Chrome, with a cold start of about a second and a half. For a single-threaded WASM build, I’ll take it.

It was also a personal first: delegating development into territory I had no background in — worker plumbing, WASM internals — and shipping a working result, months before Claude Code and its siblings arrived. Until then, LLMs had only accelerated work I already knew how to do. This was the moment the ceiling moved.

Chasing studio quality: Resemble Enhance

DeepFilterNet3 shipped, but the hunt for quality didn’t stop. Eventually I hit Resemble Enhance — a different beast entirely. Not a CLI tool but a PyTorch package promising studio-grade sound, built as a pipeline of two models: a denoiser and an enhancer. Fully open source under MIT, runnable at home — if your definition of “home” includes Python, PyTorch, and the rest of the ML ecosystem. I knew the port would eat serious time, so the idea sat on the shelf for the better part of a year. By the time I picked it up, the tooling had caught up with my ambitions: Claude Code existed now — Opus or Sonnet under the hood, whichever one worked.

PyTorch has no browser story

What’s the state of PyTorch on the web? There is no PyTorch on the web. But there’s an exit ramp: PyTorch models can be exported to ONNX, and onnxruntime-web runs in the browser. Not every operation survives that trip — the export kept tripping over unsupported instructions, with torch.stft as the chief offender: ONNX simply can’t express it. So with duct tape and stubbornness I patched Resemble Enhance’s own inference code and drew the model boundary right at the spectral layer: the graph would consume and emit spectral tensors, and the Fourier transforms would become JavaScript’s problem. Remember that decision; it comes back.

flowchart LR
IN["noisy audio"] --> STFT["STFT"]
subgraph onnx["ONNX model"]
  M["denoiser"]
end
STFT -- spectrogram --> M
M -- spectrogram --> IS["inverse STFT"]
IS --> OUT["clean audio"]

Shipping half the pipeline

Then came the scoping decision that saved the whole port. The package’s headline act is the enhancer — the CLI treats denoising as an optional flag. But in my tests the enhancer would occasionally change the voice itself: you’d get a cleaner recording of someone slightly else. My users need their noise removed, not their vocal cords remastered — and the denoiser alone was already very good. So I cut the enhancer and shipped only the denoiser step. Half the pipeline, half the weight.

Two days in Fourier hell

The browser side followed the DeepFilterNet recipe: onnxruntime-web inside a worker, audio sliced into 10-second chunks with a small crossfade overlap so the seams don’t click, outputs glued back together. First full run: model loads, tensors flow, everything looks textbook — and the output is a solid wall of squeal and crackle.

I spent two days debugging that pipeline end to end. The ffmpeg conversions — fine. Browser-side WAV decode, sample rates — fine. Which left the STFT: the short-time Fourier transform that turns audio into the spectral form the model eats. Confession number two of this post: signal processing was never my domain. To me, Fourier was a commodity in a box: PyTorch calls one standard function, nothing custom, so surely any JS library with “STFT” on the tin does the same thing. The agent thought so too. First library: noise. Second library: different noise. Third: noise with extra steps. By the end of day two my motivation was at zero, and I called it a night.

The morning question was obvious in hindsight: why am I guessing which library is “compatible” when I can read what PyTorch actually does? Plot twist: torch.stft isn’t Python — it’s C++. So I found the actual C++ source, dropped it into the agent, and had it re-implement PyTorch’s STFT and its inverse in JavaScript on top of a known FFT library — reflection padding, window normalization, and a 1680-point window that isn’t a power of two, which quietly shoves the FFT into Bluestein’s algorithm. Next run: clean audio. After two days of library roulette, I just sat there staring at it.

Halving the model

My memory insisted the first export weighed 60–80 MB; the artifacts say the fp32 ONNX export was a rounder 41 MB — memory apparently billed me for the whole PyTorch checkpoint. Still far too much for a browser tool. The cure was half precision: ONNX Runtime’s transformer optimizer converted the weights to fp16 and fused the graph (943 nodes down to 412), landing the shipped model at 20.4 MB — an exact 2.00× cut with no quality drop my ears could catch. And for a denoiser aimed at human listeners, ears are exactly the acceptance test that matters. (int8 would halve it again; that experiment is a story for another post.)

The price of quality

Even at 20 MB, the model is heavy where it counts: compute. On my MacBook Pro it processes at 0.43× real-time — a six-minute clip takes about thirteen minutes to clean. If that’s the score on a MacBook Pro, you can guess how low-end devices feel about it — a real limit of client-side inference that’s worth being honest about. The one big lever still untouched is WASM threads plus SIMD, which need a cross-origin isolation setup noisles doesn’t ship yet — so there’s headroom. That’s the price of the quality tier — and it’s not mine to pay, so I hand the choice to the user: noisles offers a choice of models, with a fair warning that Resemble Enhance is the slow, resource-hungry one. Need it fast? DeepFilterNet3. Need every last drop of quality? Accept the wait.

Ears said yes — DNSMOS agrees

Subjective quality claims deserve numbers, so I scored both models with DNSMOS P.835 — a neural estimate of how a human panel would rate the audio (1–5, higher is better) — on two of my own recordings: one mildly noisy, one buried in street noise. OVRL is the headline score, and the delta against the noisy input is the number to trust.

Two things the table says. First, Resemble Enhance earns its price tag: it tops both clips, and on the badly damaged one it drags quality from barely-listenable to solidly mid-scale. Second, the worse the input, the bigger the payoff — on the mild clip DeepFilterNet3 is nearly a wash on OVRL (it trades a little speech fidelity for a quieter background), while on the heavy one it captures most of Resemble’s gain at ~7.6× the speed. Denoisers are for noisy audio; on nearly-clean audio they’re a shrug.

(Methodology honesty: non-personalized DNSMOS on two real clips, and the absolute scores carry a small resampling bias — the input→output deltas don’t.)

Takeaways

Client-side ML is a trade. You accept a device-capability floor and give up control of the execution environment; in return, serving costs drop to almost nothing and privacy comes by architecture — audio that never leaves the user’s machine can’t leak. For a solo-maintained tool, that trade wins by a mile: noisles serves 5,000 people a month on a $75-a-year bill.

The bigger lesson is about leverage. Both ports happened inside domains I didn’t know going in — worker contexts, WASM toolchains, DSP internals — and both shipped because I paired an agent that could handle the domain with my own judgment about scope, when to stop guessing, and where to look for ground truth. The domain knowledge sticks, too: nothing teaches signal processing quite like two days of squeal. Agent does the domain, I do the decisions — that division of labor is still how I build today.

And the tinkering continues: next on the bench is a half-finished branch that replaces ffmpeg.wasm with WebCodecs entirely. Meanwhile, if you’ve got a noisy recording lying around, noisles.co is free and runs right where you’re reading this — bring your own tuk-tuks.