aufklarer/DeepFilterNet3-MLX

DeepFilterNet3 — MLX

Real-time speech enhancement for Apple Silicon. Removes background noise from speech audio. FP32 MLX weights converted from the official DeepFilterNet3 checkpoint, with BatchNorm fused into the convolutions.

• 2.1M params, fp32 safetensors, 8.1 MB
• 48 kHz native, 10 ms frames, 2-frame (20 ms) lookahead
• Network-only export: STFT, ERB feature extraction, and deep-filter application live in the runtime (auxiliary.npz ships the exact DSP constants)

Model

Detail	Value
Architecture	DeepFilterNet3 (encoder + ERB decoder + deep-filter decoder)
Parameters	2,131,824 (BatchNorm fused, GRU biases folded)
Precision	float32
Sample rate	48 kHz (FFT 960, hop 480)
ERB bands / DF bins	32 / 96
Deep-filter order	5, lookahead 2 frames

Files

File	Size	Description
model.safetensors	8.1 MB	fp32 network weights in MLX layouts
auxiliary.npz	126 KB	ERB filterbank + inverse, Vorbis window, normalization init states
config.json	1 KB	Model and DSP hyperparameters
dfn3_mlx.py	9 KB	Pure-MLX reference implementation of the network

Quality

30 VoiceBank-DEMAND test clips (16 kHz mirror, resampled to 48 kHz for the model); the neural network forward runs on MLX while STFT / ERB / deep-filter post-processing stay in the upstream PyTorch pipeline — the same methodology used for the CoreML variant.

Backend	PESQ	STOI	SI-SDR
Noisy input	2.205	0.932	9.28
PyTorch FP32 (reference)	2.900	0.947	18.19
MLX, CPU stream	2.900	0.947	18.19
MLX, GPU stream	2.902	0.947	18.18

Network outputs match PyTorch within 9e-7 on the MLX CPU stream — numerically exact for fp32. The GPU stream uses Metal fast-math transcendentals (max output delta ~4e-3) with no measurable metric impact.

Latency (Apple M5 Pro, network forward only, Python MLX)

Duration	GPU	RTF	CPU	RTF
5 s	0.14 s	0.028	0.18 s	0.036
10 s	0.29 s	0.029	0.40 s	0.039
20 s	0.54 s	0.027	0.79 s	0.040

Usage

import sys

import mlx.core as mx
from huggingface_hub import snapshot_download

model_dir = snapshot_download("aufklarer/DeepFilterNet3-MLX")
sys.path.append(model_dir)
from dfn3_mlx import DFN3MLX

model = DFN3MLX(model_dir)

# Normalized features from your DSP front-end (constants in auxiliary.npz):
feat_erb = mx.zeros((1, 100, 32, 1))   # ERB features, dB-scaled + mean-normalized
feat_spec = mx.zeros((1, 100, 96, 2))  # complex spectrum (real, imag), unit-normalized

erb_mask, df_coefs, lsnr = model(feat_erb, feat_spec)

Full audio-in/audio-out enhancement additionally needs the DeepFilterNet DSP path (STFT → features → ERB mask + deep filtering → iSTFT), documented in the upstream repository.

On Apple devices, speech-swift runs this model directly via its MLX engine (or use the CoreML variant for the Neural Engine):

import SpeechEnhancement

let enhancer = try await SpeechEnhancer.fromPretrained(engine: .mlx)
let clean = try enhancer.enhance(audio: noisyAudio, sampleRate: 48000)

CLI:

swift run speech denoise noisy.wav --engine mlx

Tensor contract

Channels-last [B, T, F, C] — time is the conv H axis, frequency the W axis. The 2-frame lookahead shift is applied to the inputs inside the model.

Direction	Name	Shape	Notes
input	feat_erb	[B, T, 32, 1]	ERB features, dB-scaled, exp-mean-normalized
input	feat_spec	[B, T, 96, 2]	complex spectrum features, unit-normalized
output	erb_mask	[B, T, 32, 1]	sigmoid ERB gain mask
output	df_coefs	[B, 5, T, 96, 2]	deep-filter coefficients (order, real/imag)
output	lsnr	[B, T, 1]	local SNR estimate, dB in [-15, 35]

Weight conventions

• Conv2d: [O, kH, kW, I/groups] (PyTorch [O, I/g, kH, kW] transposed); causal time padding, symmetric frequency padding
• Depthwise ConvTranspose2d (erb_dec.convt{1,2}.dwt): [C, kH, kW, 1], true transposed-conv kernels (not pre-flipped); k=(1,3), freq stride 2, padding 1, output padding 1
• GRU (mlx.nn.GRU convention, gate order r/z/n): Wx = weight_ih, Wh = weight_hh, b = bias_ih + [bias_hh_r; bias_hh_z; 0], bhn = bias_hh_n; run with an explicit zero initial hidden state
• Grouped linear: [groups, in/groups, out/groups], applied as einsum("btgi,gih->btgh") then flattened
• BatchNorm is fused into the preceding conv; the unused df_fc_a head is dropped

Source

• Base model: Rikorose/DeepFilterNet v0.5.6 checkpoint (Apache-2.0 / MIT)
• Paper: DeepFilterNet3

License

• Model weights: Apache-2.0 / MIT dual license
• MLX conversion: Apache-2.0

Links

• speech-swift — Apple SDK
• Docs — install and CLI docs
• soniqo.audio — website
• blog — blog