There’s a specific kind of silence that happens in a demo when the AI Agent mishears something critical. The customer is sitting across from you, the call goes through, the bot transcribes “cancel my order” as “handle my border” — and the conversation moves on awkwardly while you make a mental note to figure out what just happened.
That moment happened to us more than once. And the root cause, every time, was not the ASR model. It was the audio arriving at the ASR model.
Context: How We Handle Media at Exotel
Exotel runs stream — our internal WebSocket and media handling layer. It sits between Asterisk and the downstream AI systems: Voice AI Agents, ASR engines, analytics pipelines. Agentstream manages WebSocket connections at scale, handles the media session lifecycle, and routes audio to wherever it needs to go. It’s one of the most battle-tested components in our stack — high uptime, predictable behavior under load, years of production hardening.
When a Voice AI call happens on Exotel — a voicebot answering a customer, an AI agent handling a support call — the audio flows through AgentStream. We own that path end to end.
The problem was what that audio looked like before it reached the bot.
What Voice AI Actually Receives
A contact center caller in Pune is on a mobile network. The agent floor has 200 people on concurrent calls. By the time that audio arrives at AgentStream, it has passed through a mobile codec, a PSTN trunk, a SIP gateway, and Asterisk — carrying noise from every hop.
The ASR engine then receives 8kHz G.711 PCMU at 10 dB SNR and is expected to produce clean transcription for the Voice AI to act on. No ASR model is designed to fix bad input audio. That is simply not what they do. The result is mis-transcriptions, missed intents, and Voice AI that sounds broken — not because the AI is bad, but because the audio feeding it is.
For Voice AI to work reliably on real telephony — not on clean studio audio, not on a demo with a good headset, but on actual PSTN calls at scale — the audio needs to be cleaned before it reaches any AI system. This is the problem ClearStream solves.
Three Options, One Right Answer
Option 1: Asterisk’s built-in denoiser
Asterisk ships with a native noise reduction filter. It’s free, it’s there, and it was the first thing we looked at.
The problem is control. It’s a static DSP filter applied at the channel level. You turn it on or off in configuration. There’s no per-call control — you can’t enable it for one voicebot call and disable it for another. You can’t adjust aggressiveness based on measured noise. You can’t combine it with AGC or VAD in a coordinated pipeline. On a real Voice AI deployment with diverse noise environments and diverse customer requirements, a static non-configurable filter is not sufficient.
Option 2: A third-party noise cancellation SDK
The market has capable options. But embedding any of them into AgentStream means giving up the control that makes AgentStream reliable.
CPU and IRQ behavior. Neural noise suppression models have real CPU cost. At our call volumes, that shows up as IRQ spikes and unpredictable load curves on the media server. When that happens on the media plane, it doesn’t just affect audio quality — it affects the reliability of every concurrent session on that server. We’ve spent years making AgentStream stable under load. A third-party library we don’t own introduces a variable we can’t control.
Customization. Commercial SDKs are black boxes at the feature level. We needed per-call noise suppression aggressiveness, per-call AGC tuning, and live mid-call parameter updates from the application layer. None of the options we evaluated offered this.
Operational visibility. On our stack, we can instrument everything. With a third-party SDK, you get the telemetry surface they chose to expose. When something goes wrong at 2am on a live Voice AI call, that’s not enough.
Versioning. A third-party SDK upgrade is a risk event on the media path. We upgrade on our schedule and our rollout controls, not the vendor’s release calendar.
The fundamental issue: we would be trading the control we’ve built in AgentStream for convenience that doesn’t fit our operational model.
Option 3: Build it as a first-class service in our stack
We built ClearStream — an audio enhancement SDK in Go — that plugs into AgentStream as a native component. We own the code, the behavior, the instrumentation, and the upgrade path. It runs on the same infrastructure as AgentStream, on CPU, at the same scale.
How ClearStream Works
ClearStream sits in the AgentStream AudioSocket path between Asterisk and the downstream Voice AI system. Raw RTP audio (G.711 PCMU/PCMA, 8kHz) arrives, gets decoded and upsampled to 16kHz PCM, passes through the enhancement pipeline, gets resampled back to 8kHz, re-encoded, and forwarded. The Voice AI bot receives clean audio. The entire process adds under 4ms p50 latency — well within budget for a real-time voice call.
The pipeline has five independently controllable stages, each on or off per call:
Noise Suppression using RNNoise (ships with the binary, no external dependencies) or DeepFilter (ONNX, ~25 dB SNR improvement for higher-quality requirements). We built TieredNR on top — a three-tier SNR ladder that selects suppression level per frame based on measured noise floor, so quiet calls don’t get over-processed.
AEC (Acoustic Echo Cancellation) via an NLMS adaptive filter — 256-tap for narrowband, 512-tap for wideband. Converges in ~2 seconds. Handles speakerphone callers and headset echo bleed, both of which reliably break ASR on Voice AI calls.
VAD (Voice Activity Detection) in static or adaptive mode. Skips the suppression pass on silence frames, saving ~30% CPU at scale. On a high-concurrency deployment, this matters.
AGC (Automatic Gain Control) — the most important and least obvious piece. When RNNoise suppresses a noisy signal, the output level drops significantly (we measured ~58% RMS drop on some samples). The AGC restores the output to -18 dBFS — the sweet spot for most ASR engines — using attack/release envelopes and a soft limiter that prevents hard clipping. This is what actually moves WER numbers. No commercial pre-ASR SDK we evaluated documents AGC at all.
Diarization via an energy-based speaker separator, surfacing per-frame speaker labels. Useful for analytics on multi-party Voice AI calls.
{Coming Soon}
Per-Call Control via the API
For Voice AI, one-size-fits-all audio enhancement doesn’t work. A simple IVR call has different requirements from a voicebot handling a noisy contact center floor. A compliance recording pipeline has different requirements from a real-time bot.
ClearStream exposes per-call configuration through AudioEnhancement[...] parameters on the Calls/connect API. Every feature is off by default — no behavior change for existing calls unless explicitly set. A minimal Voice AI call looks like:
curl -X POST \
'https://<api_key>:<api_token>@api.exotel.com/v1/Accounts/<AccountSid>/Calls/connect' \
-F 'StreamType=bidirectional' \
-F 'StreamUrl=wss://your-bot.example.com/media' \
-F 'From=+91XXXXXXXXXX' \
-F 'CallerId=0XXXXXXXXXX' \
-F 'AudioEnhancement[NoiseSuppression]=true' \
-F 'AudioEnhancement[AGC]=true' \
-F 'AudioEnhancement[AGC.Preset]=asr_ready' \
-F 'AudioEnhancement[VAD]=true'
And if the bot detects mid-call that the caller has stepped outside into noisy traffic, it can update the enhancement level live via the Legs API — no session restart, no audio gap:
curl -X POST \
'.../Calls/<CallSid>/Legs/<LegSid>' \
-F 'AudioEnhancement[NoiseSuppression.Aggressiveness]=3'
Under the hood, that hits Pipeline.SetAggressiveness(3) — an atomic write, under 1ms, with zero impact on the in-flight audio stream.
The POC and Where We Are Now
We ran a POC with ClearStream plugged into the AgentStream AudioSocket path on a set of live Voice AI call samples. SNR improved measurably, ASR transcription improved, and the processing added under 4ms p50 latency — well inside our real-time budget. Importantly, CPU behavior under load was predictable because we control the code.
The POC answered “can it work.” The work we’re doing now answers “can it be reliable at production scale.”
That means soak testing at 1K concurrent sessions, CPU and IRQ profiling at production call volumes, CI gates for SNR regression and ClipCount, and Grafana dashboards showing per-session metrics in real time. We’re targeting production launch by end of October — starting with a subset of Voice AI traffic, measuring ASR improvement on real calls against baseline, and expanding from there.
It Does More Than Live Voice AI Calls
The primary use case is real-time enhancement for Voice AI. But the same SDK — same binary, same signal processing — also works in two other modes that are useful additions.
Post-processing recorded calls. Run existing recordings through ClearStream to improve audio quality retroactively. Contact center QA teams reviewing calls for compliance and coaching get better audio. ASR pipelines running on stored recordings get cleaner input. Training data for ASR fine-tuning gets denoised before it goes into the model.
# Single file
./clearstream file -i noisy_call.wav -o clean_call.wav --agc --agc-target-rms 4124
# Batch a directory
./clearstream dir -i ./recordings/ -o ./enhanced/ --workers 8
Standalone RTP proxy. Drop it between your SBC and an agent endpoint with no code changes.
./clearstream rtp --listen :5004 --forward agent:5004 --codec pcma
HTTP enhancement API. For pipelines that work with audio files and want a service interface.
./clearstream server --http :8080
curl -F audio=@noisy.wav 'http://localhost:8080/enhance?agc=true' -o clean.wav
These aren’t the primary purpose — they’re useful additions that fall naturally out of the same SDK.
Try It
ClearStream is open source under MIT. If you’re running Asterisk, FreeSWITCH, Kamailio, or any SIP/RTP infrastructure and dealing with noisy audio on Voice AI calls, clone it and run a POC in a few minutes.
git clone https://github.com/Saurabhsharma209/ClearStream
cd ClearStream
# Build (pure Go — passthrough mode, for testing)
go build ./cmd/clearstream/
# Build with RNNoise (production noise suppression)
go build -tags rnnoise ./cmd/clearstream/
# Process a file immediately to see the difference
./clearstream file -i your_noisy_call.wav -o clean.wav
# Measure SNR improvement
go run tools/snr_benchmark/main.go
# Full POC demo
make poc
Supported codecs: G.711 µ-law (PCMU), G.711 A-law (PCMA), G.722, Opus. Supported platforms: Asterisk 18.x/20.x/22.x, FreeSWITCH 1.10.x, Kamailio 6.0 + RTPEngine 11.5, Janus.
We built ClearStream because Voice AI on real telephony requires clean audio, and clean audio requires owning the enhancement layer — not borrowing it from a vendor who doesn’t know our stack. The POC proved the approach. October is when it goes to production.