TAPE: Learning General Audio Effect Representations

Phillip Long^1,2,*, Yongyi Zang^1,*, Qihui Yang¹, Christodoulos Benetatos¹, Junfei Liu¹, Randal Leistikow¹, Julian McAuley², Perry R. Cook¹

¹Smule, Inc.    ²UC San Diego    ^*Equal contribution

Abstract

Musicians and mixing engineers often can decipher the signal processing chain they hear on a reference recording, then adapt it to new material. We present TAPE (Transformations of Audio Parameter Embeddings), which estimates this processing from the processed reference audio alone. An encoder trained to identify processors in eight major effect families maps the reference to an embedding. A differentiable digital signal processing (DSP) chain decodes the embedding into editable parameters in one forward pass and applies them to a different unprocessed recording. Its modules are equalization, compression, waveshaping, a small residual network, and reverberation. TAPE achieves lower spectral error than DeepAFx-ST, the previous state of the art for single-step regression, and lower spectral error than per-example search over a chain approximating ST-ITO's, while running 275× faster than that search. Because the decoder outputs DSP parameters, its chain can also run causally: a C++ implementation with the trained weights processed 128-sample buffers 1.8× faster than real time, with no missed deadlines.

Method

TAPE (Transformations of Audio Parameter Embeddings) uses a shared audio encoder that maps a wet reference clip to an embedding z ∈ ℝ²⁵⁶ that does not depend on the source it was heard on. To decode z into human-readable primitives for signal shaping, the inverse g is a factorized differentiable chain of DSP modules for equalization, compression, waveshaping, residual color, and reverberation, each reading its parameters from z through a small head. The entire system is trained end-to-end on content-invariant pairs — the reference and target share the same plugin setting applied to different source recordings — so the only thing z can usefully describe is the treatment, not the content.

Audio Samples

Each row shows a dry input, the ground-truth reference (real plugin output), and TAPE's predictions before and after GAN fine-tuning. The system receives only the dry signal and a reference wet clip; it predicts plugin parameters to match the effect.

Text-to-Effect

A frozen sentence encoder is aligned to TAPE's embedding space, so a text description can stand in for a wet reference: the projected embedding z is decoded by the same DSP inverse and applied to the dry clip below. This demonstrates that the space is indexable from outside audio, not a retrieval benchmark.

Ablation Studies

DSP Training Strategy

We compare three strategies for training the per-module parameter MLPs: training each module separately, joint fine-tuning from separately-trained modules, and training all modules jointly from scratch.

---

Reverb Architecture

For reverberation, we compare a direct parameter regression approach (Param MLP) against an impulse-response GAN (IR GAN) that generates a full room IR conditioned on the tone embedding.

Citation

@article{long2026tape,
  title   = {TAPE: Learning General Audio Effect Representations},
  author  = {Long, Phillip and Zang, Yongyi and Yang, Qihui and Benetatos, Christodoulos and Liu, Junfei and Leistikow, Randal and McAuley, Julian and Cook, Perry R.},
  journal = {arXiv preprint arXiv:2610.40759},
  year    = {2026},
}