MIDIBack: Harmony-Aware Automatic Pitch Correction Conditioned on Symbolic Accompaniment
What MIDIBack does
Automatic pitch correction tries to move an out-of-tune singing voice onto the notes the singer meant to hit. Many systems look only at the voice, or at hand-written music rules. They miss the harmony already present in the accompaniment.
MIDIBack’s goal is to recover those intended vocal pitches by reading the backing track as symbolic MIDI together with the vocal notes. Voice and band share one OctupleMIDI sequence. Self-attention over accompaniment tokens supplies harmonic context that a vocal-only model never sees. The network outputs corrected MIDI pitches per note. Offline APC (Praat TD-PSOLA via Parselmouth) then resynthesizes the audio for listening.
On clean transcriptions MIDIBack reaches 91.6% RPA. Under realistic pitch corruptions it reaches 83.9% RPA, ahead of vocal-only and spectra-conditioned baselines.
Abstract
Automatic pitch correction (APC) aims to correct out-of-tune singing voices to their intended musical notes. Existing systems often rely on rigid music-theoretic rules, vocal-only priors, or raw acoustic spectra, overlooking polyphonic and harmonic cues in the backing track. MIDIBack conditions APC directly on symbolic accompaniment by unifying vocal and backing notes into a shared OctupleMIDI sequence, so intended vocal pitches can be inferred from fine-grained musical context. On clean transcriptions MIDIBack reaches 91.6% RPA, and 83.9% RPA under realistic pitch corruptions, outperforming vocal-only and spectra-conditioned baselines.
Listening clips apply pitch edits with Praat TD-PSOLA via Parselmouth. Test audio comes from ccmixter and MIR-1K.
Architecture
Vocal and backing transcriptions enter one CNPP sequence (Adversarial-MidiBERT). Corrected MIDI pitches then drive offline APC resynthesis for the listening mixes below.
Overall architecture and training pipeline of MIDIBack.
Listening examples
Two pages share the paper’s evaluation corruptions (GRU detuner and global outshift) on open test clips.
-
Correction
Side-by-side mixes for each song. Start from the clean original mix, hear a GRU-only detuned vocal, then the harder detune+outshift vocal that is fed to every model. Compare the three corrections: full MIDIBack, BERT-APC (GRU-only ablation), and DDPC (Uniform).
-
Held-vocal case study
Keeps the singer’s original pitches fixed while the backing is transposed in a short window (±0.25, ±0.5, ±1 st on 4-bar and 8-bar regions, with ±1 s pad). Listen to the original mix, the same vocal over the shifted band, and MIDIBack’s correction over that shifted band. This probes whether the model follows harmony instead of memorizing absolute pitch.