sintvert, a real-time wave-to-MIDI server for known waveforms

It's nice to be able to input note events from a hardware MIDI-enabled keyboard, rather than from a "virtual" mouse-based (or computer keyboard-based) one like jack-keyboard or vmpk. However, with software support, even non-MIDI (audio-only) sources can be translated to MIDI.

I have used Paul Brossier's excellent aubio package with mixed success in the past. Pitch recognition is a complex problem, and for MIDI purposes, even a 3% error in the detected frequency can swing the output by a semitone. Since most voices likely to be available as audio inputs (e.g. from an old keyboard, or from a real instrument) will have some amount of LFO, it's hard to get an error-free reading, at least without using a large delay line in the detector.

One way of easing this problem is to train the pitch detector on the entire range of possible audio signals to be recognized. For a 3-octave instrument, for example, this means training on just 36 waveforms. The detector can thus extract features from all potential inputs during training, and then work reliably even with a small buffer (which translates to a shorter delay between the input signal and the detector output). Of course, since the training signal presumably needs to include at least a half-period of the waveform, low notes in the input range will require greater latency (e.g. a C2 is about 65.4 Hz, or 15.3 / 2 = 7.15 ms half-period).

This is the approach I have taken in sintvert. It's a Jack application written in C, so it should be fairly easy to compile and install on a modern distro.


Catch up with the audio and music tools on Ubuntu Hardy

Updates: per readers' request, I have added Hardy backports to all packages mentioned in this article (except Pulseaudio where the appropriate section isn't clear) in my PPA. I have also added newer ALSA packages.

Many Ubuntu users have chosen to stick out with Hardy, the Long Term Support (LTS) release, supported until April 2011 (or 2013 for the server edition). But support mostly means security fixes, not necessarily updated versions of popular software; the Hardy Backports project doesn't necessarily keep pace with our favorite packages. In my case, I wanted the latest version of Rosegarden (1.7.2), the audio / MIDI / score editor and sequencer.

Fortunately, there is a way to enjoy fresh software while still postponing the dreaded dist-upgrade (or clean reinstall) marathon for as long as Hardy remains supported. The solution is to download source packages from later distributions, compile them on your system, and install the resulting binary packages.

You can either add entries to /etc/apt/sources.list and then run apt-get source ... (disruptive because it enables "future" versions for all packages), or use dget manually:

dpkg-source -x pulseaudio_0.9.10-3.dsc

Either way, to build the package you then

cd pulseaudio-*
debuild -uc -us -b
cd ..; dpkg -i # your packages here

(if the build fails you may need to install extra -dev libraries). Some good places to scout for updated versions include:

In all cases, you can go directly to a package by appending its name to the above URL's; and once you got to a package page, you will find a link to the .dsc file you need to dget (as explained above) on the right side. Keep in mind that not all packages will compile on Hardy — you will have to experiment.

Back to audio packages — the reason I mentioned the Debian Sid pulseaudio package is because (unlike Ubuntu) it includes module-jack-sink, which allows Pulseaudio to run on top of the low-latency jackd daemon. This means you can have Jack and some music applications (like a soft synth and Rosegarden) running perfectly, and still be able watch YouTube videos (without having to kill and later restart Jackd). As with any Pulseaudio on Hardy setup, you will still need libflashsupport in order for Mozilla to be able to connect to Pulseaudio (same with Opera).

I also recommend the following: jackd 0.1.116 (again from Debian Sid); Fluidsynth 1.0.8 (from Intrepid); qjackctl 0.3.4 (from Ubuntu Jaunty); and finally, rosegarden 1.7.2, for which I couldn't find any distribution to leech from — use the version in my PPA (obtained by updating the Intrepid package to the latest upstream release). The PPA page contains the instructions for enabling it in APT.