Linux Development: Audio, SpectMorph, BEAST,…

The open source project I’ve been working on since quite some time is called SpectMorph. It is so far a partial answer to the question what makes a piano sound like a piano. The initial idea was that by analyzing single notes of a piano, it would be possible to decompose the samples into a representation that allows to reproduce the original sound, yet allowing advanced operations (in a better way than operating on the wave files directly).

If the SpectMorph model of a piano could capture the piano-like aspects of piano samples, and if the SpectMorph model of a trumpet could capture the trumpet-like aspects of trumpet samples, then a morphing algorithm could blend these two models into a sound that sounds somewhat between a piano and a trumpet. Thats the goal of SpectMorph in general. The kind of model that SpectMorph uses is that it decomposes the original wave files into a sum of pure sine waves, with time-varying magnitude and phase information. This is a much better representation for morphing samples – at least I think so – than the original wave files. Note that I haven’t invented something new here, the basic technique is known as Spectral Modelling Synthesis.

Yesterday, I released spectmorph-0.0.2, which can analyze piano samples (and probably others like trumpet or saxophone samples) and produce a fairly convincing imitation, based on the models alone. The morphing remains to be implemented, and other problems which become apparent during listening to original and SpectMorph versions of the sounds – I created a few listening examples – remain to be fixed; mostly the sound attack of the piano samples sounds harder than the SpectMorph reproduction.

So this is not end user digestable software yet, but if you’re interested in the current state of the code, you can find it here.

While working on the performance of SpectMorph, a project I’ll blog about later, using oprofile and valgrind –tool=callgrind both indicated that much time was spent in libm sincos() and/or libm sin(). Now the libm sincos() function uses the fsincos FPU instruction on x86 processors, and this instruction takes about 100 cpu cycles. If you need really really a lot of these sin values – like I do in SpectMorph – then this adds up to a significant percentage of all CPU time used.

So the first idea was to use a table and an interpolation scheme to implement an own sincos() function that is faster than the FPU version.

However, after thinking again, there is an even better way, if you need sincos (phase), sincos (phase + phase_inc), sincos (phase + 2 * phase_inc) … sincos (phase + n * phase_inc). So if what you really want is creating a sine/cosine wave, there is a trick:

Using a complex number for the start phase c_start, and a complex number for the phase increment c_inc, you can rewrite the above as c_start, c_start * c_inc^1, c_start * c_inc^2, c_start * c_inc^3 and so on. So the whole task is reduced to multiplying c_start with c_inc again and again. From my measurements, implementing this reduces the amount of work to about 10 cpu cycles. To get good precision, every once in a while, the code should do a real sincos() call to compute a new c_start.

Is that all that can be optimized? No. The math can be made even faster when using SSE instructions. Then you can compute four of these complex multiplications at once, so you effectively end up with about 3.5 cpu cycles per sincos() value, or less than 2.5 cpu cycles if you only need sin() values. In this case the imaginary part of the complex value needs to be computed, but not the real part.

Here is the source code for generating an array of sin()/sincos() values. Note that you need to compile with -msse for SSE instructions, and that all float arrays need to be 16-byte aligned.

Also note that if you need the weighted sum of many sine waves, you might be better off using an inverse FFT; I need that sometimes but not always, so this case can be optimized seperately.

I’ve released gst123 to the public today. Its a command line tool like ogg123 or mpg123, capable of playing MP3, Ogg, FLAC and whatever other audio files gstreamer supports. So it works well on music collections where not all files are encoded in the same format.

Since I’ve been using (and thus testing) it for quite some time now, it should just work. You can grab it here.

From a composers point of view, it would be nice to have a piece of software which models a grand piano and covers the complete range of expressions which make a piano a piano. For instance, if you hit the same note harder (with higher velocity), the sound should not only be louder, but also sound more aggressive. Also using the sustain pedal has a significant influence on the sound (and cannot be simply modelled as not releasing the notes that are already pressed. Finally, and maybe hardest to model, different notes inter act with each other (one string putting another into resonance).

So, dear lazyweb, I do know commercial products for windows that deliver high quality piano sound, either as huge sample sets, or via physical modelling. But are there comparable high quality pianos which are open source?

Since I am planning a project which relies on storing encrypted information on a potentially hostile host, I tried to design a cryptographic mode, called mode FMC. This mode should, when combined with AES, provide a higher security margin than AES alone would. Basically mode FMC is a try to defend your private data (like credit card numbers or password lists stored in the application I am going to write) against yet to be discovered weaknesses in AES. As with any cryptographic project, it would be great to get review.
I’ve written a paper and published the code under LGPL. The code is implemented in C++ and includes a python binding.

I’ve spent quite some time on producing a Windows Port of BEAST (a sequencer and modular synthesizer, formerly only available for Unixoid systems). Since I got tired of manually configuring / compiling packages, I implemented a python based build system like FreeBSD ports or Gentoo emerge. This build system works fine for BEAST, but it could also be used to maintain other ports of Gtk or Gnome software. Its called BeastBuild and installs everything required to build BEAST, like mingw/msys, gtk+, libguile and portaudio. Finally a user friendly installer is created, so if a user just wants to use BEAST, without building it, he can just simply use the installer. BeastBuild and the BEAST Installer are available here.

The german computer company TWC is offering commercial support for BEAST now. After having supported aRts and BEAST development by paying travel expenses, providing bandwidth, web space and mailing list services, this is the next logical step. So if you want a specific bug to be fixed, or buy a computer with BEAST preinstalled, you can now.
* german version of the BEAST/TWC page
* english version of the BEAST/TWC page

The sequencer and modular synthesis software BEAST is now member of LinuxAudio.Org. This is good news, because most musicians don’t know yet that Linux is a great alternative for making music. Supposedly its too hard to install, maintain and use; but if you look at what experienced users really can do with Linux, this seems like an unjustified opinion formed years ago, when Linux admittedly was not so easy to use. But these days are over. Fortunately 🙂

The Fantasy Project is an annual challenge. Each artist who participates can create his own song, as long as he uses the lyrics given below (“Jean Lucs Poem”) unchanged. All styles of music, such as classical, blues, rock, metal, rap, dance, techno, house, … are welcome.
* Fantasy III
* Fantasy II
All songs are under a Creative Commons license.

Since Monday, I am working for MCS Hamburg, which is a quite professional internet provider and systems integrator. This means getting up earlier than I usually needed to, since I have to be here from 9am until 5pm. Howerever, this also means that I will be available on IRC, ICQ and via phone during these times in March. The MCS staff is very nice, and I enjoy working here a lot :).