Linux Development: Audio, SpectMorph, BEAST,…

The editors at FamousWhy have conducted an interview about gst123, which has been published today on their site. So if you’re interested in gst123, this might be useful to read.

Release early, release often is one of the most valuable things to keep in mind when doing free software development (if you haven’t heard of this advice before, try reading “The Cathedral and the Bazaar”). So… last couple of days I’ve spent making new releases of the software I am maintaining.

* The Foldroid mail folder selection tool should in the 0.0.2 release build out of the box on OpenSUSE and other systems that have ncurses includes not in /usr/include
* gst123 media player has seen a lot of improvements since the initial release, thanks to community input. The gst123-0.1.1 release should be much better for watching videos, while it maintains the ability to be run in text only mode.
* SpectMorph 0.0.3 should sound much better than any previous release; its still work in progress, but if you listen to the piano samples for 0.0.2 and 0.0.3 (on the web page), you should hear much better sound reproduction of the attack of the piano.

If you happen to find problems with any of those release, let me know 🙂 For gst123 there are new fixes in the git repo that were too late for the release, so if you run into bugs there, try git first. I will make another gst123 release in the nearby future.

I’ve been using mutt to manage my e-mail for many years now, and the one thing I’ve been missing is a nice overview over the folders that I have (due to many mailing lists I read, I use procmail for putting each list in a seperate folder). So I’ve written a curses based tool that lets you choose which (mbox style) mailbox you want to read, and it starts mutt with that mbox. It also shows the number of messages in each mbox, and the number of new messages. In my setup, I use it together with screen, so that each mutt is started in a new screen, which allows me to have many folders open at once.

So I’ve now fixed a couple of annoying things in foldroid, and made the initial public release. Its LGPLd and can be found on The Foldroid page, which also has a screenshot of foldroid at work.

The open source project I’ve been working on since quite some time is called SpectMorph. It is so far a partial answer to the question what makes a piano sound like a piano. The initial idea was that by analyzing single notes of a piano, it would be possible to decompose the samples into a representation that allows to reproduce the original sound, yet allowing advanced operations (in a better way than operating on the wave files directly).

If the SpectMorph model of a piano could capture the piano-like aspects of piano samples, and if the SpectMorph model of a trumpet could capture the trumpet-like aspects of trumpet samples, then a morphing algorithm could blend these two models into a sound that sounds somewhat between a piano and a trumpet. Thats the goal of SpectMorph in general. The kind of model that SpectMorph uses is that it decomposes the original wave files into a sum of pure sine waves, with time-varying magnitude and phase information. This is a much better representation for morphing samples – at least I think so – than the original wave files. Note that I haven’t invented something new here, the basic technique is known as Spectral Modelling Synthesis.

Yesterday, I released spectmorph-0.0.2, which can analyze piano samples (and probably others like trumpet or saxophone samples) and produce a fairly convincing imitation, based on the models alone. The morphing remains to be implemented, and other problems which become apparent during listening to original and SpectMorph versions of the sounds – I created a few listening examples – remain to be fixed; mostly the sound attack of the piano samples sounds harder than the SpectMorph reproduction.

So this is not end user digestable software yet, but if you’re interested in the current state of the code, you can find it here.

While working on the performance of SpectMorph, a project I’ll blog about later, using oprofile and valgrind –tool=callgrind both indicated that much time was spent in libm sincos() and/or libm sin(). Now the libm sincos() function uses the fsincos FPU instruction on x86 processors, and this instruction takes about 100 cpu cycles. If you need really really a lot of these sin values – like I do in SpectMorph – then this adds up to a significant percentage of all CPU time used.

So the first idea was to use a table and an interpolation scheme to implement an own sincos() function that is faster than the FPU version.

However, after thinking again, there is an even better way, if you need sincos (phase), sincos (phase + phase_inc), sincos (phase + 2 * phase_inc) … sincos (phase + n * phase_inc). So if what you really want is creating a sine/cosine wave, there is a trick:

Using a complex number for the start phase c_start, and a complex number for the phase increment c_inc, you can rewrite the above as c_start, c_start * c_inc^1, c_start * c_inc^2, c_start * c_inc^3 and so on. So the whole task is reduced to multiplying c_start with c_inc again and again. From my measurements, implementing this reduces the amount of work to about 10 cpu cycles. To get good precision, every once in a while, the code should do a real sincos() call to compute a new c_start.

Is that all that can be optimized? No. The math can be made even faster when using SSE instructions. Then you can compute four of these complex multiplications at once, so you effectively end up with about 3.5 cpu cycles per sincos() value, or less than 2.5 cpu cycles if you only need sin() values. In this case the imaginary part of the complex value needs to be computed, but not the real part.

Here is the source code for generating an array of sin()/sincos() values. Note that you need to compile with -msse for SSE instructions, and that all float arrays need to be 16-byte aligned.

Also note that if you need the weighted sum of many sine waves, you might be better off using an inverse FFT; I need that sometimes but not always, so this case can be optimized seperately.

I’ve released gst123 to the public today. Its a command line tool like ogg123 or mpg123, capable of playing MP3, Ogg, FLAC and whatever other audio files gstreamer supports. So it works well on music collections where not all files are encoded in the same format.

Since I’ve been using (and thus testing) it for quite some time now, it should just work. You can grab it here.

From a composers point of view, it would be nice to have a piece of software which models a grand piano and covers the complete range of expressions which make a piano a piano. For instance, if you hit the same note harder (with higher velocity), the sound should not only be louder, but also sound more aggressive. Also using the sustain pedal has a significant influence on the sound (and cannot be simply modelled as not releasing the notes that are already pressed. Finally, and maybe hardest to model, different notes inter act with each other (one string putting another into resonance).

So, dear lazyweb, I do know commercial products for windows that deliver high quality piano sound, either as huge sample sets, or via physical modelling. But are there comparable high quality pianos which are open source?

Since I am planning a project which relies on storing encrypted information on a potentially hostile host, I tried to design a cryptographic mode, called mode FMC. This mode should, when combined with AES, provide a higher security margin than AES alone would. Basically mode FMC is a try to defend your private data (like credit card numbers or password lists stored in the application I am going to write) against yet to be discovered weaknesses in AES. As with any cryptographic project, it would be great to get review.
I’ve written a paper and published the code under LGPL. The code is implemented in C++ and includes a python binding.

I’ve spent quite some time on producing a Windows Port of BEAST (a sequencer and modular synthesizer, formerly only available for Unixoid systems). Since I got tired of manually configuring / compiling packages, I implemented a python based build system like FreeBSD ports or Gentoo emerge. This build system works fine for BEAST, but it could also be used to maintain other ports of Gtk or Gnome software. Its called BeastBuild and installs everything required to build BEAST, like mingw/msys, gtk+, libguile and portaudio. Finally a user friendly installer is created, so if a user just wants to use BEAST, without building it, he can just simply use the installer. BeastBuild and the BEAST Installer are available here.

The german computer company TWC is offering commercial support for BEAST now. After having supported aRts and BEAST development by paying travel expenses, providing bandwidth, web space and mailing list services, this is the next logical step. So if you want a specific bug to be fixed, or buy a computer with BEAST preinstalled, you can now.
* german version of the BEAST/TWC page
* english version of the BEAST/TWC page