So people are starting to pile into the Collabora office here in Cambridge, in preparation for the GStreamer Conference 2010 and the CE Linux Europe 2010. My old friend Jan Schmidt arrived for instance from Australia yesterday and has been putting the finishing touches on his talk today. I strongly recommend seeinghis talk as he is working on something quite nifty. Not to say I don’t recommend the rest of our great list of GStreamer talks.

For those who have arrived early be aware that you can most likely register at the hotel this afternoon already (they where just waiting on some last material to open registration when I was there) and thus save yourself some time tomorrow morning.

So in preparation for the GStreamer Conference 2010 we had some shirts made for the presenters and organizers. Think they ended up looking quite swish. Here is me modelling the silver colour variation :)

Our swish new GStreamer shirts for the GStreamer Conference

Ok, so we decided to close the registration for the GStreamer Conference 2010 today as the event is
fully booked! So for those already registered I am looking forward to seeing you all here in Cambridge next week!
For those who haven’t registered yet, I apologize, but hopefully we do this again next year.

So there is only 18 days left before GStreamer Conference 2010 opens its doors in Cambridge, UK. So be sure to mark the 26th of October in your calendar. Preparations are going smoothly with final items being sorted at the venue and programs getting printed. Registrations are also going well with around 120 people signed up to attend so far. If you plan on coming but having registered yet be sure to do so quickly, because there is likely to be a last minute rush of registrations and you don’t want to risk not being able to register and attend because the venue capacity was reached.

Also be sure to check out the list of presentations which is packed full of presentations by the GStreamer community best and brightest and leading technology companies.

Also check out the agenda for the CE Linux Europe Conference, which has established itself as the leading European conference for discussing the use of linux in embedded devices. Since it happens in the same venue as the GStreamer conference, on the 27th and 28th, you can very easily combine the two.

So as I blogged about before, Collabora Multimedia has been doing a project with NLnet to improve echo cancellation support under Linux and Pulse Audio. There were and still are a lot of challenges to get it right, but we wanted to try to lay the groundwork for a system wide solution, which is why we decided to try to implement it within Pulse Audio.

For those wondering what echo cancellation actually means, it is a way to resolve the issue that if you record sound from your laptop microphone and at the same time output sound from your speakers, you easily end up with the sound looping, creating an irritating echo effect, which makes doing voice calls on a machine painful and sometimes impossible. Echo cancellation systems basically try to analyse the data coming out of the speakers so that it can filter it out and ignore it when it comes back through the microphone.

The final result is that we have created a virtual device pair which adds echo cancellation, these virtual devices are automatically used by your application if it announces itself as a ‘phone’ application to Pulse Audio. Our Empathy messaging and video conferencing client does this for instance. The bulk of our work is now done and for many use cases things should just work as soon as the output of our effort gets merged into Pulse Audio and packaged by the distributions. There are some open questions left, but we hope that by making this work available and trying to work with people within the ALSA community we will be able to resolve the remaining issues over time. Anyway, let me just give you the small report Wim Taymans wrote to summarize the work and what has been done:

Pulse Audio filter infrastructure

Currently the echo-cancel module is built upon the virtual source and sink examples, which is currently considered to be the Pulse Audio filter infrastructure.

We briefly looked into the ideas for a different generic filter infrastructure for Pulse Audio. Lennart Poetterings first attempts to implement such an infrastructure were put on hold because of the large complexity wrt to latency and rewinds. Because echo-cancellation is not purely a filter (it needs input from both sink and source) we decided to build the echo-cancel module as a virtual source and sink instead.

Echo cancelling for PulseAudio

A new module called ‘module-echo-cancel’ was added to Pulse Audio. The module adds a new echo-cancel source and sink to the existing devices. All samples played to the echo-cancel sink get echo-cancelled from the samples captured from the echo-cancel source.

The module is built so that new echo cancellation algorithms can be plugged in very easily. Two echo cancellers are implemented already, one based on Speex dsp and another based on the code from Andre Adrian.

The echo-cancel source and sink currently proxy the default source and sink in Pulse Audio. This can be changed with the pavucontrol application by changing the source and sink of the virtual streams.

Currently the echo cancellation code can deal with sources and sinks that share a common clock, such as those found on the same sound card.

For devices that don’t share a common clock, we currently don’t have accurate enough timings from pulseaudio yet in order to implement dynamic resampling. Most echo cancelling algorithms are extremely sensitive to drift and fail as soon as resampling is slightly inaccurate. This means that if you record sound using a USB microphone on your webcam for instance, there is a good chance the card will drift and the echo cancellation will cease to work.

Enable echo cancellation in Empathy

Any application can connect to the new echo-cancel source and sink to use the provided echo-cancellation.

The new echo-cancel source and sink are tagged with the ‘phone’ media.role so that the module called ‘module-intended-roles’ will automatically link Empathy to the echo-cancel source and sinks.

Echo Cancellation test application

In the case that echo cancellation doesn’t work on your system there is a good chance it is due to bugs in the audio driver, to make it more easy to test for this and provide useful bug reports we wrote a test application that can be run when echo cancellation doesn’t work. As with all such applications there is of course only a limited range of things we can test for so it will not be able to detect all types of problems, but it should be able to expose some of the more common ones.

Given that the echo canceller has to run on more than just the ALSA API (pulse audio has various backends), we implemented a new Pulse Audio module called ‘module-test’ that will run a set of tests on all existing source and sink devices.

The current tests include measuring the accuracy of the timevalues reported by the internal clock. The accuracy of this clock is one of the most crucial parts in Pulse Audio because it is directly used to estimate when samples will be played or captured. Having an accurate timevalue for when a particular sample was played and recorded is essential to implement echo cancellation.

An application called ‘patest’ is provided that runs the tests and outputs the results on the standard output.

The output of the patest application contains information about the various devices that were tested along with a min/max jitter and drift measured on those devices. The jitter and drift are mostly caused by inaccurate results returned from the ALSA drivers. A normal jitter would be around +-15us, a typical drift would be +-100us.

The module is constructed in such a way that more tests can be added later.

Future plans

We would really like to resolve the issue of using multiple sound cards, ie laptop speakers outputting from the internal sound card and microphone recording using a USB sound card in a webcam. In order to do so will will try to reach out to some of the core ALSA developers and see if we can work together to figure out if something can be done. In some cases the hardware might not be good enough to resolve the problem in software, but we hope that in at last the majority of the cases we will be able to compensate for hardware issues in the driver layer.

Another thing we hope to see is that with this infrastructure in place that maybe some commercial entities decide to open source their echo cancellation algorithms, enabling us to plug in new ones that are more robust in terms of handling clock drifting for instance. Another possibility is that academic projects working on echo cancellation can now find it easy to put their algorithms into a production system easily, so we can see some innovation in this field happen in the open source space.