GStreamer and Android

One project we have been working on for some time at Collabora Multimedia is making it easy to use GStreamer with Android. There has been some code available to do this for some time, but it was incomplete and not easy to use. Thanks to a project we did with ST Ericsson we got that code much improved and ST Ericsson kindly released that code afterwards. We then took that code and updated it to run with latest Gingerbread release of Android and also generalized it to make it easy to run with any chipset.

We have also now imported this code into the main GStreamer repositories, so that when you visit the GStreamer Git repository you find the code there along with all the other GStreamer modules. And we have also set up a GStreamer-android mailing list alongside the other GStreamer lists.

Edward Hervey sent out an email today with some the technical details of this project and how it works. But in general the goal here is to offer a transparent integration of GStreamer into Android. We also got a wiki page with full build instructions.

I recommend anyone interested to try it out to join the mailing list and engage with us on moving this code forward. Hopefully we can use it to enable a lot of cool Android devices coming out in the future using advanced GStreamer features such as video editing, Rygel DLNA support, Telepathy video conferencing and collaboration support, DVB support and more.

So a big thank you to Alessandro Decina, Reynaldo Verdejo, Thibault Saunier and Derek Foreman for the great effort they put into this and getting the code ready for release.


#1 randall on 04.13.11 at 00:29

Looks like a good effort. Hacking digital media on android is a big pain right now, and stagefright is still very immature (ie it parses MP4 but not MOV!?!).

From the code it looks like you can only reliably use surfaceflingersink for video output. The SEC OMX bits on the Nexus S don’t actually write pixel data to the output buffers — they write a physical pointer which is then interpreted by the kernel overlay driver (which performs no validation of said pointer) in what can only be described as a grotesque hack.

Stagefright has a special hack mode to get frames out of media for thumbnails to somewhat work around this nonsense. It would be nice if Google would push back on vendors on stuff like this (especially on the Nexus devices!). Physical pointers have no place in userland…

#2 Edward on 04.13.11 at 01:24

The reason we kept surfaceflingersink is because it’s the only reliable way to output video on Android regardless of the deivce, as your pointed out.

We investigated HW acceleration display, but it doesn’t get any better since you then end up in libstagefrighthw specific blobs to handle that. While possible, it would be very limited. If someone wanted to do that, he could trivially write a sink plugin reusing the code in libstagefrighthw with specific caps… but you’d need to add even more vendor-specific hacks to gst-openmax for that to work.

On a libVA/VDPAU capable system, this would be trivial to solve. The culprit isn’t really OMX here, it’s the (sadly) bastardized mix of OMX and stagefright :(

Don’t hesitate to come on the gstreamer-android ml to discuss it