I have wanted to write about programming with GStreamer and Python for a while. Jono Bacon wrote a nice introduction to GStreamer and Python a long time ago, but I want to share with you some specific tips.
At Collabora we work a lot with GStreamer including helping train developers at our customers to be better at GStreamer development. Being the lowly marketing guy at the company I don’t have the programming chops to teach the hard stuff, but I figured I should be able to put together a very simple article which explains some basics and shows of a little GStreamer development trick I have used to great success in Transmageddon.
Part of what triggered getting this little tutorial done was that I am looking into porting Transmageddon to GTK3 after its next release of Transmageddon. To understand how to write a GTK 3 Python application, using the introspection bindings, I decided a good learning tool for myself would be to try to port the 0.0.1 version of Transmageddon. This version was never released, in fact it was me trying to figure out the very basics of programming with GTK+ and GStreamer in Python.
The application litterally consists of a GTK+ UI with two buttons. One is a ‘transcode’ button which when pressed starts a GStreamer transcoding pipeline. The other is my little secret trick, called ‘Debug’. It will when pressed generate a png of the pipeline being run, or not being run for that matter. It has helped me solve a ton of bugs and issues in Transmageddon since I started the project and hopefully it can be a useful trick for you too.
You can find a tarball here with the code below, the .ui file from Glade and a which.py file (which.py is a python version of the Unix which tool, which I found online).
First let me give you the code of the application, I tried to annotate the code in detail to make it easy to follow, even if you haven’t played with either GTK3 or GStreamer before.
The first thing happening in the file after importing the basis system classes and the which.py tool, is that we set the ‘GST_DEBUG_DUMP_DOT_DIR’ environment variable. When you set this value, GStreamer will be able to at any time dump the pipeline and elements to a ‘dot’ file, which can be turned into a nice looking png by the graphviz command line tool (should be available in most distributions).
Next I import GTK and GStreamer, as you see I don’t yet use the gobject introspection version of GStreamer as that is not fully working yet, but I plan to try to port this simple application to GStreamer 1.0, in which gobject introspection will be the supported way of using Python.
Next is setting up the GStreamer pipeline. You always start by creating a pipeline object, consider this the canvas onto which you will paint the GStreamer streaming pipelines. The next step is to assemble all the GStreamer plugins we want to use in the application. First I create a filesrc object pointing to the file I want to transcode, be sure to point that to a file of your own if trying this application. Next is creating the decodebin2 element. Decodebin2 is one of a set of high level elements in GStreamer, called bins, which contains a wide range of plugins inside. These high level elements are there to make things a lot simpler, and in the case of decodebin2 it will automatically put together the plugins needed to convert your incoming file to raw audio and video (or just demux the file). This means your input doesn’t need to be a mp3 file, like I used, as decodebin2 will reconfigure itself to handle any file you throw at it. After this I create a series of elements to enable me to encode the data into a Ogg Vorbis file. I am doing that to help explain how elements are stringed together, but there is another high level element, encodebin, which I could have used instead. Transmageddon uses encodebin in its git version.
Once all the elements are created you can think of them as boxes spread around on your pipeline canvas, but in order for GStreamer to know how you want to connect them together you need to link them together, as you can see I do with statements like ‘self.filesrc.link(self.decoder)’, which connects the filesrc element I created with the decoder element.
The one special element here is decodebin, which being a dynamic element I need to link it once the pad found signal is fired. Also to link I need to request a compatible pad from the element I am linking with, in this case the audioconverter element.
The last part of the GStreamer setup is setting the pipeline to playing state, which is the state where the pipeline is running. While not a big concern in this very simple application, dealing with state changes in GStreamer is going to be one of the major items you look out for. The GStreamer plugin writers guide contains a chapter discussing the basics of the four states "NULL", "READY", "PAUSED" and "PLAYING". Your pipeline (and all elements) always start at Null state and will go through each of the other stanges to reach Playing. So while we only set state to PLAYING in this simple application, GStreamer will in the background go through READY and PAUSED. The reason the intermediary states matter is because certain things happen at each, so for instance if you want to do some analysis of a file before starting to run your pipeline fully you want to be in PAUSED state as GStreamer will then start pulling the initial data through the pipeline and thus allow you to get information from your elements about the stream or file. One important thing to keep in mind as you develop more advanced applications is that the individual elements can have a different state than the pipeline, but when the state of the pipeline changes it will change the state of the plugins along with it, so you never want your pipeline to be more than one level lower than any of your elements, as that will cause the element to jump down to that state and thus lose the negotiation and information it had assembled.
I am not going to go into a lot of detail about the GUI, it is a very simple GTK user interface built using Glade, and hooked up using the GTK3 gobject introspection bindings. If you got any questions about it post a comment and I be happy to talk about it. What I want to talk about instead is the on_debug_activate function. I wrote this for Transmageddon, but my hope is that it will be useful for anyone writing a Python application with GStreamer (and I guess it shouldn’t be to hard to port to another language). It will allow you to add a menu entry or button in your application that outputs a png file, like the one you see below, which gives you a nice full view of the pipeline used by GStreamer. Especially if you use things like decodebin2 and encodebin, or have a lot of code dynamically adding/removing elements, it can be really useful to see what pipeline ended up being used. And if you have elements that you created, but forgot to link inn, they will appear as orphaned boxes in the file, allowing you to detect such issues. The important thing to remember is that it needs the graphwiz application to be installed on your system and available in the executable path.
Anyway, I hope this has been useful and I plan to post and updated version of this simple application, ported to use encodebin and GStreamer 1.0.