Heading off to Munich in an hour, but I been spending the morning trying to get git master transmageddon into shape. I think I am getting there as things works a lot better for me now. I even added a small feature, namely output to the notification area as seen in the screenshot below. Still very basic, but I hope to improve on it over the next few weeks. Also wondering if putting the Transmageddon nuclear mushroom in the notification area is a good way to keep users calm, or if they see the icon and start wondering if their system is suffering from a meltdown :)
Spent today working on Transmageddon, trying to get it ready for the GStreamer 1.0 release. It is frustrating going as I only get a little time here and there to work on it, and thus my memory of the code is not as good as it needs to be in order to do this efficiently. But I did in the end manage to file a couple of GStreamer 1.0 bug reports and restore the preset functionality to almost where it was before the GTK3/GStreamer 1.0 port.
When starting the port I disabled and removed a lot of code that in retrospect I still needed as I made some wrong assumptions about how encodebin would work. Going over the code now I also realize I have done some things in a quite stupid manner, or maybe the approaches made sense in the original codebase and just now over a series of iterations have become cludgy, anyway tried cleaning it up a bit, but didn’t have the time to really go over the code and make sure it looks nicer.
Of big issues remaining is that the missing codec functionality is still not 100% restored and working nicely, so a little more work needed there. But if you have all codecs already installed then things are shaping up quite nicely. So I think I am getting very close to having a GTK3 and GStreamer 1.0 version of Transmageddon at feature parity with the GTK2 and GStreamer 0.10 version.
Also thanks to Kalev Lember, Transmageddon is now in Fedora (the latest 0.10 based release) and he also made me co-maintainer of the package which I a very happy about. Hopefully this means that when Fedora switches to GStreamer 1.0 I can get the new Transmageddon up and running there quite quickly.
Also Gendre Sébastien plans on working on Transmageddon going forward and have already made some really cool mockups for a revamped UI. A lot of new features are planned, like the number one request I keep getting, support for setting up batches of transcoding jobs instead of having to do them one by one like now. My plan is to release a first version with the current UI and featureset for GTK3 and GStreamer 1.0, and then work with Gendre to implement his ideas. This new UI will also be more GNOME 3 styled, so hopefully it will provide users with something that feels more at home in the GNOME Shell and on Unity.
After a long period of very slow development of Transmageddon where most work has been in debugging issues with GStreamer 0.11 and porting existing features to GTK3 and GStreamer 0.11, I decided it was time to start thinking about where to go forward again. That said, there are still some issues with the new GTK3 and GStreamer 0.11 version, but I needed a break from just tedious porting/re-implementation work and instead start looking at some new ideas and features.
There are two major items I am thinking of. The first is what to do with the usecase of people just wanting to create a manual transcode. The goal of Transmageddon is to make transcoding simple, but the problem is that I am trying to simplify something that is inherently quite complex. The current user interface doesn’t really let you set a lot of things and I know that sometimes that will create files that doesn’t conform to the needs of the user, for instance a lot of settings are just kept from the original output file, like number of audio channels or bitrate used and so on. And many others just rely on the default values of the GStreamer elements used. I don’t want to try to support tweaking all of them through the user interface as there is no way of doing so without making the user interface either cluttered or filled with what will be for most people just gibberish. It is the problem I feel for instance Handbrake is suffering from, that yes it does let you set everything you could ever need, so it can create files that will be useable in some cases where the default user interface of Transmageddon falls short, but it also becomes a hard to navigate jungle for people. My answer to the need for that level of settings is and will continue to be the device profiles option, which I also have some plans for. That said there are a couple of features I do think would be useful to enable from the non-preset interface, like being able to resize the video, choose if you want deinterlacing or not and to allow you to choose stereo output for the audio and finally I want to allow the creation of transcoding queues, so you don’t have to wait until one transcode is finished before configuring another one.
The one special feature I already got in there, the ability to rotate the video, so that if your video for instance was shot with a camera held sideways, also felt rather arbitrary (it was put in there due to a very early bug request) to have exposed in the userinterface.
So what I have been experimenting with is as user interface which has an Advanced menu option in which you can set certain things and also enable certain extra options in the user interface.
So as you see in the screenshot above there are certain options which you just turn on/off and certain options which when enabled will add extra elements to the userinterface as shown below.
If I go down this route the question of course comes what options to put into this Advanced menu and also if I should make the settings persistent accross runs of Transmageddon or if I should let Transmageddon return to its default settings every time you start it.
The Transmageddon editor
Of course the Advanced menu will never become the solution to all transcoding challenges, the device profiles will still be that. But I realized that editing textfiles is not for everyone and it also makes creating advanced profiles a task for even fewer people than there needs to be. I don’t know how many people out there have made their own profiles, but the only one I ever got submitted was from Stefan Kost who made one for the N900. That is probably mostly due to lack of documentation of how to create profiles and where to put them, a problem I have been planning to remedy. That said I realized that maybe creating some kind of editor would be an even better solution, as it could provide a lot of helpful tools for profile creation and thus making it accessible to a lot more people. Which is why I been trying to prototype what such an user interface could look like.
The interface above is what I got so far, and it is just a glorified profile viewer atm, but my hope is to make it fully functional and hopefully useful to people. One feature I really want to do is to allow you to take an existing video, have the profile editor analyze it and create a new profile that will allow you to replicate the settings of that file. So when you get a new phone or device the manufacturer most likely put a sample file on it, you can then load that sample into the editor and the editor will create a profile that matches it. This will enable you to transcode other files to that profile and thus make them work on your device.
That said this will be a major task creating this editor, because I want to to contain a lot of clever logic, so that it doesn’t just end up being a glorified text editor, but I will need to test and experiement to figure out what that logic will be and how to expose it in the user interface.
Anyway, I am hoping to hear back from the community on these two new things and playing with to hear what you think, both from a usability standpoint and of course ideas for how the features could work or should not work, and what you would need to make Transmageddon suit your needs.
So as I talked about in my last blog post we had a great GStreamer hackfest. A lot of things got done and quite a few applications got an initial port over to 0.11. For instance Edward Hervey ended up working on porting the Totem video player, or rather trying to come up with a more optimized design for the Clutter-gst as the basis port was already done.
Jonathan Matthew had flown up all the way from Australia and made great progress in porting Rhythmbox over to the 0.11 API, a port which became hugely more useful after Wim Taymans and Tim-Phillip Muller fixed a bug that caused mp3 playback not to work :).
Peteris Krisjanis made huge strides in porting Jokosher to 0.11. Although like Jason DeRose from Novacut and myself on Transmageddon he did end up spending a lot of time on debugging issues related to gobject-introspection. The challenge for non-C applications like Jokosher, Novacut, Transmageddon and PiTiVi is a combination of the API having changed quite significantly due to the switch to gobject-introspection generated bindings, some general immaturity challenges with the gobject-introspection library and finally missing or wrong annotations in the GStreamer codebase. So once all these issues are sorted things should look a lot brighter for language bindings, but as we discovered there is a lot of heavy lifting to get there. For instance I thought I had Transmageddon running quite smoothly before I suddenly triggered this gobject-introspection bug.
There was a lot of activity around PiTiVi too, with Jean-François Fortin Tam, Thibault Saunier and Antigoni Papantoni working hard on porting PiTiVi to 0.11 and the GStreamer Editing Services library. And knowing Jean-François Fortin I am sure there will soon be a blog with a lot more details about that :).
Thomas Vander Stichele, who also wrote a nice blog entry about the event, was working with Andoni Morales Alastruey, both from Flumotion, on porting Flumotion to 0.11, but due to some of the plugins needed not having been ported yet most of their effort ended up being on porting the needed plugins in GStreamer and not so much application porting, but for those of you using plugins such as multifdsink, this effort will be of great value and Andoni also got started on porting some of the non-linux plugins, like the directsoundsink for Windows.
Josep Torra from Fluendo ended up working with Edward Hervey on hammering out the design for the clutter-gst sink at the conference, but he also found some time to do a port of his nice little tuner tool as you can see from the screenshot below.
In addition to the application work, Wim Taymans, Tim-Phillip Müller and Sebastian Dröge from Collabora did a lot of core GStreamer clean ups and improvements in addition to providing a lot of assistance, bugfixing and advice for the people doing application porting. All critical items are now sorted in 0.11 although there are some nice to have’s still on the radar, and Wim plans on putting out some new releases next week, to kickstart the countdown to the first 1.0 release.
As for my own little pet project Transmageddon, it is quite far along now, with both manually configured re-encodes and profile re-encodes working. Still debugging remuxing though and I am also waiting for the deinterlacer to get ported to re-enable deinterlacing in the new version. For a screenshot take a look at the one I posted in my previous blogpost.
Things have been going really well here at the GStreamer Hackfest in Malaga. Thanks to the help of Ara and Yaiza from Nido Malaga, we have a great venue in downtown Malaga and they have also helped us greatly with sorting out food.
We have a great group of people here and are making great progress, and by tomorrow I hope we will have screenshots of quite a few applications running with GStreamer 0.11, for instance both Rhythmbox and Jokosher for instance is already screen shootable, if not fully functional :)
Also making good progress on Transmageddon, even if the move to GObject Introspection bindings are making things a bit more complicated. Screenshot below of the progress so far.
Update: Yaiza took these photos from the hackfest
Tomorrow I will be heading off to attend the GStreamer Application Porting Hackfest in Malaga, Spain. I think we have managed to pull together an absolutely incredible group of people for this event and I have great hopes that by next weekend we will have squashed a ton of bugs in GStreamer 0.11/1.0 and also have initial ports of a long range of important applications and bindings. This is the first time in GStreamer history that we are trying to hold a hackfest focused on application developers, but hopefully it will be the first of many and that they can become a good way for the core GStreamer community and the application development community to interact and collaborate more closely.
Also want to say a special thanks to the community members attending the event on their own and also to the companies sending their employees to the hackfest; Collabora, Fluendo, Flumotion and Igalia and finally a special thanks to the GNOME Foundation for sponsoring some of the attendees.
Hopefully I will be able to post some screenshots of a fully functional GStreamer 1.0 Transmageddon next weekend :)
After updating GStreamer and doing a couple of small fixes I managed to make Transmageddon work with the GTK3 and the 0.11 branch of GStreamer. Obligatory screenshot below. As you might guess from looking at the screenshot there are still some issues that needs solving, but
I am happy that I managed to get this far.
Hopefully it is a sign that the upcoming GStreamer hackfest in Malaga will be a great successful everyone who is participating.
I hope the remainder of the porting effort will be relatively simple as I would love to get back to working on real features instead of just updating old functionality to use a new backend to do the same. Having had a need for Transmageddon for a couple of work related tasks recently a couple of items, like batch job programming has moved up my priorities list.
So after a very long 1 year development cycle I finally managed to push a new release of Transmageddon. The main reason for it haven taken this long was because I decided to port to the brand new discoverer and encodebin elements, to greatly simplify the code and improve functionality. The thing was that Edward Hervey when he wrote those elements took the conscious decision to make them assume that all needed GStreamer plugins worked as they should to be perfect GStreamer citizens. As you might imagine, since many of the plugins had never been tested for all such behaviour a lot of things did not work after the port. But over the last Months I have filed bug reports and most of them are now fixed. And with todays new python-gstreamer release (0.10.22) the fix for the binding bug for encodebin is fixed and thus I decided it was time to put out a new release.
I am quite happy with my new feature list:
- Port to new plugins-base discoverer and encodebin
- Replace radiobutton lists with a combobox instead
- add support for audio only transcoding
- add support for outputting audio only from video+audio files
- Add deinterlacing
- Support container free audio formats such as FLAC, mp3 and AAC
- Add HTML5 and Nokia 900 profile
- add support for video only transcoding
- add support for mpeg1 video and mpeg2 audio
The new user interface should solve the problems people with small screens used to have with Transmageddon, like many netbook users. Transmageddon now also automatically deinterlaces deinterlaced video clips, I want to improve this feature in a later release, but making it optional to deinterlace and also use Robert Swains new elements that can help detect interlaced files that have been encoded as progressive files (as shown in his presentation in Prague).
Another major feature of this release is that audio only files are now officially supported, and you can also use Transmageddon to easily output just the audio from a audio+video clip.
I also added a HTML5 webm profile, so output to that should be easier than ever. In fact I used that profile when I transcoded 100GB of video that we had recorded at an internal session at Collabora.
My next goal is to port to GStreamer 1.0 and GTK3. That said if there turns out to be some brown paper bag issues in this release I will try to fix them and make a new release, but my guess is that most bugs people might encounter will be because there are issues that are only fixed in GStreamer git yet, so until they are all released not everything works 100% and until those releases are out there might be some small regressions from the previous release.
Anyway, I hope you head over to the Transmageddon website and grab the latest release for a test run. I will try to follow-up on bug reports, but might be a bit slow the next week as I am flying down to Lahore to celebrate my sister-in-laws wedding and also see my beautiful wife again after almost a Month apart.
I have wanted to write about programming with GStreamer and Python for a while. Jono Bacon wrote a nice introduction to GStreamer and Python a long time ago, but I want to share with you some specific tips.
At Collabora we work a lot with GStreamer including helping train developers at our customers to be better at GStreamer development. Being the lowly marketing guy at the company I don’t have the programming chops to teach the hard stuff, but I figured I should be able to put together a very simple article which explains some basics and shows of a little GStreamer development trick I have used to great success in Transmageddon.
Part of what triggered getting this little tutorial done was that I am looking into porting Transmageddon to GTK3 after its next release of Transmageddon. To understand how to write a GTK 3 Python application, using the introspection bindings, I decided a good learning tool for myself would be to try to port the 0.0.1 version of Transmageddon. This version was never released, in fact it was me trying to figure out the very basics of programming with GTK+ and GStreamer in Python.
The application litterally consists of a GTK+ UI with two buttons. One is a ‘transcode’ button which when pressed starts a GStreamer transcoding pipeline. The other is my little secret trick, called ‘Debug’. It will when pressed generate a png of the pipeline being run, or not being run for that matter. It has helped me solve a ton of bugs and issues in Transmageddon since I started the project and hopefully it can be a useful trick for you too.
You can find a tarball here with the code below, the .ui file from Glade and a which.py file (which.py is a python version of the Unix which tool, which I found online).
First let me give you the code of the application, I tried to annotate the code in detail to make it easy to follow, even if you haven’t played with either GTK3 or GStreamer before.
The first thing happening in the file after importing the basis system classes and the which.py tool, is that we set the ‘GST_DEBUG_DUMP_DOT_DIR’ environment variable. When you set this value, GStreamer will be able to at any time dump the pipeline and elements to a ‘dot’ file, which can be turned into a nice looking png by the graphviz command line tool (should be available in most distributions).
Next I import GTK and GStreamer, as you see I don’t yet use the gobject introspection version of GStreamer as that is not fully working yet, but I plan to try to port this simple application to GStreamer 1.0, in which gobject introspection will be the supported way of using Python.
Next is setting up the GStreamer pipeline. You always start by creating a pipeline object, consider this the canvas onto which you will paint the GStreamer streaming pipelines. The next step is to assemble all the GStreamer plugins we want to use in the application. First I create a filesrc object pointing to the file I want to transcode, be sure to point that to a file of your own if trying this application. Next is creating the decodebin2 element. Decodebin2 is one of a set of high level elements in GStreamer, called bins, which contains a wide range of plugins inside. These high level elements are there to make things a lot simpler, and in the case of decodebin2 it will automatically put together the plugins needed to convert your incoming file to raw audio and video (or just demux the file). This means your input doesn’t need to be a mp3 file, like I used, as decodebin2 will reconfigure itself to handle any file you throw at it. After this I create a series of elements to enable me to encode the data into a Ogg Vorbis file. I am doing that to help explain how elements are stringed together, but there is another high level element, encodebin, which I could have used instead. Transmageddon uses encodebin in its git version.
Once all the elements are created you can think of them as boxes spread around on your pipeline canvas, but in order for GStreamer to know how you want to connect them together you need to link them together, as you can see I do with statements like ‘self.filesrc.link(self.decoder)’, which connects the filesrc element I created with the decoder element.
The one special element here is decodebin, which being a dynamic element I need to link it once the pad found signal is fired. Also to link I need to request a compatible pad from the element I am linking with, in this case the audioconverter element.
The last part of the GStreamer setup is setting the pipeline to playing state, which is the state where the pipeline is running. While not a big concern in this very simple application, dealing with state changes in GStreamer is going to be one of the major items you look out for. The GStreamer plugin writers guide contains a chapter discussing the basics of the four states "NULL", "READY", "PAUSED" and "PLAYING". Your pipeline (and all elements) always start at Null state and will go through each of the other stanges to reach Playing. So while we only set state to PLAYING in this simple application, GStreamer will in the background go through READY and PAUSED. The reason the intermediary states matter is because certain things happen at each, so for instance if you want to do some analysis of a file before starting to run your pipeline fully you want to be in PAUSED state as GStreamer will then start pulling the initial data through the pipeline and thus allow you to get information from your elements about the stream or file. One important thing to keep in mind as you develop more advanced applications is that the individual elements can have a different state than the pipeline, but when the state of the pipeline changes it will change the state of the plugins along with it, so you never want your pipeline to be more than one level lower than any of your elements, as that will cause the element to jump down to that state and thus lose the negotiation and information it had assembled.
I am not going to go into a lot of detail about the GUI, it is a very simple GTK user interface built using Glade, and hooked up using the GTK3 gobject introspection bindings. If you got any questions about it post a comment and I be happy to talk about it. What I want to talk about instead is the on_debug_activate function. I wrote this for Transmageddon, but my hope is that it will be useful for anyone writing a Python application with GStreamer (and I guess it shouldn’t be to hard to port to another language). It will allow you to add a menu entry or button in your application that outputs a png file, like the one you see below, which gives you a nice full view of the pipeline used by GStreamer. Especially if you use things like decodebin2 and encodebin, or have a lot of code dynamically adding/removing elements, it can be really useful to see what pipeline ended up being used. And if you have elements that you created, but forgot to link inn, they will appear as orphaned boxes in the file, allowing you to detect such issues. The important thing to remember is that it needs the graphwiz application to be installed on your system and available in the executable path.
Anyway, I hope this has been useful and I plan to post and updated version of this simple application, ported to use encodebin and GStreamer 1.0.
Spent some time this weekend hacking on Transmageddon. Fixed various small bugs and UI issues that I had punted up until now for the UI. For instance with latest git when you create a pure audio file it doesn’t automatically get the suffix .mp3, which is nice in the cases when you are not creating a mp3 file :) And if you put aac into a quicktime container the file gets named .m4a instead of .mov.
Also started looking into the issue of how to handle multiple audio streams in the file being transcoded. Currently all streams gets transcoded to the same chosen format if the container format support its, if the container only supports 1 audio stream you get one by random. This is not ideal :)
Ended up filing this bug with a request for how we can improve the GStreamer API to make handling such things easier for application developers. Discoverer, uridecodebin and encodebin makes a lot of things a lot easier, but for handling files with multiple streams of the same type I think we still need some improvements.