Category: gstreamer

gstreamer wishlist

This is sort of a followup to my earlier post about gstreamer. To make it rock even more, I’d like to see the following enhancements:

  • Integrate videoscale with pitivi’s wonderful smartvideoscale element. The main difference is that the latter keeps the aspect ration untouched by adding black stripes to the appropriate sides when scaling. The logic here is pretty straightforward so any gstreamer hacker should be able to merge these two (smartvideoscale uses videoscale internally and is implemented in Python).
  • Implement an encodebin element to allow applications to use pre-defined profiles for encoding. See below for details

The encodebin proposal

The way I see encodebin would make the element itself pretty simple. To make it useful, it should try to stay profile-agnostic and all specific details should be part of the profiles. Provided that queue can handle multiple streams, the profiles themselves could be as simple as that:

[gstreamer-profile]
Name=PlayStation 3 console
Name=Konsola PlayStation 3
Type=device
Icon=computer-gaming-sony-ps3
Pipeline="queue name=input input. ! ffmpegcolorspace ! x264enc ! queue ! output. input. ! queue ! audioconvert ! faac ! queue ! output. queue name=output"

The element itself would only have to enumerate the profiles and use the one indicated by the profile property. Placing the profiles in a common %_datadir location would allow external packages to add profiles for both common container formats and for specific devices (gaming consoles that only handle certain formats, mobile devices that require lower resolutions etc.).

Ideally the encodebin element should accept video, audio and subtitle streams (as choosing a font size requires you to know the output resolution). This would allow one to build a re-muxing application while focusing on the GUI and not on all the devices to support.

All comments welcome. Disclaimer: I am certainly not a gstreamer hacker so I don’t know poo about gstreamer’s internals. Please be kind.

Muxing with gstreamer

Last week I tried to build a little GNOME application to convert movies and videos to formats supported by our PS3 and PSP consoles. I started by playing with the command line. Sounded simple enough. The only problem I has was subtitle support but the nice guys at #gstreamer answered all my questions and I was ready to go. The first thing I tried was this:

gst-launch filesrc location=test.txt ! \
subparse subtitle-encoding=cp1250 ! subs. \
filesrc location=test.avi ! \
decodebin name=demuxer \
demuxer. ! queue ! ffmpegcolorspace ! \
textoverlay name=subs font-desc="Sans Bold 20" ! \
ffmpegcolorspace ! x264enc ! queue ! muxer. \
demuxer. ! queue ! audioconvert ! faac ! queue ! muxer. \
ffmux_mp4 name=muxer ! \
filesink location=output.mp4

It loos complicated but it’s not. It basically works like this:

  1. filesrc opens test.txt
  2. subparse decodes the file as one of the supported subtitle formats and sets fallbac encoding to cp1250 (unfortunately one of the most popular encodings used for subtitles in Poland)
  3. another filesrc opens test.avi
  4. decodebin detects the container format used, demuxes it and decodes the audio and video streams inside
  5. the video stream gets passed to textoverlay which takes the subtitle stream decoded earlier and overlays it in the Sans Bold 20 font
  6. x264enc takes the resulting video stream from and encodes it into the H264 format used by PS3
  7. faac takes the audio stream and encodes it to the AAC format used by PS3
  8. ffmux_mp4 takes both audio and video from the earlier elements and muxes it into the mp4 format
  9. filesink writes the results back to disk

Unfortunately it turns out the mp4 muxer does not work as intended. Either that or all the sound encoders are broken. The resulting file plays fine in mplayer but any attempt to use it with gstreamer gives you smooth video with broken audio (one sample per half a second). Playing it bac on the PS3 is impossible as it claims the file is simply corrupt.

Then I thought well, I can still use xvid, right? Wrong. xvidenc + lame + avimux = broken file. Again, mplayer plays the video without any sound, gstreamer just hangs on the first frame. This time I’m pretty sure it’s my fault but I have no idea what params to pass to the encoders and the muxer. Even the avimux example from the gstreamer docs results in an unplayable file on my machine.

Simple idea: how about creating a encodebin element to mimick the decodebin behavior? It could work like this:

audiotestsrc ! encoder. \
videotestsrc ! encoder. \
encodebin name=encoder profile=ogg ! \
filesink location=output.ogm
Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Poland
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Poland.