Category: GNOME

gstreamer wishlist

This is sort of a followup to my earlier post about gstreamer. To make it rock even more, I’d like to see the following enhancements:

  • Integrate videoscale with pitivi’s wonderful smartvideoscale element. The main difference is that the latter keeps the aspect ration untouched by adding black stripes to the appropriate sides when scaling. The logic here is pretty straightforward so any gstreamer hacker should be able to merge these two (smartvideoscale uses videoscale internally and is implemented in Python).
  • Implement an encodebin element to allow applications to use pre-defined profiles for encoding. See below for details

The encodebin proposal

The way I see encodebin would make the element itself pretty simple. To make it useful, it should try to stay profile-agnostic and all specific details should be part of the profiles. Provided that queue can handle multiple streams, the profiles themselves could be as simple as that:

[gstreamer-profile]
Name=PlayStation 3 console
Name=Konsola PlayStation 3
Type=device
Icon=computer-gaming-sony-ps3
Pipeline="queue name=input input. ! ffmpegcolorspace ! x264enc ! queue ! output. input. ! queue ! audioconvert ! faac ! queue ! output. queue name=output"

The element itself would only have to enumerate the profiles and use the one indicated by the profile property. Placing the profiles in a common %_datadir location would allow external packages to add profiles for both common container formats and for specific devices (gaming consoles that only handle certain formats, mobile devices that require lower resolutions etc.).

Ideally the encodebin element should accept video, audio and subtitle streams (as choosing a font size requires you to know the output resolution). This would allow one to build a re-muxing application while focusing on the GUI and not on all the devices to support.

All comments welcome. Disclaimer: I am certainly not a gstreamer hacker so I don’t know poo about gstreamer’s internals. Please be kind.

Muxing with gstreamer

Last week I tried to build a little GNOME application to convert movies and videos to formats supported by our PS3 and PSP consoles. I started by playing with the command line. Sounded simple enough. The only problem I has was subtitle support but the nice guys at #gstreamer answered all my questions and I was ready to go. The first thing I tried was this:

gst-launch filesrc location=test.txt ! \
subparse subtitle-encoding=cp1250 ! subs. \
filesrc location=test.avi ! \
decodebin name=demuxer \
demuxer. ! queue ! ffmpegcolorspace ! \
textoverlay name=subs font-desc="Sans Bold 20" ! \
ffmpegcolorspace ! x264enc ! queue ! muxer. \
demuxer. ! queue ! audioconvert ! faac ! queue ! muxer. \
ffmux_mp4 name=muxer ! \
filesink location=output.mp4

It loos complicated but it’s not. It basically works like this:

  1. filesrc opens test.txt
  2. subparse decodes the file as one of the supported subtitle formats and sets fallbac encoding to cp1250 (unfortunately one of the most popular encodings used for subtitles in Poland)
  3. another filesrc opens test.avi
  4. decodebin detects the container format used, demuxes it and decodes the audio and video streams inside
  5. the video stream gets passed to textoverlay which takes the subtitle stream decoded earlier and overlays it in the Sans Bold 20 font
  6. x264enc takes the resulting video stream from and encodes it into the H264 format used by PS3
  7. faac takes the audio stream and encodes it to the AAC format used by PS3
  8. ffmux_mp4 takes both audio and video from the earlier elements and muxes it into the mp4 format
  9. filesink writes the results back to disk

Unfortunately it turns out the mp4 muxer does not work as intended. Either that or all the sound encoders are broken. The resulting file plays fine in mplayer but any attempt to use it with gstreamer gives you smooth video with broken audio (one sample per half a second). Playing it bac on the PS3 is impossible as it claims the file is simply corrupt.

Then I thought well, I can still use xvid, right? Wrong. xvidenc + lame + avimux = broken file. Again, mplayer plays the video without any sound, gstreamer just hangs on the first frame. This time I’m pretty sure it’s my fault but I have no idea what params to pass to the encoders and the muxer. Even the avimux example from the gstreamer docs results in an unplayable file on my machine.

Simple idea: how about creating a encodebin element to mimick the decodebin behavior? It could work like this:

audiotestsrc ! encoder. \
videotestsrc ! encoder. \
encodebin name=encoder profile=ogg ! \
filesink location=output.ogm

PackageKit GUI ideas

I think we all agree that while PackageKit is the way to go, its interface is less than pretty. Or useful for that matter. That’s why I tried to list the biggest annoyances:

Updates window problems

  • Has a useless step 1 that tells you the total number of updates available. In theory it groups updates by severity. In reality such information is only available in two distros or so.
  • Same step 1: I think we are trying to dumb the interface down too much so we ended up adding more buttons than necessary. Even Apple does not hide the list of updates from the user.
  • Every time you click anything the windows resizes itself. Usually moving along to try and stay within the screen boundary.
  • When you try to update anything, another small window appears in a different part of the screen. The main window is gone so what is exactly the point of having multiple windows here?
  • When the update is done the smallish window disappears and yet another small window appears. This time offering you to go back. I think we all agree that if the update is successful a notification would be enough. And the go back button is only there because the UI is broken and hides the interface while doing the work.

Add/Remove Software problems

  • Every time you click a package, half of the window gets covered with package details and buttons. These are all stacked vertically so they take a whole effing lot of useful place. Not to mention the package you clicked suddenly scrolls out of view.
  • Every time you install or remove anything the window blanks for a brief moment then it gets repopulated with packages, putting you back at the top of the results list. Doesn’t sound bad unless you try to install a bunch of gstreamer packages (and there are 195 matching results in PLD).

Proposal

  • Try to keep it each interface in one window.
  • Really try to keep it all in one window.
  • Don’t show technical details each time I click.

gpk-update

This is the update mockup I came up with some time ago. Think of it as the main part of the window that allows you to select packages to update. The point is once you click update instead of hiding the window, we can hide unselected packages and show live progress in the very same window. Without resizing it. And then hide it if the update is successful.

If it’s not? Well, see that Failed button? The label is wrong but it should be able to show you the details. We all agreed that update is really a background process so popping a big cryptic Update failed window in the middle of my solitaire session is not a thing to do. Just pop a critical notification and I’ll get back to the window to check the details once I’m done with my current task.

Bonus: It also lets you see the whole queue while updating which is quite useful.

Oh and by the way. One failing update should not immediately break the transaction. This forces me to update packages one by one and is actually slower than typing upgrade * in poldek’s interactive shell (not sure if single failed package breaks transaction applies to other backends as well).

gpk-add-remove

Here’s a mockup for Add/Remove Software. The main difference is that instead of annoying users with a panel that pops up each time you select anything, once you click a package, it shows more info inline. Here more info does not include files and dependencies. How often do users choose their software basing on a number of packaged files? There’s an Information button for that. The one below Install (it actually says Add on the picture above but that’s because there is no stock Install button and I’m lazy).

Bonus: clicking install on any package should just display a small progress bar beside that package. See previous mockup for details. And when it’s done, just update the icons of affected packages instead of refreshing the whole window. This way I can scroll and click Install on any number of packages without having to go through the whole install-wait-refresh-wait-scroll-to-previous-position cycle.

Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Poland
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Poland.