threading

Everybody that has ever tried to use GStreamer in a multithreaded way knows that it is horribly broken. If you have a pipeline running in a thread and want to unref that pipeline in an EOS callback when it’s done, you can’t. You need to put an idle handler into your application’s main loop and unref the thread from there. Ask the Rb and Totem guys about how intuitive that is. So what are the requirements for a threaded media playing framework? I’ve come up with these (in order):
– The obvious part: Doesn’t crash, no unguarded variables, …
– It doesn’t deadlock unless the app does something wrong and that something is well documented – or better: obvious.
– The pipeline has a minimal overhead when running in non-threaded mode.
– Every element in GStreamer runs all of its code in the same thread. Since an element is in essence a vtable that’s filled by the plugin, this means that all of those vtable entries are called by the same thread. (This is done for mainly 2 reasons: It avoids the need to lock inside elements, which eases plugin development and some stuff, like OpenGL, expect to run in the same thread all the time.)

Now here are things people want to do:
– as mentioned above: unref a thread when it hits EOS
– be able to change the running pipeline in a different thread (think gst-editor here)
– be able to change properties of elements in different threads (think volume in a media player)
– be able to listen to signals from a running thread (think error signal or volume-changed in a media player)

marshalling

After discussing this quite a bit the last days (thanks Mike for the input), I came up with the solution of marshalling calls to an element into the thread the element belongs to and dispatch them there. Block the calling thread until the call is dispatched and then continue. Of course this doesn’t work for a simple example that even exists in current code:

• user changes volume
• GUI thread wants to set the “volume” property on the volume element in the gst thread. This is marhsalled into the gst thread, the GUI thread blocks.
• The gst thread changes the volume. This change triggers the notify::volume signal. The GUI thread has connected a signal handler for that to change the volume slider position. So the call is marshalled into the GUI thread. The gst thread blocks until done.
• Noticed something? Right, at this point both threads are blocked waiting for the other to be done.

I quickly came up with the idea to not just block a thread when we wait for a dispatch, but to allow dispatching of other threads’ marshalling requests while blocking. This nicely works in my example code, but unfortunately, it has an important issue.

reentrancy

What’s this and why is it a problem? This is easily answered by Google. Unfortunately reentrant marshalling would make the following GStreamer pseudocode wrong in the general case:

if (gst_pad_link (pad1, pad2))
  g_assert (gst_pads_are_linked (pad1, pad2);

Since gst_pad_link is reentrant, the pads could have been unlinked by other code that happened to be dispatched before returning.
I’m currently investigating if this is a real issue, or if there are constraints that make it impossible for such things to happen. Another thing I’m wondering is how reentrant our code already is (g_signal_emit requires reentrancy for example – or do you know what happens inside signal handlers?) The last thing I’m wondering is how much of a burden this is for GStreamer’s target audience: application writers and plugin coders. Since GStreamer itself doesn’t do much unless you ask it to, most reentrancy issues should be caused by application code. Plugins only handle their own stuff and should never call out anyway. Or am I missing something?

I’m really not sure on the big picture here, since it’s a very hard problem where I don’t have much experience. So if you happen to have run into problems with this that I’m missing, feel free to mail me. And if you’re one of the bonobo/Orbit guys that knows URLs to the reentrancy issues those projects had, so I can read up on it, get in touch with me.

life

I’ve quit my student part-time job. Writing Office macros annoyed me so much I didn’t want to anymore. And since I still can’t convince louie to make Novell hire German students working on GStreamer, I’ll soon go workless. Sucks. But not as much as writing Office macros.