Month: March 2018

GTask and Threaded Workers

GTask is super handy, but it’s important you’re very careful with it when threading is involved. For example, the normal threaded use case might be something like this:

state = g_slice_new0 (State);
state->frob = get_frob_state (self);
state->baz = get_baz_state (self);

task = g_task_new (self, cancellable, callback, user_data);
g_task_set_task_data (task, state, state_free);
g_task_run_in_thread (task, state_worker_func);

The idea here is that you create your state upfront, and pass that state to the worker thread so that you don’t race accessing self-> fields from multiple threads. The “shared nothing” approach, if you will.

However, even this isn’t safe if self has thread usage requirements. For example, if self is a GtkWidget or some other object that is expected to only be used from the main-thread, there is a chance your object could be finalized in a thread.

Furthermore, the task_data you set could also be finalized in the thread. If your task data also holds references to objects which have thread requirements, those too can be unref’d from the thread (thereby cascading through the object graph should you hit this undesirable race).

Such can happen when you call g_task_return_pointer() or any of the other return variants from the worker thread. That call will queue the result to be dispatched to the GMainContext that created the task. If your CPU task-switches to that thread before the worker thread has released it’s reference you risk the chance the thread holds the last reference to the task.

In that situation self and task_data will both be finalized in that worker thread.

Addressing this in Builder

We already have various thread pools in Builder for work items so it would be nice if we could both fix the issue in our usage as well as unify the thread pools. Additionally, there are cases where it would be nice to “chain task results” to avoid doing duplicate work when two subsystems request the same work to be performed.

So now Builder has IdeTask which is very similar in API to GTask but provides some additional guarantees that would be very difficult to introduce back into the GTask implementation (without breaking semantics). We do this by passing the result and the threads last ownership reference to the IdeTask back to the GMainContext at the same time, ensuring the last unref happens in the expected context.

While I was at it, I added a bunch of debugging tools for myself which caught some bugs in my previous usage of GTask. Bugs were filed, GTask has been improved, yadda yadda.

But I anticipate the threading situation to remain in GTask and you should be aware of that if you’re writing new code using GTask.

Device Integration

I’ve been working on some groundwork features to sneak into Builder 3.28 so that we can build upon them for 3.30. In particular, I’ve started to land device abstractions. The goal around this is to make it easier to do cross-architecture development as well as support devices like phones, tablets, and IoT.

For every class and kind of device we want to support, some level of integration work will be required for a great experience. But a lot of it shares some common abstractions. Builder 3.28 will include some of that plumbing.

For example, we can detect various Qemu configurations and provide cross-architecture building for Flatpak. This isn’t using a cross-compiler, but rather a GCC compiled for aarch64 as part of the Flatpak SDK. This is much slower than a proper cross-compiler toolchain, but it does help prove some of our abstractions are working correctly (and that was the goal for this early stage).

Some of the big things we need to address as we head towards 3.30 include running the app on a remote device as well as hooking up gdb. I’d like to see Flatpak gain the support for providing a cross-compiler via an SDK extension too.

I’d like to gain support for simulators in 3.30 as well. My personal effort is going to be based around GNU-based systems. However, if others join in and want to provide support for other sorts of devices, I’d be happy to get the contributions.

Anyway, Builder 3.28 can do some interesting things if you know where to find them. So for 3.30 we’ll polish them up and make them useful to vastly more people.