glib

Using GAsyncResult APIs with Python’s asyncio

With a GLib implementation of the Python asyncio event loop, I can easily mix asyncio code with GLib/GTK code in the same thread. The next step is to see whether we can use this to make any APIs more convenient to use. A good candidate is APIs that make use of GAsyncResult.

These APIs generally consist of one function call that initiates the asynchronous job and takes a callback. The callback will be invoked sometime later with a GAsyncResult object, which can be passed to a “finish” function to convert this to the result type relevant to the original call. This sort of API is a good candidate to convert to an asyncio coroutine.

We can do this by writing a ready callback that simply stores the result in a future, and then have our coroutine await that future after initiating the job. For example, the following will asynchronously connect to the session bus:

import asyncio
from gi.repository import GLib, Gio

async def session_bus():
    loop = asyncio.get_running_loop()
    bus_ready = loop.create_future()
    def ready_callback(obj, result):
        try:
            bus = Gio.bus_get_finish(result)
        except GLib.Error as exc:
            loop.call_soon_threadsafe(bus_ready.set_exception, exc)
            return
        loop.call_soon_threadsafe(bus_ready.set_result, bus)

    Gio.bus_get(Gio.BusType.SESSION, None, ready_callback)
    return await bus_ready

We’ve now got an API that is conceptually as simple to use as the synchronous Gio.bus_get_sync call, but won’t block other work the application might be performing.

Most of the code is fairly straight forward: the main wart is the two loop.call_soon_threadsafe calls. While everything is executing in the same thread, my asyncio-glib library does not currently wake the asyncio event loop when called from a GLib callback. The call_soon_threadsafe method does the trick by generating some dummy IO to cause a wake up.

Cancellation

One feature we’ve lost with this wrapper is the ability to cancel the asynchronous job. On the GLib side, this is handled with the GCancellable object. On the asyncio side, tasks are cancelled by injecting an asyncio.CancelledError exception into the coroutine. We can propagate this cancellation to the GLib side fairly seamlessly:

async def session_bus():
    ...
    cancellable = Gio.Cancellable()
    Gio.bus_get(Gio.BusType.SESSION, cancellable, ready_callback)
    try:
        return await bus_ready
    except asyncio.CancelledError:
        cancellable.cancel()
        raise

It’s important to re-raise the CancelledError exception, so that it will propagate up to any calling coroutines and let them perform their own cleanup.

By following this pattern I was able to build enough wrappers to let me connect to the D-Bus daemon and issue asynchronous method calls without needing to chain together large sequences of callbacks. The wrappers were all similar enough that it shouldn’t be too difficult to factor out the common code.

GLib integration for the Python asyncio event loop

As an evening project, I’ve been working on a small library that integrates the GLib main loop with Python’s asyncio. I think I’ve gotten to the point where it might be useful to other people, so have pushed it up here:

https://github.com/jhenstridge/asyncio-glib

This isn’t the only attempt to integrate the two event loops, but the other I found (Gbulb) is unmaintained and seems to reimplement a fair bit of the asyncio (e.g. it has its own transport classes). So I thought I’d see if I could write something smaller and more maintainable, reusing as much code from the standard library as possible.

My first step was writing an implementation of the selectors.BaseSelector interface in terms of the GLib main loop. The select() method just runs a GMainLoop with a custom source that will quit the loop if any of the file descriptors are ready, or the timeout is reached.

For the asyncio event loop, I was able to reuse the standard library asyncio.SelectorEventLoop with my new selector. In action, it looks something like this:

  1. Let the GMainLoop spin until any asyncio events come in.
  2. Return control to the asyncio event loop to process those events.
  3. Repeat

As far as testing goes, the Python standard library comes with a suite of tests parameterised on an event loop implementation. So I’ve just reused that as the bulk of my test suite, and done the same with the selector tests. There are a handful of test failures I still need to diagnose, but for the most part things just work.

Making an asyncio application use this event loop is simple:

import asyncio
import asyncio_glib
asyncio.set_event_loop_policy(asyncio_glib.GLibEventLoopPolicy())

The main limitation of this code is that it relies on asyncio running the GLib main loop. If some other piece of code runs the main loop, asyncio callbacks will not be triggered and will probably lead to busy looping. This isn’t a problem my project (an asyncio server making use of GStreamer), but would be a problem for e.g. a graphical application calling gtk_dialog_run().