Previously on Futures, Work-Stealing, and such.
One thing I admired during the early days of Python Twisted was how it took advantage of the language to write asynchronous code which read as synchronous. Pretty much all the modern languages now have some form of this in stackless or stackfull form.
C has been able to do it for ages but it generally doesn’t play well outside of constrained applications. The primary things you have to worry about are functional tooling like debuggers (thread apply all bt wont show you all your fibers) and features like thread-local-storage.
If you’re careful about when you suspend your fiber, the later isn’t so much of an issue. Where it can become a serious issue is if you do something like call an intermediate function which uses callbacks and the callback suspends. In this case, the intermediate function (out of your control) might have some TLS state cached on the stack, which of course could be modified before resume is called.
But either way, dex (wip title newlib) has support for fibers now which can be spawned using dex_scheduler_spawn(). Fiber stacks are given a guard page so that stack overflows are still guarded. It tries to do a bit of madvise() when it makes sense to.
That means I can finally write async code for GNOME Builder similar to:
/* Suspend fiber until read_bytes future completes */ g_autoptr(GBytes) bytes = dex_await_boxed (dex_input_stream_read_bytes (input, count, 0), &error); /* Use bytes from above after resuming and suspend until write completes */ gssize n_written = dex_await_int64 (dex_output_stream_write_bytes (output, bytes, 0), &error);
There is also API to do this using plain file-descriptors which is backed by io_uring on Linux. However, that is still private API because I’m not sure how I want to expose it (if at all).
You can await any DexFuture sub-type which allows for complex composition. Fibers themselves are a DexFuture, which means you can await on another fiber completing as part of the composition.
Fibers can run on any scheduler of your choosing. They integrate into the GMainContext on that scheduler which may defer to a sub-scheduler. For example, the thread pool scheduler will pin the fiber to a sub-scheduler on a single worker thread.
I’ve tested it on Linux, macOS, FreeBSD 13 and Illumos (OpenIndiana) with success so far. The fiber suspend/resume scheduling is done with makecontext()/swapcontext() APIs but uses assembly form Russ Cox’s libtask for the places where that would otherwise incur an unnecessary syscall.
While porting to FreeBSD 13 I noticed that eventfd() is supported there now. Cool! Same goes for Illumos which apparently also supports epoll (we use io_uring, but still nice to see Linux APIs proliferate to ease porting efforts).