Limiters in libdex

Libdex now has DexLimiter, a small utility for bounding how much asynchronous work runs at once.

This is useful when a workload can produce more parallelism than the underlying machine, subsystem, or service should actually handle. Common examples include indexing files, downloading URLs, generating thumbnails, parsing documents, or querying a service with a fixed concurrency budget.

The usual API is dex_limiter_run(). It acquires a permit, starts a fiber, and releases the permit when that fiber finishes.

static DexFuture *
load_one_file (gpointer user_data)
{
  GFile *file = user_data;

  return dex_file_load_contents_bytes (file);
}

DexLimiter *limiter = dex_limiter_new (8);
DexFuture *future = dex_limiter_run (limiter,
                                     NULL,
                                     0,
                                     load_one_file,
                                     g_object_ref (file),
                                     g_object_unref);

In this example, no more than eight file loads will run at the same time, regardless of how many files are queued. The returned DexFuture resolves or rejects with the result of the spawned fiber.

One important detail is that dropping the returned future does not cancel a fiber that has already started. Once work has acquired a permit, it is allowed to complete so that the limiter can release the permit cleanly.

For more specialized cases, DexLimiter also supports manual acquire and release:

g_autoptr(GError) error = NULL;

if (dex_await (dex_limiter_acquire (limiter), &error))
  {
    do_limited_work ();
    dex_limiter_release (limiter);
  }

This is useful when the limited section is not naturally represented by a single fiber. However, callers must release exactly once for every successful acquire. In most cases, dex_limiter_run() is preferable because it handles release on both success and failure paths.

The limit should describe the constrained resource, not the number of items being processed. Remote APIs and databases may need a small limit. CPU-heavy work should usually be near the amount of useful worker parallelism. Local I/O can often tolerate a larger value, depending on the storage system. Separate resources should usually have separate limiters, so one workload does not consume another workload’s concurrency budget.

Finally, dex_limiter_close() can be used during shutdown. Once closed, pending and future acquisitions reject with DEX_ERROR_SEMAPHORE_CLOSED. Work that already holds a permit may continue, but releasing after close does not make new permits available.

The goal is to make bounded parallelism simple: queue as much asynchronous work as you need, but only run as much of it as the system should handle.

A Small Update from France

For about the past month, I have no longer been with Red Hat.

That is a strange sentence to write after so many years, but life has a way of changing the scenery whether or not one has finished packing. My family and I have made it safely to France, and we are quite happy here. The light is different, the pace is different, and there is a great deal to learn. For now, that is exactly where our attention needs to be.

I also think there is a broader lesson here for people whose safety, immigration status, or family stability may depend on employer flexibility. Do not assume that long tenure, remote work history, or prior verbal guidance will be enough. My own experience left me with the uncomfortable conclusion that these processes can become very narrow exactly when the human stakes are highest. Get things in writing, understand the policy surface area, and protect your family first.

This also means that some of the things I wrote about in my earlier mid-life update remain unresolved. I am currently in France on a visitor visa, which does not authorize work here. Our focus is on integration, language learning, and getting ourselves properly settled for the long term. That takes time, patience, paperwork, and a certain tolerance for being a beginner again.

As a result, I will not be taking on ongoing software maintenance responsibilities for the foreseeable future. I may still scratch the occasional itch where it directly improves my own computing life, but I am not currently able to provide the kind of broad, sustained stewardship that many projects deserve.

That is not an easy thing to say. Open source is entering an especially difficult period. We are now seeing AI systems used not only to generate code, but also to probe, disrupt, and attack critical software infrastructure. I suspect this will have a negative effect on the maintenance burden for a lot of projects, particularly the foundational pieces that distributions, companies, and users all rely on without always seeing the people behind them.

But there is a limit to what can reasonably be carried as unpaid labor, especially when the primary financial beneficiaries are downstream organizations with considerably more resources than the individual maintainers doing the work. At the moment, I need to prioritize my family, our stability, and the next chapter of our life here.

That said, I am still reachable for appropriate professional inquiries, advisory conversations, or consulting opportunities where the structure and location make sense. The best address for that now is christian at sourceandstack dot com.

For now, we are safe, settling in, and doing our best to build something durable out of a rather odd moment in the world. There are worse places to begin again than France.

A calm Mediterranean shoreline at sunset, with gentle waves rolling onto a pebbled beach beneath a wide blue sky streaked with soft clouds and pastel pink light on the horizon.