Lazy Loading Using The Main Loop

Sometimes you have a long list of simple stuff you want to feed into a GtkListStore in order to view it with a GtkTreeView. Soon, you realize that simply dogfeeding the data using a loop like this:

  GtkListStore *store;
  GtkWidget *view;

  store = gtk_list_store_new (3,
                              G_TYPE_STRING, /* foo */
                              G_TYPE_STRING, /* bar */
                              G_TYPE_BOOLEAN /* baz */);

  view = gtk_tree_view_new_with_model (GTK_TREE_MODEL (store));
  for (i = 0; i < stuff_len; i++)
    {
      GtkTreeIter iter;

      gtk_list_store_append (store, &iter);
      gtk_list_store_set (store, &iter,
                          COLUMN_FOO, your_object_get_foo (obj),
                          COLUMN_BAR, your_object_get_bar (obj),
                          COLUMN_BAZ, your_object_get_baz (obj),
                          -1);
    }

is really inefficient, as the GtkTreeView has to handle every row insertion. Moving the assignment of the model after the loop makes things better:

  view = gtk_tree_view_new ();
  for (i = 0; i < stuff_len; i++)
    {
      GtkTreeIter iter;

      gtk_list_store_append (store, &iter);
      gtk_list_store_set (store, &iter,
                          COLUMN_FOO, your_object_get_foo (obj),
                          COLUMN_BAR, your_object_get_bar (obj),
                          COLUMN_BAZ, your_object_get_baz (obj),
                          -1);
    }
  gtk_tree_view_set_model (GTK_TREE_VIEW (view),
                           GTK_TREE_MODEL (store));

but the population of the ListStore might take quite some time nevertheless, and block your UI.

At this point, people will begin thinking about using a thread to do the list loading without interfering with the main application loop.

Threads are desiderable only if your application is already threaded – but introducing threads just to load stuff into a ListStore is simply overkill; also, making the threading work with the GTK is always a bit hairy, especially if you also want to update the GUI while loading. Instead, you should use the great main loop implementation that GLib has. All you need to do is set up a little finite state machine for loading stuff, use an idle function that gets called with low priority, and then feed the populated ListStore to the TreeView.

First step: set up the finite state machine. We use four states, two of which are the actual states for loading the items into the model and the model to the view:

enum {
  STATE_STARTED,  /* start state */

  STATE_LOADING,  /* feeding the items to the store */
  STATE_COMPLETE, /* feeding the store to the view */

  STATE_FINISHED  /* finish state - not used */
};

we also set up an opaque container for some data to be used in the idle callbacks:

typedef struct
{
  guint load_state;
  guint load_id;

  GtkListStore *list_store;
  GtkWidget *tree_view;

  gint n_items;
  gint n_loaded;
  GList *items;
} IdleData;

Then, we set up the idle functions; we use the g_idle_add_full() function because we will reach the FINISHED state in the clean up function:

static void
lazy_load_items (GtkWidget    *tree_view,
                 GtkListStore *list_store,
                 GList        *items)
{
  IdleData *data;

  data = g_new (IdleData, 1);

  data->items = items;

  data->n_items = 0;
  data->n_loaded = 0;

  data->list_store = list_store;
  data->tree_view = tree_view;

  data->load_state = STATE_STARTED;
  load_id = g_idle_add_full (G_PRIORITY_HIGH_IDLE,
                             load_items,
                             data,
                             cleanup_load_items);
}

The load_items function will be called each time the main loop is in idle, and will load the content of the items list one item at a time; as soon as the load_items has finished its run on the entire list, the cleanup_load_items will be executed. Here’s the load_items:

static gboolean
load_items (gpointer data)
{
  IdleData *id = data;
  YourObject *obj;
  GtkTreeIter iter;

  /* make sure we're in the right state */
  g_assert (id->state == STATE_STARTED || id->state == STATE_LOADING)

  /* no items */
  if (!id->items)
    {
      id->load_state = STATE_COMPLETE;

      return FALSE;
    }

  /* is this the first run ? */
  if (!id->n_items)
    {
      id->n_items = g_list_length (id->items);
      id->n_loaded = 0;
      id->load_state = STATE_LOADING;
    }

  /* get the item in the list */
  obj = g_list_nth_data (id->items, id->n_loaded);
  g_assert (obj != NULL);

  gtk_list_store_append (id->list_store, &iter);
  gtk_list_store_set (id->list_store, &iter,
                      COLUMN_FOO, your_object_get_foo (obj),
                      COLUMN_BAR, your_object_get_bar (obj),
                      COLUMN_BAZ, your_object_get_baz (obj),
                      -1);
  id->n_loaded += 1;

  /* we can also update the UI, like with a nice progress bar */
  update_progress (id->n_items, id->n_loaded);

  if (id->n_loaded == id->n_items)
    {
      /* we loaded everything, so we can change state
       * and remove the idle callback function; after
       * use, the cleanup_load_items function will
       * be called
       */
      id->load_state = STATE_COMPLETE;
      id->n_loaded = 0;
      id->n_items = 0;
      id->items = NULL;

      return FALSE;
    }
  else
    return TRUE;
}

As you can see, this function appends one item at a time, using the IdleData structure to store the data between the runs.

Finally, here’s the cleanup_load_items function:

static void
cleanup_load_items (gpointer data)
{
  IdleData *id = data;

  g_assert (id->load_state == STATE_COMPLETE);

  /* actually load the model inside the view */
  gtk_tree_view_set_model (GTK_TREE_VIEW (id->tree_view),
                           GTK_TREE_MODEL (id->list_store));

  g_free (id);
}

This is it – now the ListStore will get loaded lazily inside an idle callback that should not mess up with the responsivness of your UI, without having to resort to threads. Another reason why GTK rocks!

Language bindings

• James Henstridge re-implemented the lazy loading in python, using iterators and yield to avoid the state machine.
• Arturo González Ferrer re-implemented it C#