Writing Tests is Humbling

I have spent some time recently writing import/export tests cases for Gnumeric. It is what you do when you see a mistake that it should not have been possible to make, in this case a hang when writing certain strings to the obsolete xls/biff7 format.

Writing these tests has been a very humbling experience. Highly recommended.

A lot of the code being subjected to tests is quite old: 10-15 years old. You would think that by now it would have had any obvious bugs beating out of it. No such luck. Not only were there ancient bugs — such as the direction of diagonal borders being flipped on load+save — but there were also bugs accidentally introduced when fixing other things.

Bugs happen, even where you think you are being careful. And that is not a problem. The problem arises when the bugs are not caught and make it into releases. This is where a brutal test suite is needed. Our test suite clearly was not evil enough for import and export of various formats, so I have been adding something I call round-trip tests for ods, xls/biff7, xls/biff8, and xlsx.

A round-trip test is a test that when we convert to a given format and back to our own Gnumeric format, nothing changes. Or, if something does change, then only parts we understand change. If the format is deficient, there is not much we can do. For example: ods cannot store patterned background or the sheet size; xlsx cannot store solver parameters; xls/biff7 cannot store arbitrary unicode; and xls/biff8 has a fixed sheet size of 64k-by-256. Note, that by itself a round-trip test does not guarantee that what we produce is correct. We could, hypothetically, have swapped division and multiplication are still gotten a perfect round-trip. To test that the generated files are correct one has to load the resulting sheet in Excel or LibreOffice (which, despite claims to the contrary, are what really defines xls/xlsx and ods formats). Unfortunately, I do not know how to script that so it is not automatic.

As a result of all the new tests, the recently released Gnumeric 1.12.12 should interact better with other spreadsheets.

Note: some of these new tests were probably a decade overdue. My excuse is that The Gnumeric Team is fairly small. I do hope that LO/OO already have an evil test suite, but I am not optimistic. I ran a few of my test sheets through LO and saw things like truncated strings.

Gcc vs. Clang for Error Messages

So the gcc vs. clang debate flamed up again. I thought I would deliver my few cents too.

It is claimed from time to time that clang has more helpful error messages. It is, in my opinion, a clain that is just plain wrong. They both stink. Let’s look at a few samples:

static int foo (int a, int b) { return a + b; }
int bar (int a) { return foo (a (4 + 1) * 2); }

gcc says (excerpts):

e1.c:2:33: error: called object ‘a’ is not a function or function pointer
e1.c:2:1: error: too few arguments to function ‘foo’

clang says (excerpts):

e1.c:2:33: error: called object type 'int' is not a function or function pointer

The best thing you can say about the error messages here is that they at least point you to the right location. gcc is a tad better by virtue of printing the second error message which at least hints of the real problem, but neither compiler tell us what the problem is: “missing comma”. It looks like clang is suppressing the second and further errors on a line. Note, however, that in this case it has suppressed the more informative error.

Moving on with a missing opening parenthesis:

static int foo (int a, int b) { return a + b; }
int bar (int a) { return foo a); }

From gcc we get the wisdom

e2.c:2:19: warning: return makes integer from pointer without a cast [enabled by default]
e2.c:2:30: error: expected ‘;’ before ‘a’
e2.c:2:31: error: expected statement before ‘)’ token

while clang produces

e2.c:2:26: warning: incompatible pointer to integer conversion returning
'int (int, int)' from a function with result type 'int' [-Wint-conversion]
e2.c:2:29: error: expected ';' after return statement

I don’t see that one set of utter nonsense is better than the other and spending time colour coding the output shows a dubious set of priorities. Clang would do well to add hyphens to “pointer to integer”.

How about this?

#include <stdio.h>
#define EMIT(c) fprintf(stderr,"%c",(c))

Nothing from gcc, nothing from clang, nothing from sparse. Yet it’s a clear violation of C99’s paragraph 7.26.3.

C++ doesn’t fare any better:

#include <vector>
std::vector<int,int> foo; // should have been map

gcc delivers 67 lines of nonsense starting with

/usr/include/c++/4.8/ext/alloc_traits.h:199:53: error: ‘int’ is not a class, str
uct, or union type
typedef typename _Alloc::pointer pointer;

whereas clang emits 87 lines of garbage starting with

/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/alloc_traits.h
:199:22: error: type 'int' cannot be used prior to '::' because it has no member
typedef typename _Alloc::pointer pointer;

Again, the error messages are startingly useless in both cases. A sane error message would start with “type int is not valid for the second template argument to std::vector”.

The quality of error messages has been the subject of jokes for decades. Insofar clang is new code, it would appear that they have squandered any opportunity for making real improvements opting instead for putting lipstick on a pig.

Spreadsheets and the Command Line

Spreadsheets are not the most obvious type of document to manipulate from the command line. They are essentially a visual tool meant for interactively exploring data. Experience has shown, however, that there are certain spreadsheet tasks for which the command line is very useful and Gnumeric supplies a number of command line tools for this.

  • ssconvert converts spreadsheets from one format to another, for example from xls to ods. That sounds fairly simple — load one, save the other — and it pretty much is although there are things such as merging several files or extracting parts of files that add a little complication. Since Gnumeric can save as pdf, this tool also allows command-line printing of spreadsheets.
  • ssgrep is like grep is for text files. And it has about the same set of options.
  • ssindex is used by things like tracker and beagle to find of pieces of text in spreadsheet files.
  • ssdiff is a new tool in the upcoming release. It compares two spreadsheets and outputs a list of differences between them. There are three output modes so far: (1) a text format, (2) an xml format, and (3) a mode that outputs a copy of one of the input files with differing cells marked in neon yellow.

None of these programs are big: 300-1100 lines of C code, ssdiff being the largest only because of its three output modes.

GMail Cross-Mailbox Information Leakage

GMail likes to present ads that are relevant to you by looking at the information in your mailbox. Fine. That is well known and just using the information you have chosen to store at Google.

However, when you use GMail to reply to a message sent by another GMail user, you will be shown ads based on the other user’s recent email activity.

That is news to me and outright scary.

I.e., do not use GMail to communicate to your doctor about an embarrassing disease because next time you write to your mother-in-law she will know. (Even if she always suspected.)

Norway

Unreal.

I have been to events like the one on Utøya. Several such events. Not the exact one, but change political denomination, move a country south, and go back 15 years. Details, in other words, that the crackpot-du-jour randomly decided. For all practical purposes I could have been there.

Let the wheels of the justice system grind. Followed, one hopes, by a long, very lonely stay in a gray cell with easy access to newspapers documenting him fading into a position next to Quisling in public memory.

Sorting Icons Theme Mess

In my long-running series on why themes are evil, I bring you the newest installment.

Consider the gtk stock icon GTK_STOCK_SORT_ASCENDING which is supposed to represent sorting elements to make them increasing according to some order, typically numerically or alphabetically. The icon for such an action is supposed to somehow convey what happens when it is pressed, all in, say, 24×24 pixels.

Take a look at different themes and how they implement the icon:

eog `find /usr/share/icons/ -print | grep sort-ascending`

This command will show you the icon images with some duplication due to multiple sizes.

Observations:

  • Some show an up-arrow, others show a down-arrow. Yet others show a diagonal arrow which isn’t as bad as it sounds because such arrows are annotated.
  • Some arrows have no annotations, some are annotated by “1..9”, and yet others are annotated to “a..z”.

Officially this is a mess[tm]. When annotations are present they hint at either numerical or alphabetical ordering which may or may not match what the application does. That’s minor. But when no annotations are present, the situation is far worse: my sort-ascending button looks like someone else’s sort-descending simply because of theme differences!

I don’t know how this mess came about, but it ought to be resolved. I suggest that when the icons look like vertical arrows, sort-ascending should point down because the elements of a list will then be increasing in the direction of the arrow.

Hunting Leaks in GTK+ Applications

Hunting leaks in GTK+ application used to be fairly simple: you would run your application under Purify (or, later, Valgrind) and the leak reports would pretty much tell you where to go plug.

That was a long time ago. In the meantime, GTK+ has gotten more complex over its iterations with more caches, more inter-object links, and deliberately-unfreed objects. On top of that, Valgrind and Purify are not particular well suited for finding the cause of leaks: by design they will tell you the backtrace of the call that allocated memory which was never leaked. In a ref-counted world that information is often quite insufficient: the leaked widget was allocated by the gui builder — oh goodie! What you really want to know is who holds the extra ref.

Enter the gobject debugger first introduced by Danielle here. After some major internal work, it has become mature.

I used this for Gnumeric, which was already one of the strictest leak-policed applications in GTK+ land. We leaked a number of GtkTreeModel/GtkListStore objects, for example. Easily fixed. Also, when touching print code or file choosers we leaked massively: one object per printer and/or two objects per file in the current directory. A sequence of bug reports (646815, 646462, 646446, 646461, 646460, 646458, 646457, and 645483) later, GTK+ is now behaving much better. All but the last of these have been fixed. With this, we are down to leaking about 20 Gtk-related objects: the recent-documents manager, im-module objects, the default icon factory, and theme engines. Basically stuff that GTK+ does not want to release.

Please try this on your applications, especially if they are long-running. I still have to kill gnome-terminal, banshee, metacity, and gvfsd from time to time when they grow to absurd sizes. That doesn’t have to be caused by GObject leaks, but it might very well. (I know some of these samples are obsolete; I am not naive enough to believe their replacements would fare any better.)

This might be a good time to remind people that g_object_get and gtk_tree_model_get will give you a reference when you use them to retrieve GObjects. You need to unref when done. The problem is that it is not immediately clear from the g_object_get/gtk_tree_model_get call whether existing code is getting objects, so a certain knowledge of the code is needed.

GHashTable Memory Requirements

Someone threw a 8-million cell csv file at Gnumeric. We handle it, but barely. Barely is better than LibreOffice and Excel if you don’t mind that it takes 10 minutes to load. And if you have lots of memory.

I looked at the memory consumption and, quite surprisingly, the GHashTable that we use for cells in sheet is on top of the list: a GHashTable with 8 million entries uses 600MB!

Here’s why:

  • We are on a 64-bit platform so each GHashNode takes 24 bytes including four bytes of padding.
  • At a little less than 8 million entries the hash table is resized to about 16 million entries.
  • While resizing, we therefore have 24 million GHashNodes around.
  • 24*24000000 is around 600M, all of which is being accessed during the resize.

So what can be done about it? Here are a few things:

  • Some GHashTables have identical keys and values. For such tables, there’s no need to store both.
  • If the hash function is cheap, there’s no need to keep the hash values around. This gets a little tricky with the unused/tombstone special pseudo-hash values used by the current implementation. It can be done, though.

I wrote a proof of concept which skips things like key destructors because I don’t need them. This uses 1/3 the memory of GHashTable. It might be possible to lower this further if an in-place resize algorithm could be worked out.

On Profiling and Sharks

Be careful in applying simplified models to complicated systems.

For example, Federico has been a persistent proponent for using profiling as the major (only?) guide about where to improve performance: Profiling A+B. And that is great as far as it goes. But no further.

If you were studying sharks on the decks of the fishing boat that caught them you might describe sharks as nearly blind, having no sense of smell, and unable to acquire enough oxygen by themselves. That is not a very accurate description of how they appear in their natural environment.

The point of that is that the A+B model argument applies to a system where only one program is running. The model has been simplified to the point where there is no interaction with the outside, so the A+B model is not well suited to describing the programs behaviour in a realistic system. For example, if B is very I/O intensive then it might well be the right place of concentrate performance efforts.

I/O is not the only way to hit other programs hard. Memory usage is another — especially if the program is long running.