scrambled tofu

kpartx – mounting partitions inside disk images

This is a neat trick, and so I thought worth reblogging. I was handed a corrupt SD memory card yesterday and asked if I could try and get some Christmas photos off it.

Typically, the first thing I did was image the card with ddrescue (package gddrescue in Debian/Ubuntu), confirming that the partition table was intact, it looks like the problem was a corrupted FAT, so I wanted to run fsck on the partition.

Back in the day this would have involved either dd-ing the partition out of the image, or re-imaging the card, both slow, or screwing around with loopback offsets. Instead I discovered a tool called kpartx which maps the partitions from the image into the device mapper, giving a device like /dev/mapper/loop0p1, which can be used like a regular block device.

Unfortunately while different, both FATs were damaged, and has been incorrectly repaired by something, and I ended up recovering the files with photorec instead.

Linux work available in Perth, Australia

Happy new year, everyone!

Once again my former employer, Fugro Seismic Imaging, is looking for two new R&D staff for their office in Perth, Western Australia (one to replace someone who’s moving on, and another to grow the team).

The team there is approximately half a dozen people, generally open-source experienced, who are primarily developing graphical front ends to geophysics applications (there’s also some other stuff, in general it’s pretty cool). You’ll almost certainly have to know C. Knowing any of C++, Python, Perl, GTK+, Qt or Fortran also a benefit. You don’t have to know any geophysics, but knowing a bit of maths and physics helps. You’ll easily pick up what you don’t know as you go along.

Pay and conditions are very good. FSI have previously provided sponsored work visas for skilled applicants looking to work in Australia.

If you’re interested, send your CV to the regional manager, Kelly Beauglehole.

more on DPI

Daniel Stone recently wrote this amusing summary of why saying OMG I ABSOLUTELY MUST HAVE MY DPI CORRECT is rarely, if ever, correct.

Since most people who say this aren’t considering the two monitor case, I thought I’d provide a illustrative screenshot.

This is an old screenshot of the resolution-independence branch of GTK+ running the demo program on two different monitors. It shows that both apps the physical same size, with the same physical size fonts (ignore the size of the title bars, those are provided by Metacity, which is using the system GTK+).

However, the font rendering looks different. This is because of the amount of anti-aliasing that goes into making fonts look smooth when you have less pixels to play with. Notice how in the title bars the font looks the same, even though they’re different sizes. That’s because they’re using the same number of pixels, and thus the same amount of anti-aliasing.

I couldn’t find a screenshot of what happens when you have a window that crosses between two monitors. [Perhaps it’s a nonsense case, but it can come up.] At this point everything just looks like crap. We have to choose the scaling of one monitor or the other. It will look wrong on one monitor or the other. It will be a different size to everything else on that monitor. [Before you suggest it; we can’t divide the window and calculate the scaling separately for each monitor across single widgets].

The thing is, for day to day computing, you don’t really care about the DPI, you care about having readable, attractive fonts and visible, clickable icons. So Federico is right. For some specific applications, you do care about DPI, but those applications can already use the XRandR data to take care of themselves [I have written visualisation apps which do this, presenting a scale on the screen of millimetres on screen to metres in the real world. ((And in using these apps, I’ve found plenty of monitors that return nonsense DPIs, forcing me to hardcode the monitor size in X11.))]

This said, I’m not sure why the only font size settings now in GNOME is a quantised Small, Normal, Large, Larger setting in the Universal Access settings pane. gnome-tweak-tool includes a scaling slider. The slider is hard to use though, especially when your text reflows while using it. Federico’s idea seems better.

As DPI continues to increase, it’s probably worth also applying scaling factors to our scalable window graphics, lest icons and the like become increasingly harder to hit. At this point we will likely need to adopt a resolution independent GTK+. However, again what will matter will be readable fonts and visible icons. Using the actual monitor DPIs will likely not be that important.

the death of the engineer

I find it so amusing yet depressing how afraid your average engineer now is now afraid of a computer.

My brother started his first real engineering job and rapidly found himself with a problem he needed to solve. He can’t afford Matlab or Mathematica, so I suggested he teach himself Python and use that. After all, Python has excellent scientific credentials. He looked at Python and agreed it looked easy to learn and met his needs.

His boss however, vetoed the idea on the basis that if a tool were written in Python, no one would be able to come back later and edit it. The implicit assumption here seems to be that programming is hard. That no one else could learn enough of the language to later edit the program.

Once upon a time, computers didn’t do very much. They could read and write data and execute programs. Engineers and scientists had to teach themselves to write their own programs to solve their problems.

Over time programs got bigger and more complex. Then people started collecting them together into packages of tools and turning them into a product to sell to other people. They hired engineer-programmers to work on them. Eventually they glued user interfaces on top and so we get the engineering and scientific packages you see today. [My first programming job was working on one of these packages.]

New engineers start in the field and they are simply taught the package at the user interface level. Sometimes they hit the limits of the interface, and learn how to drive the individual programs underneath.

Rarely though do they still see the computer as the blank canvas which they can use to solve their problem. It does so much now that the idea of learning to make it do more seems foreign. Engineers are now afraid of the tool.

It is my belief that engineering students should be compulsorily exposed to some computer science; The core stuff with useful tools. They should have to implement algorithms that have meaning to engineers: solving equations, not a booking system for tennis matches. Perhaps then they’ll stop being afraid of using computers to do computation.

let us not mourn telepathy-python

But let us instead celebrate gobject-introspection.

While there has been no official announcement, I think it’s probably time we declared telepathy-python to have passed away. Deprecated. In the great attic of the sky.

The python-telepathy client-side bindings have been in a state of disrepair for a long time now, not able to do the heavy lifting that other bindings such as tp-glib and tp-qt4 have been focusing on.

telepathy-butterfly is not really maintained, isn’t supported properly by empathy master, and is looking to be replaced in the future by the MSN Live XMPP support. In the meantime bug #663829 migrates existing butterfly accounts to telepathy-haze. I’m not sure about telepathy-sunshine, but I suspect it also won’t work with the latest Empathy.

Through work I started, and a bunch of other people have continued, telepathy-glib client-side bindings have been growing increasingly more introspectable as time passes. This is now used by gnome-shell, and it works pretty well.

Since someone asked today, here’s how you set your global presence in Python:

[python]
from gi.repository import TelepathyGLib as Tp
from gi.repository import GObject

loop = GObject.MainLoop()
am = Tp.AccountManager.dup()
am.prepare_async(None, lambda *args: loop.quit(), None)
loop.run()

am.set_all_requested_presences(Tp.ConnectionPresenceType.AVAILABLE,
‘available’, ‘Set by script’)
[/python]

Unfortunately we can’t introspect the service-side bindings, they do some things that are very creative and clever, but that gobject-introspection just doesn’t understand. Which means you can’t write a CM using PyGObject yet. The solution, I think, is likely to come with a future, GVariant-based telepathy-glib, but I’m still trying to figure out how we can make things (like mixins) work in a way that g-i can understand.

mistakes with g_value_set_boxed()

In today’s PSA, mistakes with g_value_set_boxed(). A mistake that’s been made several times by yours truly, and only realised today thanks to Xavier.

At some point in the GLib 2.22 cycle, types such as GArray, GPtrArray, GByteArray and GHashTable ((You can discover what g_boxed_copy() and g_boxed_free() will do for a type by looking for its G_DEFINE_BOXED_TYPE in GLib, which is probably in gobject/gboxed.c.)) gained ref() and unref() methods, which allowed things like g_boxed_copy() and g_boxed_free() to be a stack faster for those types. This was mid-2008.

Danni, who didn’t get the memo, had been occasionally writing bits of code like this:

[c]GArray *array = g_array_new (…);

g_value_set_boxed (value, array);
g_array_free (array, TRUE);[/c]

Which, if you read the code, will keep the wrapper alive, so your code won’t crash, but will release all the data the array contains. Similarly for g_ptr_array_free() etc. It nicely zeros out the length of the array, so when you come to read it in your GValue, it looks empty.

The correct thing to do (of course) is unref your data, which will then free the memory and the wrapper when the refcount reaches zero.

[c]GArray *array = g_array_new (…);

g_value_set_boxed (value, array);
g_array_unref (array);[/c]

Update: As pointed out in this simple example, you can replace this with the following pattern:

[c]GArray *array = g_array_new (…);

g_value_take_boxed (value, array);[/c]

Which passes your ownership to the GValue. Where this falls down is for more complex values built for dbus-glib, where the ownership isn’t clear-cut, and so you need to free the components separately afterwards.

a guide to writing git commit messages

The GOPW application period is coming to a close soon, and so potential interns are currently getting their initial contribution in to qualify.

Obviously the nicest way to submit your patch is as a commit, or series of commits using git, either via git bz or git format-patch (a branch is nice too, but submitting the patch in Bugzilla allows for inline review).

Of course, after your patch is correct, the maintainer is going to want to merge it into the tree, in which case you need to have a useful commit message. GNOME has some guidelines on commit messages, but here are the important dot points:

the first line should be a short description, no more than 72 characters and does not end with a full stop;
the next line should be empty space;
following this is an optional longer description, this should be written in regular paragraphs with full sentence structure;
finally if this patch fixes an open bug, there should be a link to the bug report, in a paragraph by itself.

Here are some good examples taken from Empathy:
Don't destroy MainWindow on delete-event, just hide it


This stops MainWindow being unreferenced, and prevents crashes caused by things which then try to use it.

Fixes: https://bugzilla.gnome.org/show_bug.cgi?id=659118

This one has a tag:
e-account-widget-jabber: Use GtkGrid instead of GtkTable


This works around bug #657621.

https://bugzilla.gnome.org/show_bug.cgi?id=646630

Here’s another way of doing a tag (preferred by some projects):
[contact-chooser] emit 'activate' on 'row-activated' as well

This means the selection can be activated when the treeview is focused.

Or a one-liner commit:
[contact-chooser] Move the selection up/down without defocusing the entry

Remember that your commit message is important. They tell other developers what’s changed. What a commit fixes, or does, or is meant to do, and occasionally, what it doesn’t do. Commit messages are often used by the maintainer to write the release notes. When things break, people come back looking through the code for what broke it (e.g. via a git bisect) so clear and useful commit messages are essential.

gnome-terminal menubar issue in Ubuntu 11.10

I upgraded to Ubuntu 11.10 today, which seems to have done a nice enough job of integrating GNOME 3.2, but I was instantly driven mad by all of my GNOME Terminals showing their menubars, in what appeared to be direct contravention of what the setting appeared to be (or even what GTK+ thought the visibility status of the widget was!). This was GNOME bug #658760.

After a stack of debugging and one lucky break, I discovered the culprit was infact Ubuntu’s appmenu module.

[The lucky break was that while setting up a bisect, I realised I could replicate the bug only in the GTK+ 3 build of the terminal, which was printing out warnings about appmenu, something I’d already uninstalled for GTK+ 2.]

Anyway, you can fix the problem by removing appmenu-gtk3 (personally I’d remove appmenu-*) or unsetting the environment variable UBUNTU_MENUPROXY which is set in /etc/Xsession.d/80appmenu or something. The Ubuntu bug is #879190.

~~Random side note: can someone update my hackergotchi please?~~

Finding redundant GObject classes in Empathy

As part of a bugfix I did today ((Thanks, as always, to my employer, Collabora, for letting me work on Empathy)), I made a commit to remove a redundant class from Empathy, which made me wonder if there were any other redundant classes in Empathy. A quick grep of G_DEFINE_ told me there are some 116 classes in Empathy, so I wasn’t going to check them all by hand.

Instead I put together this script, which I share with you in case it’s useful, which basically checks for all classes defined with G_DEFINE_TYPE and then looks to see if anything with that namespace is used in another file. It generates false positives for classes that aren’t used outside the file they’re defined in, or classes that have different namespaces to the classname, but it produces a much more manageable list.

I did consider looking for unused symbols, but couldn’t work out an easy way to do it properly. Empathy’s compile process is split into two archive libraries (libempathy.a and libempathy-gtk.a) and several binaries (empathy, empathy-call, empathy-accounts, etc.), so I couldn’t think of a way to ask the linker to find any unused symbols. I put together this second script, which will build a list of symbols in archives and look for those symbols copied into the binaries, which gets some of the way there, but will miss any unused symbols defined in the binaries’ sources (src/).

Finding Ada — Chandni Verma and Tiffany Antopolski

October 7th is Ada Lovelace Day. A day that showcases women in science and technology by profiling a woman technologist or scientist on your blog.

This year I wanted to write about two women who I’ve met and worked with in the last year as a mentor of the GNOME Outreach Programme for Women and the Google Summer of Code.

Chandni Verma
I met Chandni when she applied to be an Empathy intern for the GNOME Outreach Programme for Women. Chandni had never worked in open source before, but she approached it with an amazingly keen enthusiasm. Before too long Chandni began to grasp the ropes, learning GTK+ and Telepathy. She then went on to fix bugs in Empathy as well as the Telepathy connection managers.

Chandni studied computer science at the Dr M.C. Saxena College Of Engineering & Technology and has a passion for algorithms and would regularly participate in algorithm competitions. She learned about open source from a friend and in her final year of college and ordered Ubuntu CDs to check it out. Chandni says “I liked the idea of open source particularly since it allowed one to learn and evolve without any restrictions even if you are a student.”

Chandni applied for GOPW after completing her final year and has remained working on Empathy since the end of her internship. Chandni gave her first conference presentation, detailing her experience with the programme, at GNOME.Asia. She had hoped to give it again at the Desktop Summit, but unfortunately bureaucracy got in the way. She looks forward obtaining a position with a company working on GNOME.

Tiffany Antopolski
Tiffany was in the same intake of the GOPW as Chandni, working on the documentation for GNOME with Paul Cutler. I then met Tiffany when she applied for the Google Summer of Code to work on integrating Empathy and Evince.

Tiffany is studying software engineering at McMaster University and also taught first year programming to students of the same course, having helped to design the curriculum. Before studying software engineering, Tiffany was a veterinary technician, and used to work in an emergency vet clinic. She also works part time in a neuroscience research lab.

Tiffany got involved in GNOME through the GNOME Documentation Project after being an Ubuntu user for around two years prior. In 2011 Tiffany attended the GNOME docs hackfest, the Open Documentation Conference and the Desktop Summit, where she worked on improving the documentation for GNOME. She successfully applied for Google Summer of Code with a project to work on integrating Telepathy into Evince. Recently Tiffany began working on the Esperanto translation for GNOME.