AppData and the gettext domain

When users are searching for software in GNOME Software it is very important to answer the the question “Is this localized in my language?” If you can only speak Swedish then an application talking just in American English is not much use at all. The way we calculate this in the AppStream builder is to look at the compiled .mo files, breaking them apart and then using statistics to work out what locales are included.

When we’re processing distro packages we usually extract them one at a time. We first try for a gettext domain (the .mo file name) of the distro package name, and if that’s not found then we just try and find the first .mo file in any of the locale directories. This works about 70% of the time (which is good) but fails about 30% of the time (which is bad). For xdg-app we build the application in a special prefix, along with any dependent libraries. We don’t have a distro package name for the bundle (only the application ID) and so the “first .mo file we can find” heuristic fails more often that it works. We clearly need some more information about the gettext domain from the upstream project.

AppData to the rescue. By adding this in the AppData file informs the AppStream generation code in the xdg-app builder what gettext domain to use for an application. To use this you just need to add:

  <translation type="gettext">the_gettext_domain_here</translation>

under the <component> tag. The gettext domain is normally set in the configure.ac file with the GETTEXT_PACKAGE define. If you don’t have this extra data in your application then appstream-util validate is soon going to fail, and your application isn’t going to get the language metadata and so will be lower in the search results for users using GNOME Software in a non-C locale. If your GNOME application is available in jhbuild the good news is that I’ve automatically added the <translation> tag to 104 projects semi-automatically today. For XFCE and KDE I’m going to be sending emails to the development mailing lists tomorrow. For all other applications I’m going to be using the <update_contact> email address set in the AppData file for another mass-emailing.

Although it seems I’m asking people to do more things again and again I can assure you that slowly we’re putting the foundations in place for an awesome software installer experience. Just today I merged in the xdg-app branch into gnome-software and so I’m hoping to have a per-user xdg-app preview available in Fedora 24. Exciting times. :)

Published by

hughsie

Richard has over 10 years of experience developing open source software. He is the maintainer of GNOME Software, PackageKit, GNOME Packagekit, GNOME Power Manager, GNOME Color Manager, colord, and UPower and also contributes to many other projects and opensource standards. Richard has three main areas of interest on the free desktop, color management, package management, and power management. Richard graduated a few years ago from the University of Surrey with a Masters in Electronics Engineering. He now works for Red Hat in the desktop group, and also manages a company selling open source calibration equipment. Richard's outside interests include taking photos and eating good food.

17 thoughts on “AppData and the gettext domain”

  2. And if my appdata.xml has no I should just add one right under the top-level element?

    (I’d use ‘appstream-util validate’ to confirm my guess, but it fails due to “Not enough tags for a good description”, and I’ve no idea if it prints all errors, or if it stops after the 1st error, and I just can’t re-write the today.)

    1. Your blog silently eats XML tags in the comments, and it has no preview.

      This was supposed to say “has no LESS THAN SIGN component GREATER THAN SIGN I should” and “top-level LESS THAN SIGN application GREATER THAN SIGN element” and “rewrite the LESS THAN SIGN description GREATER THAN SIGN today”.

      1. Yes, sorry, I’m just using the default wordpress options. If your AppData file has an <application> tag rather than a <component> tag this is fine (it’s the old syntax, but still very much supported) but you can easily upgrade your file using appstream-util upgrade name-of-appdata.appdata.xml which converts everything automatically.

    1. The AppStream metadata does have <lang> tags with exactly this information in. The percentage actually corresponds to “how complete is the .mo file”.

  4. Please keep in mind that running in a certain locale doesn’t mean that I don’t understand any other language. In smaller languages it is common to have a desktop with applications in multiple languages. Untranslated applications should still be easily visible and not outranked by any and all translated choices. I’d be very happy if you can share some details on exactly how this information is used for ranking.

      1. Thanks. I’m not seeing anything that looks related to translations. Is this part of what is returned by gs_app_get_search_sort_key ()?

        Regardless, thanks for the work on this.

  5. > If you don’t have this extra data in your application then
    > appstream-util validate is soon going to fail, and your
    > application isn’t going to get the language metadata

    (a) What is the right thing to do if the application has no translations? (in order to avoid the appstream-util validate fail I mean, clearly the right thing to do to avoid dropping down the list is i18n).

    (b) should add-ons do this too?

    1. Good questions. For apps with no translations but wanting to validate I guess just acknowledging the fact should be enough, e.g. doing <translation/> — if that fails to validate let me know and I’ll fix it. If addons also include a <translation> tag that would be fine, but not required for the validator. I don’t know if it makes sense to “promote” or average out the addon language stats with the main app, but including the tag isn’t going to hurt anything.

      1. Thanks! Yes the [translation/] single self-closing tag seems to validate (with appstream-glib as packaged in Fedora 23).

  6. What about applications using a different translation system, e.g. Qt .qm files (Note that KDE applications typically DO use gettext, through kf5-ki18n, but some Qt applications do not want to depend on ki18n and prefer using the system Qt provides.), Java .properties translations, etc. (Some applications even make up their own translation system.)? Do you have any plans to not make those show up as “untranslated”?

    1. I added support for the QT translation system this week, to use this just use <translation type=”qt”>id<translation> where the ID is the name before the locale eg id-en_GB.qm

      If any app does something crazy (e.g. the eclipse system) you can always specify <lang> values in the AppData file as well, although this isn’t awesome as it requires you to manually update them.

  7. Finally linux will have proper software deployment. Great work. However I quite don’t understand difference between xdg-app and Canonical’s Snappy, hope they are both different beasts.

Comments are closed.