Category: General

nautilus-open-terminal 0.9 has been released. Get it! It has GIO support, and auto-detects the remote shell for SFTP terminals. Thanks to Alex Larsson and Wouter Bolsterlee!

This release is dedicated to Telltale Games, creator of the “Sam and Max” episode games series, a sequel to the infamous “Sam and Max – Hit the Road” point & click adventure

I’d also like to thank all the nautilus-open-terminal translators, but I seem to be to stupid to insert a well-formated monospace excerpt from the NEWS file here, that is not destroyed by the (otherwise impressive) WordPress layout engine.

You may remember my posting regarding thumbnail caching.

The problem is that all applications using GnomeThumbnail will read the entries ~/.thumbnails/normal or ~/.thumbnails/large [and ~/.thumbnails/failed] synchronously, as the first thumbnail request is made, which means a notable time [more than a few seconds] for a few hundred MB of thumbnails. There is a long-standing bug report about that issue.

Why the cache, in the first place?

The idea was to keep around an in-memory list (a cache) of all available thumbnails, and additionally to store the previously-requested thumbnails in memory, as it is likely that previously-requested thumbnails are used again.

The problematic aspect of the cache is that ~/.thumbnails may contain hundrets of megabytes, and thus be very big. This implies that the cache refresh is very expensive.

First idea: Get rid of cache

My initial idea was to solve this by removing the in-memory cache altogether. While users reported that this solved their problems and didn’t cause any performance impact, this may not be generalized: Currently, the user-visible thumbnail loading time is dominated by the time to synchronously read the thumbs, not by the cache-lookup time. This will not be true anymore as we have asynchronous thumbnail loading in Nautilus [which is not written yet, but should heavily improve performance].

Second idea: Multi-threaded cache voodoo

So the next idea was to use a multi-threaded solution, which would refresh the cache in a worker thread, and circumvent it entirely during refresh when requesting/generating a thumbnail. While it sounds good, it gets really nasty as you realize that POSIX doesn’t specify what happens as a directory changes during readdir(). Assuming you refresh the cache from disk [worker thread], and at the same generate a new thumbnail without using the cache [main thread], you’ll end up with a modified file system and you cannot be sure about the validity of any entries you read – you’d have to reopen the directory and reread it entirely. This means no cache hits during thumbnail generation, i.e. if you open two directories simultanously with Nautilus and the thumbs for one are generated, and the thumbs for the others aren’t, you’ll get no cache hits at all. Maybe we could use file change notification, but it is platform-dependant and we don’t want to explicitly write code against Linux or some other UNIX and have gazillions of #ifdefs in the code for dnotify, inotify etc.

Third idea: (to-be-written?) on-demand in-memory file systems

I think the smartest solution [that happens to mean no work for us, i.e. just drop our cache, cf. first idea] involves finding an on-demand in-memory file system, which forms an ideal cache for many applications. Dear lazyweb, do you know of any FS that does the following:

• Store the entire FS contents in a file if it’s not mounted
• Load the entire FS directory structure into memory as it is mounted
• Load the file contents into memory as it is requested
• Store the file contents into memory and back into the image as it is written
• Live in user-space, possibly using shared memory, as a bonus you’d tell it anytime to prefetch the FS structure at any given point [in out case as the session starts]

You’d just mount it to ~/.thumbnails and let the voodoo happen. Yes, this is also platform-dependant, but it can be implemented for all platforms and doesn’t make us depend on the platforms file system capabilities, falling back to a performance-reduced but not memory-intense behavior.

Thanks to the guys at the freenode ##c channel for the fruitful discussion, especially wobster for the POSIX and RAMFS hints.

Comments?

Federico wrote:

Simon Holm Thøgersen looked a bit more into yesterday’s timeline, and he found that Nautilus causes all of ~/.thumbnails to be scanned when creating the desktop window. The thumbnails will need to be read eventually when displaying the files in ~/Desktop, but definitely not when the desktop is being initialized.

This is a very good catch.

I investigated the libgnomeui thumbnailing code. When the first thumbnail is looked up, it seems to read the entire thumbnail directory, and add all existing thumbnail files to the global thumbnail cache, which will cause readdir(), and malloc() calls. It will then save the thumbnail directory’s mtime and a thumbnail is requested, it will stat the entire directory again if a thumbnail is changed (and thus changed the directory’s mtime).

Denoting whether a thumbnail file is present or not is the file system’s job, there is no point in duplicating the FS info in a hash table.

It would IMO be way better to just purge the cache containing all thumbnails, and instead only cache the previously requested ones. We’d directly stat() each thumbnail file, and store and cache the mtime per thumbnail file in a hash table containing only the previouly-seen thumbs, and check whether the requested one matches the cached one.

I’ll do some more investigations and report about my findings, but this looks like it may well be the reason that users report desktop startup speed differences of orders of magnitude, depending on the number of stored thumbnails.

Update

I submitted a libgnomeui patch against the related bug report (#430123). Even with a warm filesystem cache, the subjective Nautilus startup could be significantly reduced with a ~160 MB thumbnail cache.

Today I had to solve the following quite interesting mostly mathematical problem:

I received an instruction manual with continuous pages (1, 2, …, n), where n is a multiple of 4.

I wanted to print them as DIN A5 on a DIN A4 sheet, two-sided. Usually, this means printing [ (1, 2), (3, 4) ], …, [ (n-3), (n-2), (n-1, n)] each of these pairs (“tuples”) on a side of a sheet. The [ … ] represents a DIN A4 sheet.

In my special case however, I wanted to stack all of the DIN A4 papers, fold them in the middle and staple them to stick together. Just pull out a random technical manual and you’ll see what I mean: The outer DIN A4 sheet contains the front and back cover, and the first and last inner page, the inner-next sheet contains the second and third, and penultimate and pre-penultimate inner page.

When the manual is closed, all the DIN A4 pages will be nested.

Since no office software I knew offered this special feature, I was stuck.

The required steps were:
1) Reorder the pages (1, …, n). The resulting ordered set (called n-tuple, which will contain all numbers between 1 and n) could look like (…, 1, …, n, …).
2) Print the reordered pages on a duplex printer.

That’s how I realized 1)

a) Consider the correct permutation. You’ll soon find out that for the outer side of the outer sheet, you’ll need (n, 1). This is because an open book viewed from the back has its back on the left side and its front on the right side.

For the inner side of the cover, you’ll need (2, n-1), then you’ll need (n-2, 3) and so on until you’re finished, i.e. you’ve reached (n/2, n/2+1).

I wrote a little python permutation algorithm as a proof of concept that also explains the algorithm.

#!/usr/bin/python

# reorder the numbers between 1 and n (argv[1]) according to the
# "alternating cut-zip sorting" algorithm, used for printing
# continuous pages (resized) to pages of twice the original size,
# in an order appropriate for instruction manuals that
# consist of nested sheets.
#
# I just called this algorithm "alternating cut-zip sorting" algorithm,
# feel free to correct me if there is another name for it!
#
# We have n pages, n must be even, and should be divisible by 4 to yield
# proper results. If it is not divisible by 4, you'll need a blank front
# and back page.
#
# Example:
#   Assuming n=8, we previously had DIN A5 pages as an n-tuple
#   (1, 2, 3, 4, 5, 6, 7, 8)
#   Now, we want to print this in duplex mode on DIN A4 paper like
#
#    sheet 1: [(Page 8 | Page 1), (Page 2 | Page 7)]
#    sheet 2: [(Page 6 | Page 3), (Page 4 | Page 5)]
#
#   This algorithm will arrange for the (8, 1, 2, 7, 6, 3, 4, 5)
#   order, and the printer driver just has to print the new document
#   duplex 2-on-1 resized on DIN A4 paper.
#
# Construction of this permutation:
#  We 'cut' the original tuple in the middle,
#  count up in one half and count down in the other half,
#  and in each iteration glue a pair together, swapping order
#  (greater number left, smaller number left) in each iteration.
#
#  The swapping is required because once you turn over pages, you'll
#  jump from the left half of a DIN A4 page to the right half and vice
#  versa.
#
# General pattern for new page order (spaces added for better readability):
#  n, 1,   2, n-1,   n-2, 3,   4, n-3,   , ..., ...,   (n/2), (n/2)+1

import sys

n = int (sys.argv[1])

if n % 2:
  print "expected even number, but " + str(n) + " is not even! Aborting."
  sys.exit(1)

for i in range(1, n/2+1):
  if i != 1:
    sys.stdout.write (' ')

  if i % 2:
    sys.stdout.write (str(n+1-i) + ' ' + str(i))
  else:
    sys.stdout.write (str(i) + ' ' + str(n+1-i))

sys.stdout.write ('\n')
sys.exit (0)

b) Modify an existing python script that extracts pages from a PostScript file and reverses them to use the permutation algorithm.

c) Use the modified script to reorder the pages of a PDF file.

This is done by converting the original PDF file with acroread, using

acroread -toPostScript -size A5 myfile.pdf

We use acroread because it yields way smaller compressed PS files than pdf2ps can ever produce due to usage of patented algorithms.

Now, the PS pages are reordered with the modified script, and converted back:

ps2pdf -sPAPERSIZE=a5 myreorderedfile.ps

Now proceed to step 2) and you have a pretty instruction manual.

Update

The comments on this article yielded some interesting comments: My python script could have been shorter and more elegant, the psutils package offers similar functionality and OpenOffice.org offers brochure printing which seems to be exactly what I’m looking for. Thanks for all your feedback.

Get it from ftp.gnome.org. Spread it by voting for it.

This is the first release that is distributed through the ftp.gnome.org server. Thanks to Davyd for making it happen.

Unfortunately, a pal of mine behind cluecoder.org that was used to host the old packages is still in clinch with the old hoster so I can’t access my old tarballs and some privare data.

nautilus-open-terminal 0.8
==========================
"Alive"

        * Dedicated to Klaus Kreuzeder, an excellent Saxophonist. http://www.trick-music.de/klaus_kreuzeder.htm

        * Depend on Nautilus 2.17.2
        * Use activation URI of selected files for determining URI. This allows to use the extension for your Desktop bookmarks.
        * Also deal with "ssh" URIs and treat them like "sftp" URIs.
        * Use open() and cwd() to change directories. Should fix trouble with blocked media ejection.

        * Added translations
                ar      Khaled Hosny                    
                bn_IN   Runa Bhattacharjee              
                be      Ihar Hrachyshka                 
                da      Peter Bach                      
                dz      sonam pelden
                et      Ivar Smolin                     
                gu      Ankit Patel                     
                ka      Vladimer Sichinava              
                ml      Ani Peter                       
                sl      Matej Urbančič                  

        * Updated translations
                ca      Gil Forcada                     
                en_GB   David Lodge                     
                ne      Mahesh subedi                   
                nl      Wouter Bolsterlee               

Joe [1]: I already proposed a “desktop-entry-hotplug-spec” [2] some time ago. I sent it two times and didn’t receive any feedback, so I wasn’t sure whether it is considered important.

The other feature you demanded (actions for files) is IIRC already implemented by KDE. I’m not sure how we should tackle this and I am not so happy with the KDE system.

Example:


[Desktop Action reSize640x480]
Name=Scale to 640x480
Icon=images
Exec=mogrify -resize 640x480! %U
Terminal=false
Type=Application


While it allows for maximum flexibility we’ll end up with a can of worms when internationalizing this since it will be distributed with each application, especially with routine actions like printing. Also, it doesn’t consider how many items are selected – it is common to use ngettext() for internationalizing strings referring to countable objects.

It would really be great if we at least had some actions – maybe similar to those in [2] – for printing and enqueueing. Applications would simply provide a special Exec parameter, and we could immediately see that an application provides this capability.

Oh, and while we’re at it I also have an old “keyword-spec” draft that might be of general interest. I’ll sent it to you by email to get some feedback.

[1] http://joeshaw.org/2007/01/14/452

[2] http://mail.gnome.org/archives/gnome-hackers/2006-October/msg00002.html

For years I’ve been very unhappy with the MIME type/application association.

The Ubuntu guys did a spec [0] and came up with a not-so-pretty UI, so I hacked something together. The (not yet implemented) architecture is described under [1], a very first (failed) UI experiment can be found under [2] and the first serious proposal under [3]. I really like it :).

[2] and [3] provide python scripts that should run flawlessly under Ubuntu Edgy. It turns out that script languages extremely simplifiy communication between developers and usability experts as both can run a script and there is no compilation hurdle.

Feedback appreciated, preferably on the usability list.

I still don’t have a concept how we can properly integrate URI handling with MIME type handling. Maybe it’s best to just set the HTML/email message MIME type handler when changing the URI handler (the latter is stored in GConf).

[0] https://wiki.ubuntu.com/Usability/SpecEnhancedPreferredApps

[1] http://mail.gnome.org/archives/usability/2007-January/msg00064.html

[2] http://mail.gnome.org/archives/usability/2007-January/msg00065.html

[3] http://mail.gnome.org/archives/usability/2007-January/msg00068.html