Comments on: Thumbnail Followup (#2)

By: Julien

Julien — Thu, 30 Aug 2007 08:43:29 +0000

For me it seems like duplicating new Tracker functionalities. Tracker provides on-the-fly thumbnail generation. Would it not be easier to just requests thumbnails to Tracker through dbus?

Cheers

By: Xav

Xav — Wed, 29 Aug 2007 20:39:44 +0000

How about using a generic metadata daemon, like Tracker ?

By: Greg

Greg — Wed, 29 Aug 2007 20:22:49 +0000

Concerning the second idea: what if you generate thumbnails in a separate directory, and you “merge” them in synchronously to the actual thumbnail cache dir?

By: Jerome

Jerome — Wed, 29 Aug 2007 19:39:57 +0000

Yeah. I’d vote for not using .thumbnails, for tons of reasons. It’s just a needless disconnect. I’d just put it in the directory of the actual files, like every other OS on the planet does. If there’s no permission to do so, just don’t. Recalculate them each time. No big deal.

It also clogs up my NFS home dirs. Makes things more difficult than neccessary.

By: Henry

Henry — Wed, 29 Aug 2007 18:34:48 +0000

4: Cache in the dir the image lives in. Bonus: rmdir gets rid of the cache too.

By: Detlef

Detlef — Wed, 29 Aug 2007 18:07:03 +0000

I think a very important fact these days is that you have much crap in your thumbs directory.

Many programs don’t update theire thumbs if they move documents around or delete them. So a big win would be to create a cron job that scans all thumbs and remove them if the asociated document is gone.

For my own use i wrote a little ruby script for this task. With the first run it deleted several thousend thumbs.

The fastest work is the work you don’t have to do.

Cheers
detlef

By: John

John — Wed, 29 Aug 2007 17:43:15 +0000

Well, the OS already does what you want (except for loading the directory structure into memory upon mount, which could be done with a userspace daemon). Have you considered just letting the OS worry about doing what it was designed to do?

By: Jerome

Jerome — Wed, 29 Aug 2007 17:23:53 +0000

As for cleaning the cache, seems reasonable to just scan for old files in the background in some fashion and blow them away. Do this in another thread. Easily isolated. Thread does one job and does not need to notify on completion.

By: Jerome

Jerome — Wed, 29 Aug 2007 17:21:05 +0000

This proposal seems extremely… hacky, for something which is arguably such a simple problem. Why not just attempt to read the thumbnail file async as it’s needed? This seems extremely obvious, and works quite well for other situations: evolutions email cache, etc.

Let the file system handle keeping the file in memory if it needs be. That may mean it gets left in the block cache.

I don’t really understand why this problem is hard, actually. Maybe I’m missing something.

By: blah

blah — Wed, 29 Aug 2007 17:19:51 +0000

Hmm, i don’t really see the problem with your threads based idea.

Everytime you process a new file you should:

open() it.
fstat() it.
process it.
fstat() it again.
close() it.

Only if the access times in the stat struct did not change while the file was processed you have a valid cache entry. If it changed you should drop what you just processed and move it to the end of your work queue.

The readdir() calls should be nothing more than a feeder for the work queue. If files disappear or are modified should not matter to the work queue.