There have been a few complaints that it takes too long time to start up python programs, especially the ones which has been written to use GTK and other GNOME libraries.
Yesterday I wrote a program called python-launcher, inspired by maemo-launcher which is a way to reduce start up time and memory usage used by python processes.
The trick works in the following way:
When the system is started, a daemon is run which imports all the modules which we want to cache. The normal launcher (/usr/bin/python) is replaced by a small client which communicates with the header and tells it to fork a new process and then execute the python script.
The gtk module is already imported in the parent process so there’s no need to re-import it for each client (eg, application) that is going to use it.
The “import gtk” statement is now instantaneous, but there’s a small hidden cost of creating a new GdkDisplay and associating a cached GdkSettings inside the deamon itself before the application is executed.
Especially the OLPC suffers due to excessive start-up costs.
I got the following numbers:
importing gtk normally: 1000ms importing gtk with help if the launcher: less than 1 ms hidden cost in the daemon for each child: around ~60ms
I looked quickly at the smap script written by Ben Maurer and it appears that the added memory cost of using a deamon is already saved when launching the second process using gtk.
Update: Hi Planet OLPC! Thanks Ivan.
Excellent!
Thanks for your work. It’s like a desktop mod_python. I had an idea like this but forgot about it. Since more and more GNOME apps and applets are using python, this makes great sense – as does the work on removing timers from python to save on power consumption. Progress!
Perhaps it would be better to have the daemon started with the first pygtk app rather than with the system. The Linux desktop already has enough boot speed issues.
Like said though, more and more applications are using python and will probably continue in the future so this is definetely a good thing.
If you preload GTK in the daemon, then you’ll also save lots of memory, because forked processes share memory whenever possible. I’m also using such a technique to save memory in some my Perl apps.
But how do you deal with terminals issues? Do you deal with them at all? My Perl apps have a readline-based console interface and I had to use all kinds of hacks to make sure that the forked process uses the right console. This doesn’t even work completely – for example I can’t call tcsetgrp() on a PID that is not part of the current session.
Hongli Lai: I’m not dealing with terminal issues at all.
I once wrote another little hack I could reuse inside python-launcher to make it possible to use as a normal terminal.
Can this also be used to help the startup times of apps which just use python for plugins? I did some profiling a while ago and starting up vms for plugins was a large part of many profiles. If it can be reduced, that that’s a big argument to use python based plugins everywhere.
Mark: No, python-launcher is tied to python scripts. Have a look at maemo-launcher, for which a python plugin could be written which would solve that problem.