Speeding up NewsBruiser – Initial results

Switched to the hotshot profiler instead of profile. The hotshot profile is better than profile. Made two changes, one to delay reading a notebooks configuration until something actually needs something out of it. Second change is storing the order of the blogs in a seperate file. This avoids NewsBruiser reading the order from each blog.

The hotshot profiler produces different results. A profile before above changes:

         77867 function calls (75825 primitive calls) in 0.905 CPU seconds

   Ordered by: internal time, call count
   List reduced from 856 to 100 due to restriction 

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      139    0.179    0.001    0.347    0.002 Notebook.py:626(readConfiguration)
        1    0.062    0.062    0.165    0.165 core.py:15(?)
        3    0.062    0.021    0.063    0.021 __init__.py:9(?)
       86    0.058    0.001    0.506    0.006 NBConfig.py:156(__registerPluginDir)
    15078    0.057    0.000    0.057    0.000 string.py:351(find)
   402/71    0.039    0.000    0.095    0.001 sre_parse.py:374(_parse)
       26    0.039    0.001    0.481    0.019 __init__.py:1(?)
     6459    0.038    0.000    0.049    0.000 util.py:59(replaceBaseURLs)
     6526    0.031    0.000    0.048    0.000 IWantOptions.py:154(getOption)

And after:

         28853 function calls (26806 primitive calls) in 0.467 CPU seconds

   Ordered by: internal time, call count
   List reduced from 856 to 100 due to restriction 

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.060    0.060    0.077    0.077 drv_libxml2.py:35(?)
   402/71    0.040    0.000    0.106    0.001 sre_parse.py:374(_parse)
        1    0.037    0.037    0.114    0.114 saxexts.py:41(_create_parser)
       26    0.020    0.001    0.697    0.027 __init__.py:1(?)
        1    0.019    0.019    0.312    0.312 feedparser.py:12(?)
   747/67    0.018    0.000    0.038    0.001 sre_compile.py:27(_compile)
     3541    0.017    0.000    0.025    0.000 sre_parse.py:201(get)
        3    0.016    0.005    0.065    0.022 __init__.py:3(?)
        1    0.015    0.015    0.051    0.051 cookielib.py:26(?)

Have to do a little work before I can commit this. I need to ensure the cache file will be saved whenever the order changes (or a new blog is added). Also need to rest it on Python 2.2 (version on the server). Still pretty good for a 120 line patch (large part consists of indenting changes and some debugging code).

Update: Patch has been committed.