Sunday, February 01, 2009

Profiling with kcachegrind made easier

During the third day of this years Baarn UI sprint, I couldn't resist to look into profiling Plone's startup speed again. As I've been recently getting used to using kcachegrind for visualizing profiling data, I wanted to make it easier to get the profiling data right in this tool.

When doing profiling on full application requests the WSGI middleware repoze.profile is an excellent tool to use. But for profiling startup speed I needed to get back to profiling an individual function call. profilehooks to the rescue? Sadly not.

In order to convert the profiling data into a format understandable to kcachegrind I needed the data to be in the cProfile format produced by Python 2.5's new profiling library. profilehooks still lives in the Python 2.4 world and has profile and hotshot support.

So I made a new tiny package called profilestats which will use the cProfile module instead. It works on both Python 2.5 and 2.6 and writes out the profiling data directly into a file called cachegrind.out.profilestats for kcachegrind to grok.

On the actual Plone startup speed front there isn't any real news. Parsing and executing ZCML, deepcopying Archetypes schemata and just the sheer number of imports and packages to load are still the slowest parts. Especially the recursive algorithms used in zope.interface and zope.component are still the slowest individual parts by a large offset.


  1. It's a great shame that your blog isn't syndicated on Planet Python.

    profilehooks now supports cProfile, although it doesn't (yet?) interface with kcachegrind directly.

  2. I recently added my blog to Planet Python but only for posts I mark with the Python tag. I don't feel like spamming the general community with Plone specific stuff. This post should have gone to Planet Python indeed :)

  3. maybe you guys should merge profilestats and profilehooks. it's confusing to have 2 very similar projects which are both a work in progress. or am i missing something?