Even though it is weekend we are still hard at work at the Plone Performance Sprint to make Plone perform better. At the Plone Conference, Alexander Limi showed something he and Mike from YouTube were working on — a new template system code-named NKOTB — and showed some figures that showed it outperformed current template systems by several orders of magnitude. Since Hanno mentioned that the navigation tree is a bit slow at the moment, even though it uses extensive caching we wanted to see if NKOTB could help us there.

Compiled template

We tried compiling the template into python code by first running the NKOTB compiler, and then optimizing by hand as the compiler is not quite production ready yet. Both the template for the navigation portlet and the recursive template used for each element in the navigation was turned into python code.

When testing this, we turned off ramcache of navigation tree, created a folder with 100 subfolders and a default page (to avoid folder listing affecting the results too much) and measured the time to get the folder default page with a navtree with 100 items. We measured the average of 100 page view, and tests were performed on a MacBook Pro.

Render method benchmark

We first used profilehook timecall on the render method to  measure the improvements on the navigation portlet itself, independent of total page view time.

This is pretty impressive if I may say so. The data is assembled in a dictionary, which means that there are no security checks on rendering. It's all about pushing the contents of a dictionary to a buffer, and the compiled template beats the regular page template one.

Page view benchmark

Next we tested the entire page view to see how big the total effect is.

This is faster for sure, 30ms slower than cached but without the memory usage. Psyco fast is the optimized code with psyco enabled.

The navigation portlet is cached per user per URL. When a user first loads a page, the response time of the green or the orange bar would apply. On subsequent loads of the same page, the blue cached one would apply. If your use case is mainly anonymous users, the portlet will usually be retrieved from cache. If you have a site with a lot of authenticated users, such as a typical intranet, with a lot of users, there would be a lot of slow cache misses, and the cache would fill up with a lot of HTML, increasing memory usage.

Check it out for yourself

If you want to use this for yourself, you can check it out from collective:

https://svn.plone.org/svn/collective/experimental.portlet.navtree/trunk

Then add it to your base.cfg or buildout.cfg

zcml =
    experimental.portlet.navtree
    experimental.portlet.navtree-overrides

and

eggs =
    experimental.portlet.navtree

Buildout, restart and enjoy. At the moment this is only intended as an experiment for Plone developers, which is why we only have a buildout-based installation procedure at the moment.

A taste of the future

Hanno and Florian have just gotten page template compilation working, and we are definitely going to pursue the page template compilation further to see what we can do with performance. Even if we don't get the same improvements as in the hand optimized navigation tree portlet, it seems like there can be real improvements for most templates.

Disclaimer

As mentioned before, these are experiments we do to see if there are ways to improve Plone's overall performance. It doesn't necessarily mean that we suddenly switch template languages. But if we do, we want to have some hard numbers first, to make sure that it's worth it — i.e. a 20-50% total speedup isn't worth switching for — whereas a 200-400% speedup might be.

-- Helge Tesdal