Redundant reindexing

While creating a new object using the factory tool, the new object would be reindexed four times.

1. ObjectAddedEvent event handler
2. Finish construction in factory method
3. processForm in Archetypes.BaseObject
4. Rename after creation

This is 3 times more than necessary. The first two are called by factory_tool.doCreate, and the two last are in processForm.

Avoiding reindexing

All the reindexing was happening in methods called by content_edit_impl, and all reindexing went through CatalogMultiplex. In experimental.contentcreation we already take advantage of the fact that CatalogMultiplex gets a list of all catalogs to index in, and by returning an empty list, the object is not indexed. The strategy would be to let content_edit_impl change the result of the catalog listing while the reindexing method was called, then finally do 1 reindex afterwards.

First of all we needed to override the content_edit_impl script. In Plone, it is implemented as a skin script, which enables easy customization. As we needed trusted code, we added it to the patch in experimental.contentcreation and set it as an attribute to the Portal class, which prevents the use of the script in portal_skins.

Before calling doCreate and processForm from content_edit_impl, we create a lambda function that returns an empty list and set the getCatalogsByType method of the Archetypes tool object to the newly created lambda function. In addition, we set _p_changed to False on the Archetypes tool to avoid the transaction machinery trying to save the function during savepoint.

After doCreate and processForm, we delete the function object and reset _p_changed again, before doing a single and final reindexObject.

The result

No performance blog posting without a graph. We tested the setup with JMeter on a laptop, 10 loops for each scenario. The blue bar is content creation with experimental.contentcreation with 1 reindex.

Our tests show that time went down from 567ms in regular Plone to 203ms with experimental.contentcreation and the redundant reindexing removal. experimental.contentcreation without reindexing removal was 284ms.

If you have converters to handle uploaded files and more fields and indexes in the catalog, the time difference will be even bigger.

Future

We also created a plone.app.content based content type with formlib forms to compare response times compared to regular Plone. While not directly comparable and not feature complete, the numbers give an indication of what to expect in the future, and why people are looking into Zope 3 technology.

Try it out for yourself

To try it out for yourself, you can check out experimental.contentcreation from SVN, add it to your buildout (don't forget the ZCML slug) and restart Zope.

-- Helge Tesdal