One of our customers were experiencing sluggishness and slowdowns when creating content in their Plone 2.5-based intranet. They asked us for help. We really like improving Plone performance,  so we gave it a shot.

3 of the 10 developers attending the upcoming performance sprint in Copenhagen are from Jarn.  We'll try to integrate as many as possible of the improvements found in this sessions into Plone itself at the sprint.

If you just want the results and not the dirty details about Plone innards, skip to the last paragraph.

Profiling

Profiling was mainly done using PTProfiler, ZopeProfiler and the profilehooks. When viewing the add form, ZopeProfiler indicated that a lot of the roughly 15 seconds (!) taken to create an object was spent in Portal Factory.

Pinpoint profiling with profilehooks indicated that the __call__ method of FactoryTool took 10.7 seconds. The __call__ method is responsible for creating the temporary object and rendering the edit form on top, which includes some template rendering.

Speeding up Portal Factory

Given that the Factory tool was responsible for such a large part of the time spent, we focused our efforts there: 10.7 of the 15 seconds total.

Superflous indexing

The temporary object created by Portal Factory would be indexed in the catalog and immediately unindexed again. The improvement idea was to inject a fake archetype_tool which would proxy the real archetype_tool for all methods except the one used to list the catalogs. This is exactly as ugly as it sounds, and fits very well with the rest of the factory tool ;-) .
When implemented, tests showed that time spent in factory tool was down from 10.7 to approximately 5 seconds.

listTypeInfo

listTypeInfo was used several times, and was rather slow, as the site has a lot of additional content types installed. For each content type listed, a permission check would be performed to see if the user has permission to add the content. This permission check involves looking up roles and local roles, and is potentially expensive. The strategy would be to gather all roles in a context (once instead of once for each content type), then compare it to the roles required for creating each type. When implemented, testing indicated that the total time spent (originally 10.7 seconds) went further down to approximately 2 seconds.

_getTempFolder

_getTempFolder in FactoryTool copies all security settings (the ones visible in manage_access) to the TempFolder (created on each request). This makes sure all permission restrictions are always correct, but takes extra processing time. By removing these checks a user could technically, (by manual URL mangling) get access to add forms for content types she is not allowed to create —  but it will still not be possible to create the content. (Am I missing something here?). When omitting these security settings, the total time in Portal Factory was down to 1.7 seconds. (These can not just be removed in Plone, but we can probably find a better way of doing this soon)

Portal factory totals

With these improvements, we moved from 10.7 seconds  to 1.7 seconds spent in Portal Factory, and from 15 seconds to 6 seconds total!

Additional speedups — outside Portal Factory

On profiling content creation (folder in particular), it turned out that getDefaultAddableTypes of constraintypes was called ~10 times(!), always on the same piece of content, and always yielding the same result (vocabulary methods), taking a total of 1.15 seconds. When caching this (and listTypeInfo) time spent in getDefaultAddableTypes was down to 0.24 seconds.

We also added caching to methods like __ac_local_roles__ in the temporary folder to avoid it being computed each time a security check is done.

End result

In total, page creation went from 15 seconds to less than 5 seconds, and displaying the edit forms and browsing the site was also faster. There are still more improvements that can be done, but this was a good result, especially considering the time spent.

--
Helge Tesdal