Whilst updating some core code for our CMS (Osiris) recently I came across an interesting tidbit that I thought was worth sharing here. After updating some code and cleaning up and migrating it to the new Factory/Object interface a particular part of the admin interface slowed down incredibly. Now given that a lot of the updates were supposed to speed up the CMS this was worrying, all is well now and the answer was discovered, however, I thought that sharing some more detail was required.
When thinking about the solution and what to write here, I suddenly remembered the puzzles I used to do while on holiday. In particular, the Logic Puzzles first introduced to the world by Charles Lutwidge Dodgson. Why? Well, I remembered that once you got past the easy answers (Fred owns the Goldfish) most of the solution were actually what could be deduced from what wasn’t true (the pigeon is grey, means that any reference to the grey pet means that actually, it has nothing to do with Fred).
In our case, the slow down was actually caused by what happened when something wasn’t true (or actually when something wasn’t found). In particular, Osiris has a system of Site properties, these are Domain-based (meaning you can override them on a per domain basis) with a Global or Default setting to fall back on.
The new system works by checking to see if the property is in memory if it’s not it checks a memory cache (APC/Memcached depending on configuration), if not it checks the Database. It can potentially run through this process twice (once for the current domain and then again for the Default setting if not present in the current domain). All of this means that the system is (or rather was) optimised to bring back the data in the quickest way possible. However, it didn’t take the option of what to do if it couldn’t find it far enough.
What this means is that in a loop situation when checking for a property that doesn’t exist it goes through that process every time (meaning it checks memory/Cache/DB before returning a not found message), if the property didn’t exist in both the local domain and the Global settings, it would do it for both for as long as the loop ran.
In fact, the entirety of the new Factory/Object methodology suffered from this issue, it just so happens that one of the few places it really affects radically is the properties system, which highlighted it for us.
And the solution? Well in the core Factory system I added a NotFound Cache, so for every factory (Controller), whenever we go looking for an object by its key, if we don’t find it after going through all of the processes the first time we store it in the factory’s NotFound Cache and the next time we look for it, if it’s present there (in memory), instead of checking APC/Memcache and then the DB we immediately return a not found. The effects!! Well a really badly performing page, taking 7 seconds to load went down to loading in 0.5 seconds
So if you are suffering from slowdowns in your code and you have optimised bringing back existing bits of data, then start considering what happens in your code for legitimately non-existant pieces of data