#014797: Optimize common.php and other commonly used functions

Description:
  • common.php is used by compiled templates, and it could be beneficial to optimize the code in there if called often.
  • global instead of EZ_TEMPLATE_COMPILER_COMMON_CODE maybe (to use isset instead of defined)
  • Likewise look into most used functions in ezp and optimize them (trace with xdebug?)
  • Generally optimize foreach( array_key( $somevar ) as $key ) and array_key_exists throughout the code.

- Attachments

No attachments for this issue.


- Comments

And perhaps replace is_null( $foobar ) by $foobar === null ?

#261313 by Damien POBEL on April 20th, 2009 [Permanent Link]

And perhaps check the use of preg_* functions. On the homepage of an admin interface (with already generated caches), I have about 96 calls to a preg_* function.

Looking quickly at the code, I think that some preg_* functions calls can be improved or replaced, for instance:

  • in access.php:321, preg_replace is used to replace / by \/, str_replace would far more efficient
  • in eZDir::cleanPath() (called 67 times on the cached homepage of my admin), preg_replace is used to replace double (or more) directory separators by one, I think that a test to check if a double separator is there with strpos() could avoid a lot of preg_replace call and would be more efficient (having a double separator or more seems to be a quite rare case).
#261314 by Damien POBEL on April 21st, 2009 [Permanent Link]

What do you use for counting preg calls?

#261321 by André R. on April 21st, 2009 [Permanent Link]

I generate a profile file with XDebug and then I use Kcachegrind to analyse it (see attached screenshot).

#261327 by Damien POBEL on April 21st, 2009 [Permanent Link]
- Attachments: » kcachegrind.png

Attached patch for access.php and ezdir.php.
Avoid ~120 preg_replace function call on the homepage of the admin interface and ~400 after clearing all caches.

#261445 by Damien POBEL on April 30th, 2009 [Permanent Link]
- Attachments: » preg_opt.diff

Anybody knows the reason for usage of the 'foreach( array_key( $somevar ) as $key )' construct btw?

#261396 by Gaetano Giunta on April 26th, 2009 [Permanent Link]

No, only thing I can think about is that php probably didn't support $key => $value many years ago (PHP 3.x probably) and some of the early developers started with that version.

As for array_key_exists, thats another story, as it isn't the same as isset. So think we should leave those as is, as the performance difference is close to zero.

#261397 by André R. on April 26th, 2009 [Permanent Link]

"As for array_key_exists, thats another story, as it isn't the same as isset. So think we should leave those as is, as the performance difference is close to zero."
I disagree with that. When used for the same purpose, isset() is more than 10 times quicker (tested with the following code).
In compile templates, it's difficult to find if array_key_exists can be replaced by isset() (I think that most of the time it's not possible because null is a regular value in templates) but there are other part of the code where it's quite trivial like in autoload.php for instance.


<?php
$t = array();
$start = microtime( true );
for( $i = 0; $i != 1000; $i++ )
{
    if ( isset( $t[$i] ) )
    //if ( array_key_exists( $i, $t ) )
    {

    }
}
$end = microtime( true );
$total = $end - $start;
echo $total . "\n";
#261399 by Damien POBEL on April 26th, 2009 [Permanent Link]

Attached patch for autoload.php. Replace array_key_exists and is_null by isset and "=== null".

#261446 by Damien POBEL on April 30th, 2009 [Permanent Link]
- Attachments: » array_key_exists_is_null_autoload.diff

Thanks Damien, and let me rephrase my "we should leave those as is" sentence to: "lets only change it the places we know for sure isset can be used instead"..
Like this one ;)

#261451 by André R. on May 1st, 2009 [Permanent Link]

I think that foreach( array_keys() ) was interesting in PHP4 when looping in an array of objects as it avoids the copy of each element (very slow) and perhaps a problem related to reference.

#261398 by Damien POBEL on April 26th, 2009 [Permanent Link]

This is an interesting issue, that I'd like to bench one day - php has used copy-on-write (aka ref. counting) for as long as I remember.
Lots of people claim that using foreach will double the amount of mem used, as well as be dead slow because of copying, but I think it does not copy anything at all - unless you later modify the object that you got within the loop - and limits itself to a little bookkeeping, so the slowdown shpumd not be so big after all...

#261400 by Gaetano Giunta on April 27th, 2009 [Permanent Link]

Thinking more about it: it must in fact have been used in cases where what was needed to be modified later was the object within the array. In this case it makes effectively little sense to get a handle to a copy of an array value to just discard it and get a new handle to it by reference. Perf/mem tradeoffs might apply, depending on size of array...

#261401 by Gaetano Giunta on April 27th, 2009 [Permanent Link]

...and in php5 you can use value-by-ref on foreach loops, so it still makes sense to carry out this micro-optimization (ie. removing one array_keys call)

#261407 by Gaetano Giunta on April 27th, 2009 [Permanent Link]

From Derick:
"I think that foreach( array_keys() ) was interesting in PHP4 when
looping in an array of objects as it avoids the copy of each element
(very slow) and perhaps a problem related to reference."

"PHP has copy-on write mechanisms. Not
even in PHP 4 this would cause any performance hits (besides the
initializing of two vars (key and value) instead of just key)."

#261405 by André R. on April 27th, 2009 [Permanent Link]

Look into using mt_rand() instead of rand().

ref php doc:
"It uses a random number generator with known characteristics using the » Mersenne Twister, which will produce random numbers four times faster than what the average libc rand() provides."

#261414 by André R. on April 27th, 2009 [Permanent Link]

Implemented in
trunk (4.2.0Alpha1) Rev. 23763

#262116 by André R. on June 30th, 2009 [Permanent Link]

So far implemented in trunk
Optimize eZINI::instance and eZINI->variable in Rev. 23519
Change / avoid preg_replace calls ezdir and access.php in Rev. 23520
array_key_exists -> isset autoload.php in Rev. 23521
is_null -> === null in Rev. 23522
Removed 2 redundant eZINI::instance calls from index.php in Rev. 23523
array_key_exists -> isset ezdebug.php in Rev. 23524
Make eZINI->assign use internal data instead of public api in Rev. 23525
Make eZINI->variableArray use internal data instead of public api in Rev. 23526

#261482 by André R. on May 6th, 2009 [Permanent Link]

All isset parts of the patches where reverted / removed after internal discussions, reason being code readability and the fact that it doesn't affect eZ Publish performance.

autoload.php reverted in Rev. 23532
ezdebug.php reverted in Rev. 23533
ezini.php reverted in Rev. 23534

#261508 by André R. on May 8th, 2009 [Permanent Link]

Proposed pattern to reduce calls to eZINI::instance and eZDir::path


Index: index.php
===================================================================
--- index.php    (revision 23526)
+++ index.php    (working copy)
@@ -152,7 +152,7 @@
 list( $iniFilePermission, $iniDirPermission ) =
     $ini->variableMulti( 'FileSettings', array( 'StorageFilePermissions', 'StorageDirPermissions' ) );
 
-$iniVarDirectory = eZSys::cacheDirectory() ;
+$iniVarDirectory = eZSys::cacheDirectory( false ) ;
 
 // OPTIMIZATION:
 // Sets permission array as global variable, this avoids the eZCodePage include
Index: lib/ezutils/classes/ezsys.php
===================================================================
--- lib/ezutils/classes/ezsys.php    (revision 23526)
+++ lib/ezutils/classes/ezsys.php    (working copy)
@@ -453,19 +453,31 @@
      \return the directory used for storing cache files.
      \note This will include the varDirectory().
     */
-    static function cacheDirectory()
+    static function cacheDirectory( $allowCachingPath = true )
     {
-        $ini = eZINI::instance();
+        static $staticCacheDir = null;
+
+        if ( $staticCacheDir !== null && $allowCachingPath === true  )
+        {
+            return $staticCacheDir;
+        }
+
+        $ini = eZINI::instance();
         $cacheDir = $ini->variable( 'FileSettings', 'CacheDir' );
 
-        if ( $cacheDir[0] == "/" )
+        if ( $cacheDir[0] === '/' )
         {
-            return eZDir::path( array( $cacheDir ) );
+            $cacheDir = eZDir::path( array( $cacheDir ) );
         }
         else
         {
-            return eZDir::path( array( eZSys::varDirectory(), $cacheDir ) );
+            $cacheDir = eZDir::path( array( eZSys::varDirectory(), $cacheDir ) );
         }
+        if ( $allowCachingPath === true  )
+        {
+            $staticCacheDir = $cacheDir;
+        }
+        return $cacheDir;
     }
 
     /*!

#261483 by André R. on May 7th, 2009 [Permanent Link]

Attached patch avoids about 10000 (!) preg_match calls after clearing all caches.

#261527 by Damien POBEL on May 9th, 2009 [Permanent Link]
- Attachments: » template_parser.diff

Implemented in
trunk (4.2.0Alpha1) Rev. 23762

#262115 by André R. on June 30th, 2009 [Permanent Link]

Optimization for eZPersistentObject classes ::definition() calls.

#261536 by André R. on May 11th, 2009 [Permanent Link]
- Attachments: » definition.diff

Implemented in
trunk (4.2.0Alpha1) Rev. 23761

#262114 by André R. on June 30th, 2009 [Permanent Link]

In addition, after clearing all caches, it seems that there are a lots of calls to eZDir::recursiveFindRelative() (so a lot of opendir/readdir/preg_match calls) because the complete list of templates through all designs and all extensions is build several times during the execution of the page. With a quick and dirty in memory cache in eZTemplateDesignResource::overrideArray(), the home page of my back office takes 12 seconds to be generated after a clearing all caches, without it, it takes about 18 seconds !

#261588 by Damien POBEL on May 13th, 2009 [Permanent Link]

Confirmed, went from ~35 seconds to 6-7 seconds with attached patch.

EDIT: Windows notebook..

#261606 by André R. on May 14th, 2009 [Permanent Link]
- Attachments: » override_array_cache.diff

A bit more sophisticated patch, witch makes sure the in memory cache is cleared when you clear override cache as well.

#261610 by André R. on May 14th, 2009 [Permanent Link]
- Attachments: » override_array_cache2.diff

Implemented in
trunk (4.2.0Alpha1) Rev. 23612

With phpdoc instead of doxygen doc.

#261725 by André R. on May 26th, 2009 [Permanent Link]

Small stuff in common.php, should also look into the merging the function_exists calls or remove them (the compiled templates both check presence of constant and use include_once so shouldn't need them), and benchmark defined() vs other means of knowing that file is loaded (global for instance), or just move these functions to a class that is loaded anyway and call them statically.


Index: eztemplatecompiler.php
--- eztemplatecompiler.php Base (BASE)
+++ eztemplatecompiler.php Locally Modified (Based On LOCAL)
@@ -1932,9 +1932,7 @@
 $lbracket
     function compiledFetchVariable( \$vars, \$namespace, \$name )
     $lbracket
-        \$exists = ( array_key_exists( \$namespace, \$vars ) and
-                    array_key_exists( \$name, \$vars[\$namespace] ) );
-        if ( \$exists )
+        if ( isset( \$vars[\$namespace][\$name] ) )
         $lbracket
             return \$vars[\$namespace][\$name];
         $rbracket
@@ -1970,8 +1968,8 @@
     $lbracket
         if ( is_object( \$value ) )
         $lbracket
-            if ( method_exists( \$value, \"attribute\" ) and
-                 method_exists( \$value, \"hasattribute\" ) )
+            if ( method_exists( \$value, 'attribute' ) and
+                 method_exists( \$value, 'hasattribute' ) )
             $lbracket
                 if ( \$value->hasAttribute( \$attributeValue ) )
                 $lbracket
#262127 by André R. on June 30th, 2009 [Permanent Link]

I just tried it and I am afraid this patch has no effect.

#262190 by Jérôme Renard on July 3rd, 2009 [Permanent Link]

Implemented in
trunk (4.2.0Alpha1) see above for the various revisions.

#263578 by André R. on October 16th, 2009 [Permanent Link]

- History
Properties
Type Enhancement
Priority Medium
Component Misc
Affects 4.1.0
Fix Version 4.2.0alpha1
Reporter André R.
Responsible André R.
Status 0 Code Review
Resolution Implemented
Created April 20th, 2009
Updated October 16th, 2009
Resolved October 16th, 2009
 
Navigation [Permanent Link]
Previous Issue
Back to Issues List
Next Issue: #016124
  eZContentObjectTreeNode::createPermissionCheckingSQL() should define an index in temporary table