Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-10687

"Empty trash" always fail on huge number of objects : potential PHP_TIME_LIMIT or InnoDB deadlocks problems *can* be circumvented



    • Type: Bug
    • Status: Closed
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 3.8.7, 4.0.7, 4.1.4, 4.2.0
    • Fix Version/s: 4.1.5, 4.2.1, 4.3.0alpha1
    • Labels:
    • Environment:

      Operating System: Linux
      PHP Version: 4.4.6-0.dotdeb.2
      Database and version: MySQL 5.0.32-Dotdeb_1.dotdeb.1-log
      Browser (and version): ALL


      When facing a huge number of objects in the trash ( > 10 k), it might not be possible to delete anything at all from the trash when using MySQL InnoDB or any other transactionnal database.

      Cause of the problem : the $db->begin() and $db->commit() surround the whole loop on the total objects to purge.
      Either the script will hit the PHP_TIME_LIMIT ( on a 10k+ objects purge and a 20k+ records in a content object tree ) without $db->commit() being issued : the whole first part of the purge is rollbacked internally in MySQL
      Or the script will be the chosen victim of a InnoDB deadlock situation (hence an eZ Fatal Error without the commit being issued to the database)

      Proposed enhancement : as the "empty trash" is a definitive destructive operation chosen by the user, one can reduce the atomicity of the operation by "batching" the content objects purge :

      kernel/content/trash.php around line 92

      Index: D:/workspace/ezpublish_trunk/ezpublish_trunk_3.8/kernel/content/trash.php
      --- D:/workspace/ezpublish_trunk/ezpublish_trunk_3.8/kernel/content/trash.php	(revision 18795)
      +++ D:/workspace/ezpublish_trunk/ezpublish_trunk_3.8/kernel/content/trash.php	(working copy)
      @@ -95,6 +95,13 @@
                                                    true );
      +        // FIX : purge (at least a part) of a huge number (1k+) of objects
      +        $useFakeBatchMode = false; $fakeBatchModeLimit = 25; $fakeBatchModeCounter = 0;
      +        if (count($objectList) > 25) 
      +        {
      +            $useFakeBatchMode = true;
      +        }
      +        // EOF FIX : purge (at least a part) of a huge number (1k+) of objects
               $db =& eZDB::instance();
               foreach ( array_keys( $objectList ) as $key )
      @@ -101,6 +108,17 @@
                   $object =& $objectList[$key];
      +            // FIX : purge (at least a part) of a huge number (1k+) of objects
      +            if ($useFakeBatchMode == true)
      +            {
      +                $fakeBatchModeCounter++;
      +                if ($fakeBatchModeCounter > $fakeBatchModeLimit)
      +                {
      +                    $db->commit();
      +                    $db->begin();
      +                }
      +            }
      +            // EOF FIX : purge (at least a part) of a huge number (1k+) of objects 

      ( Not proposed as file attachement because it is a "rough" patch : one could wish to change the object count threshold - here 25 - according to its web server/database server architecture )

      This enhancement also diminish the period of time during which InnoDB will maintain the locks on the impacted rows/tables (hence, limiting the number of deadlocks related eZ Fatal Errors on other pages calls).
      If the script dies anyway, at least some of the trash has been emptied ( I was able to empty a 15k+ objects trash in four steps ).


          Issue Links



              gl (inactive) Gunnstein Lye
              rba_ic Richard Bayet
              0 Vote for this issue
              0 Start watching this issue