Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-17277

ezcontentcache.php : prevent memory hogging and allow handling of huge (thousand) subtrees

    XMLWordPrintable

Details

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Medium Medium
    • 4.5.0alpha
    • 4.3.0
    • Caching, Cronjobs
    • None
    • Operating System: Debian Lenny
      PHP Version: CLI PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2
      Database and version: mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (i486) using readline 5.2
      Browser (and version):

    Description

      Hi,

      The problem is that ezcontentcache.php can't be used on a huge (4000+ objects) subtree with a reasonnable PHP memory_limit (256M).
      => When using the "clear-subtree" option, $eZContentObjectContentObjectCache is not purged (when looping on ContentObjectIDs), hence the memory hogging when working on big objects and huge subtrees.

      For instance :
      subtree with 4307 objects (only main_node)

      Native ezcontentcache.php :
      Peak memory usage: 819,152.7734 KB
      Mysql_queries: 47152 queries
      ( Total script time: 118.5349 sec )

      Custom ezcontentcache.php with rough patch included :
      Peak memory usage: 10,199.5117 KB
      Mysql_queries: 46996 queries
      ( Total script time: 65.7508 sec )

      My example is a run on a subtree of content main nodes, so of course on a "regular" content tree with multipositioned content objects, purging $eZContentObjectContentObjectCache would mean more queries to the database.

      Suggestion :

      • provide an option to preserve memory usage (over mysql queries) which would follow such a behaviour
        ( - do the same on the "clear-node" option ? )
      • provide an option to override the fetch limit of nodes ($limit)

      Context: I'm currently working on a project with daily ezpublish content import (add, update, delete) of huge objects (50+ attributes).
      I decided to use my own publish operation (disabling content indexing, content cache purge and cache generation) in my import cronjob and opted for a manual purge of content cache via script bin/php/ezcontentcache.php.

      Attachments

        Activity

          People

            andre1 andre1
            rba_ic rba_ic
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: