Details

      Description

      The ezfindindexsubtree will roun out of memory if the data being processed is large enough. the exact data is not important, it can be many small objects, or less, but large objects. PHP memory resource is all consumed, and an error shows:

      Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 71 bytes) in /kernel/classes/XXXX.php on line YYY           
       
      Fatal error: eZ Publish did not finish its request
      The execution of eZ Publish was abruptly ended, the debug output is present below.
      

      message modified to XXX.php to avoid referring to a specific file, since it's not important, the actual break will be in a random, depending when the last available bytes are consumed.

      test:
      modify the ezfindex to output the memory usage with each object processed.

      • create some folder with 100 sub objects (enough for to commit cycles inside the script)
      • hide/reveal any subtree
      • run the cronjob (first aneable the ezfindindex cron part in settings)
      • note how the memory always increases for each object.

      additional problem:
      when the script breaks, the subtrees already processed are not removed from the pending actions. so on next time they will re-run again, and the cronjob will never work.

      test:

      • run the previous steps
      • break the script before it fails/terminates
      • run the script again
      • note how it starts processing the same pending actions(object ids)

        Issue Links

          Activity

          Hide
          Yannick Roger (Inactive) added a comment -

          Could this be related: EZP-20952

          Show
          Yannick Roger (Inactive) added a comment - Could this be related: EZP-20952
          Hide
          Paulo Bras (Inactive) added a comment -

          yes, that patch makes a good difference.
          test results:
          run without patch
          Commiting.....
          \ #11256(RAM: 9101376)
          Indexing object IDs:
          Commiting.....
          \ #11306(RAM: 10895288)
          Indexing object IDs:
          Commiting.....
          \ #11356(RAM: 12631556)

          run with patch
          Commiting.....
          \ #11256(RAM: 8324752)
          Indexing object IDs:
          Commiting.....
          \ #11306(RAM: 9344868 )
          Indexing object IDs:
          Commiting.....
          \ #11356(RAM: 10342788)

          so, for same objects run, the leak is much less. this is only a batch of 200 articles, the memory saved should be much more with big objects made up of lots of attributes.

          Show
          Paulo Bras (Inactive) added a comment - yes, that patch makes a good difference. test results: run without patch Commiting..... \ #11256(RAM: 9101376) Indexing object IDs: Commiting..... \ #11306(RAM: 10895288) Indexing object IDs: Commiting..... \ #11356(RAM: 12631556) run with patch Commiting..... \ #11256(RAM: 8324752) Indexing object IDs: Commiting..... \ #11306(RAM: 9344868 ) Indexing object IDs: Commiting..... \ #11356(RAM: 10342788) so, for same objects run, the leak is much less. this is only a batch of 200 articles, the memory saved should be much more with big objects made up of lots of attributes.
          Hide
          Yannick Roger (Inactive) added a comment -

          Closing this issue as duplicate as EZP-20952 fixes it.

          Show
          Yannick Roger (Inactive) added a comment - Closing this issue as duplicate as EZP-20952 fixes it.

            People

            • Assignee:
              Unassigned
              Reporter:
              Paulo Bras (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: