Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-25325

Out-of-memory on Solr Search Engine Bundle solr_create_index

    Details

      Description

      Steps to reproduce:

      1. Prepare a test eZ Publish 5.4 instalation, fully updated to 5.4.5;
      2. Install Solr Search Engine Bundle, as described here:

      https://doc.ez.no/display/EZP/Solr+Search+Engine+Bundle

      3. Following step 6 of the above documentation, run initial indexation. On websites with large databases, indexation will fail due to lack of memory:

      $ php ezpublish/console ezplatform:solr_create_index -vvv
      Indexing Content...
       17406/60392 [========>-------------------]  28% Elapsed: 15 minsPHP Fatal error:  Allowed memory size of 2147483648 bytes exhausted (tried to allocate 262209 bytes) in /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php on line 302
      PHP Stack trace:
      PHP   1. {main}() /home/noliveira/dev/backend/ezpublish/ezpublish/console:0
      PHP   2. Symfony\Component\Console\Application->run() /home/noliveira/dev/backend/ezpublish/ezpublish/console:30
      PHP   3. eZ\Bundle\EzPublishCoreBundle\Console\Application->doRun() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:120
      PHP   4. Symfony\Bundle\FrameworkBundle\Console\Application->doRun() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Bundle/EzPublishCoreBundle/Console/Application.php:41
      PHP   5. Symfony\Component\Console\Application->doRun() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96
      PHP   6. Symfony\Component\Console\Application->doRunCommand() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:189
      PHP   7. Symfony\Component\Console\Command\Command->run() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:846
      PHP   8. EzSystems\EzPlatformSolrSearchEngineBundle\Command\SolrCreateIndexCommand->execute() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:256
      PHP   9. eZ\Publish\Core\Persistence\Cache\ContentHandler->load() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezplatform-solr-search-engine/bundle/Command/SolrCreateIndexCommand.php:92
      PHP  10. eZPublishCorePersistenceLegacyContentHandler_00000000088f87b70000000073d480098f23d777407472c18f6db08da97f09ef->load() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Cache/ContentHandler.php:71
      PHP  11. eZ\Publish\Core\Persistence\Legacy\Content\Handler->load() /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:49049
      PHP  12. eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractContentFromRows() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Handler.php:322
      PHP  13. eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldFromRow() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:208
      PHP  14. eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldValueFromRow() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:379
      PHP  15. eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->toFieldValue() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:416
      PHP  16. eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->parseLegacyXml() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:168
      PHP  17. eZPublishCoreFieldTypeImageIOLegacy_00000000088f8b4e0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:199
      PHP  18. eZ\Publish\Core\FieldType\Image\IO\Legacy->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53269
      PHP  19. eZPublishCoreIOTolerantIOService_00000000088f95fb0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/FieldType/Image/IO/Legacy.php:148
      PHP  20. eZ\Publish\Core\IO\TolerantIOService->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53599
      PHP  21. eZ\Publish\Core\IO\TolerantIOService->loadBinaryFile() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:100
      PHP  22. eZ\Publish\Core\IO\IOService->buildDomainBinaryFileObject() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:93
      PHP  23. eZ\Publish\Core\IO\IOMetadataHandler\Flysystem->getMimeType() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOService.php:350
      PHP  24. League\Flysystem\Filesystem->getMimetype() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOMetadataHandler/Flysystem.php:70
      PHP  25. League\Flysystem\Adapter\Local->getMimetype() /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Filesystem.php:559
      PHP  26. finfo->file() /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php:302
       
       
       
        [Symfony\Component\Debug\Exception\OutOfMemoryException]
        Error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate
         262209 bytes)
       
       
       
      Exception trace:
       () at /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php:302
       finfo->file() at /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php:302
       League\Flysystem\Adapter\Local->getMimetype() at /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Filesystem.php:559
       League\Flysystem\Filesystem->getMimetype() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOMetadataHandler/Flysystem.php:70
       eZ\Publish\Core\IO\IOMetadataHandler\Flysystem->getMimeType() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOService.php:350
       eZ\Publish\Core\IO\IOService->buildDomainBinaryFileObject() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:93
       eZ\Publish\Core\IO\TolerantIOService->loadBinaryFile() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:100
       eZ\Publish\Core\IO\TolerantIOService->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53599
       eZPublishCoreIOTolerantIOService_00000000088f95fb0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/FieldType/Image/IO/Legacy.php:148
       eZ\Publish\Core\FieldType\Image\IO\Legacy->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53269
       eZPublishCoreFieldTypeImageIOLegacy_00000000088f8b4e0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:199
       eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->parseLegacyXml() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:168
       eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->toFieldValue() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:416
       eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldValueFromRow() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:379
       eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldFromRow() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:208
       eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractContentFromRows() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Handler.php:322
       eZ\Publish\Core\Persistence\Legacy\Content\Handler->load() at /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:49049
       eZPublishCorePersistenceLegacyContentHandler_00000000088f87b70000000073d480098f23d777407472c18f6db08da97f09ef->load() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Cache/ContentHandler.php:71
       eZ\Publish\Core\Persistence\Cache\ContentHandler->load() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezplatform-solr-search-engine/bundle/Command/SolrCreateIndexCommand.php:92
       EzSystems\EzPlatformSolrSearchEngineBundle\Command\SolrCreateIndexCommand->execute() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:256
       Symfony\Component\Console\Command\Command->run() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:846
       Symfony\Component\Console\Application->doRunCommand() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:189
       Symfony\Component\Console\Application->doRun() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96
       Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Bundle/EzPublishCoreBundle/Console/Application.php:41
       eZ\Bundle\EzPublishCoreBundle\Console\Application->doRun() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:120
       Symfony\Component\Console\Application->run() at /home/noliveira/dev/backend/ezpublish/ezpublish/console:30
       {main}() at /home/noliveira/dev/backend/ezpublish/ezpublish/console:0
       
       
      ezplatform:solr_create_index [<bulk_count>]
      

        Issue Links

          Activity

          Hide
          Hannes Giesenow added a comment -

          I don't want a warning, i want a working solution. In my opinion, the warning from stash should be considered and correctly be implemented inside the script.

          Show
          Hannes Giesenow added a comment - I don't want a warning, i want a working solution. In my opinion, the warning from stash should be considered and correctly be implemented inside the script.
          Hide
          Nuno Oliveira (Inactive) added a comment -

          Dear Hannes Giesenow,

          You can do it either way, whichever suits your operations best. But it is a know fact that inMemory caching can have a big impact on long running scripts. We warn users about it here:

          https://doc.ez.no/display/EZP/Persistence+cache+configuration

          Note for "inMemory" cache with long running scripts

          Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen!

          You can see a similar warning on the StashBundle own page, in the "In-Memory" section:

          http://www.stashphp.com/Symfony.html

          By default, every cache service includes in-memory caching: during the lifetime of a single request, any values stored or retrieved from the cache service will be stored in memory, with the in-memory representation being checked before any other drivers. In some circumstances, however (such as long-running CLI batch scripts) this may not be desirable. In those cases, the in-memory driver can be disabled:

          stash:
          drivers: [ Apc ]
          inMemory: false
          Apc: ~

          Show
          Nuno Oliveira (Inactive) added a comment - Dear Hannes Giesenow, You can do it either way, whichever suits your operations best. But it is a know fact that inMemory caching can have a big impact on long running scripts. We warn users about it here: https://doc.ez.no/display/EZP/Persistence+cache+configuration Note for "inMemory" cache with long running scripts Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen! You can see a similar warning on the StashBundle own page, in the "In-Memory" section: http://www.stashphp.com/Symfony.html By default, every cache service includes in-memory caching: during the lifetime of a single request, any values stored or retrieved from the cache service will be stored in memory, with the in-memory representation being checked before any other drivers. In some circumstances, however (such as long-running CLI batch scripts) this may not be desirable. In those cases, the in-memory driver can be disabled: stash: drivers: [ Apc ] inMemory: false Apc: ~
          Hide
          Hannes Giesenow added a comment -

          So what is the suggested solution to solve the issue? Disabling the inMemory cache at all? Or changing the config every time the indexing runs?
          Wouldn't it be the best solution, when the script disables the inMemory cache by itself?

          Show
          Hannes Giesenow added a comment - So what is the suggested solution to solve the issue? Disabling the inMemory cache at all? Or changing the config every time the indexing runs? Wouldn't it be the best solution, when the script disables the inMemory cache by itself?
          Hide
          Yannick Roger (Inactive) added a comment -

          Please reopen if need feedback is provided.

          Show
          Yannick Roger (Inactive) added a comment - Please reopen if need feedback is provided.
          Hide
          Joao Inacio (Inactive) added a comment -

          After investigating this it was found that there is no problem on the SolrSearchEngine indexing, but rather on the configuration and environment of long-running scripts.

          The major culprits of growing memory usage are:

          • the usage of dev environment, in particular with stash tracking
          • the usage of stash inMemory cache

          There is a documentation notice regarding the persistence cache at https://doc.ez.no/display/EZP/Persistence+cache+configuration:

          Note for "inMemory" cache with long running scripts
          Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen!

          In addition to the above, the flysystem IOMetadata handler does some caching as well, but the amount of memory used shouldn't have a big impact.

          Given the above, this ticket could be closed as "won't fix".

          Show
          Joao Inacio (Inactive) added a comment - After investigating this it was found that there is no problem on the SolrSearchEngine indexing, but rather on the configuration and environment of long-running scripts. The major culprits of growing memory usage are: the usage of dev environment, in particular with stash tracking the usage of stash inMemory cache There is a documentation notice regarding the persistence cache at https://doc.ez.no/display/EZP/Persistence+cache+configuration: Note for "inMemory" cache with long running scripts Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen! In addition to the above, the flysystem IOMetadata handler does some caching as well, but the amount of memory used shouldn't have a big impact. Given the above, this ticket could be closed as "won't fix".

            People

            • Assignee:
              Unassigned
              Reporter:
              Nuno Oliveira (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: