Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-25325

Out-of-memory on Solr Search Engine Bundle solr_create_index

        Details

          Description

          Steps to reproduce:

          1. Prepare a test eZ Publish 5.4 instalation, fully updated to 5.4.5;
          2. Install Solr Search Engine Bundle, as described here:

          https://doc.ez.no/display/EZP/Solr+Search+Engine+Bundle

          3. Following step 6 of the above documentation, run initial indexation. On websites with large databases, indexation will fail due to lack of memory:

          $ php ezpublish/console ezplatform:solr_create_index -vvv
          		 
          Indexing Content...
          		 
           17406/60392 [========>-------------------]  28% Elapsed: 15 minsPHP Fatal error:  Allowed memory size of 2147483648 bytes exhausted (tried to allocate 262209 bytes) in /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php on line 302
          		 
          PHP Stack trace:
          		 
          PHP   1. {main}() /home/noliveira/dev/backend/ezpublish/ezpublish/console:0
          		 
          PHP   2. Symfony\Component\Console\Application->run() /home/noliveira/dev/backend/ezpublish/ezpublish/console:30
          		 
          PHP   3. eZ\Bundle\EzPublishCoreBundle\Console\Application->doRun() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:120
          		 
          PHP   4. Symfony\Bundle\FrameworkBundle\Console\Application->doRun() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Bundle/EzPublishCoreBundle/Console/Application.php:41
          		 
          PHP   5. Symfony\Component\Console\Application->doRun() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96
          		 
          PHP   6. Symfony\Component\Console\Application->doRunCommand() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:189
          		 
          PHP   7. Symfony\Component\Console\Command\Command->run() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:846
          		 
          PHP   8. EzSystems\EzPlatformSolrSearchEngineBundle\Command\SolrCreateIndexCommand->execute() /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:256
          		 
          PHP   9. eZ\Publish\Core\Persistence\Cache\ContentHandler->load() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezplatform-solr-search-engine/bundle/Command/SolrCreateIndexCommand.php:92
          		 
          PHP  10. eZPublishCorePersistenceLegacyContentHandler_00000000088f87b70000000073d480098f23d777407472c18f6db08da97f09ef->load() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Cache/ContentHandler.php:71
          		 
          PHP  11. eZ\Publish\Core\Persistence\Legacy\Content\Handler->load() /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:49049
          		 
          PHP  12. eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractContentFromRows() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Handler.php:322
          		 
          PHP  13. eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldFromRow() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:208
          		 
          PHP  14. eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldValueFromRow() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:379
          		 
          PHP  15. eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->toFieldValue() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:416
          		 
          PHP  16. eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->parseLegacyXml() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:168
          		 
          PHP  17. eZPublishCoreFieldTypeImageIOLegacy_00000000088f8b4e0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:199
          		 
          PHP  18. eZ\Publish\Core\FieldType\Image\IO\Legacy->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53269
          		 
          PHP  19. eZPublishCoreIOTolerantIOService_00000000088f95fb0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/FieldType/Image/IO/Legacy.php:148
          		 
          PHP  20. eZ\Publish\Core\IO\TolerantIOService->loadBinaryFileByUri() /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53599
          		 
          PHP  21. eZ\Publish\Core\IO\TolerantIOService->loadBinaryFile() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:100
          		 
          PHP  22. eZ\Publish\Core\IO\IOService->buildDomainBinaryFileObject() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:93
          		 
          PHP  23. eZ\Publish\Core\IO\IOMetadataHandler\Flysystem->getMimeType() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOService.php:350
          		 
          PHP  24. League\Flysystem\Filesystem->getMimetype() /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOMetadataHandler/Flysystem.php:70
          		 
          PHP  25. League\Flysystem\Adapter\Local->getMimetype() /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Filesystem.php:559
          		 
          PHP  26. finfo->file() /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php:302
          		 
           
          		 
           
          		 
           
          		 
            [Symfony\Component\Debug\Exception\OutOfMemoryException]
          		 
            Error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate
          		 
             262209 bytes)
          		 
           
          		 
           
          		 
           
          		 
          Exception trace:
          		 
           () at /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php:302
          		 
           finfo->file() at /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Adapter/Local.php:302
          		 
           League\Flysystem\Adapter\Local->getMimetype() at /home/noliveira/dev/backend/ezpublish/vendor/league/flysystem/src/Filesystem.php:559
          		 
           League\Flysystem\Filesystem->getMimetype() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOMetadataHandler/Flysystem.php:70
          		 
           eZ\Publish\Core\IO\IOMetadataHandler\Flysystem->getMimeType() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/IOService.php:350
          		 
           eZ\Publish\Core\IO\IOService->buildDomainBinaryFileObject() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:93
          		 
           eZ\Publish\Core\IO\TolerantIOService->loadBinaryFile() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/IO/TolerantIOService.php:100
          		 
           eZ\Publish\Core\IO\TolerantIOService->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53599
          		 
           eZPublishCoreIOTolerantIOService_00000000088f95fb0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/FieldType/Image/IO/Legacy.php:148
          		 
           eZ\Publish\Core\FieldType\Image\IO\Legacy->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:53269
          		 
           eZPublishCoreFieldTypeImageIOLegacy_00000000088f8b4e0000000073d480098f23d777407472c18f6db08da97f09ef->loadBinaryFileByUri() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:199
          		 
           eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->parseLegacyXml() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/FieldValue/Converter/Image.php:168
          		 
           eZ\Publish\Core\Persistence\Legacy\Content\FieldValue\Converter\Image->toFieldValue() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:416
          		 
           eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldValueFromRow() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:379
          		 
           eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractFieldFromRow() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Mapper.php:208
          		 
           eZ\Publish\Core\Persistence\Legacy\Content\Mapper->extractContentFromRows() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Legacy/Content/Handler.php:322
          		 
           eZ\Publish\Core\Persistence\Legacy\Content\Handler->load() at /home/noliveira/dev/backend/ezpublish/ezpublish/cache/noliveira/ezpublishnoliveiraDebugProjectContainer.php:49049
          		 
           eZPublishCorePersistenceLegacyContentHandler_00000000088f87b70000000073d480098f23d777407472c18f6db08da97f09ef->load() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Persistence/Cache/ContentHandler.php:71
          		 
           eZ\Publish\Core\Persistence\Cache\ContentHandler->load() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezplatform-solr-search-engine/bundle/Command/SolrCreateIndexCommand.php:92
          		 
           EzSystems\EzPlatformSolrSearchEngineBundle\Command\SolrCreateIndexCommand->execute() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:256
          		 
           Symfony\Component\Console\Command\Command->run() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:846
          		 
           Symfony\Component\Console\Application->doRunCommand() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:189
          		 
           Symfony\Component\Console\Application->doRun() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96
          		 
           Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /home/noliveira/dev/backend/ezpublish/vendor/ezsystems/ezpublish-kernel/eZ/Bundle/EzPublishCoreBundle/Console/Application.php:41
          		 
           eZ\Bundle\EzPublishCoreBundle\Console\Application->doRun() at /home/noliveira/dev/backend/ezpublish/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:120
          		 
           Symfony\Component\Console\Application->run() at /home/noliveira/dev/backend/ezpublish/ezpublish/console:30
          		 
           {main}() at /home/noliveira/dev/backend/ezpublish/ezpublish/console:0
          		 
           
          		 
           
          		 
          ezplatform:solr_create_index [<bulk_count>]

            Issue Links

            relates to

            Bug - A problem which impairs or prevents the functions of the product. EZP-25538 Content creation via Public API causes memory leak

            • High - Major loss of functionality.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              Joao Inacio (Inactive) added a comment -

              After investigating this it was found that there is no problem on the SolrSearchEngine indexing, but rather on the configuration and environment of long-running scripts.

              The major culprits of growing memory usage are:

              • the usage of dev environment, in particular with stash tracking
              • the usage of stash inMemory cache

              There is a documentation notice regarding the persistence cache at https://doc.ez.no/display/EZP/Persistence+cache+configuration:

              Note for "inMemory" cache with long running scripts
              Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen!

              In addition to the above, the flysystem IOMetadata handler does some caching as well, but the amount of memory used shouldn't have a big impact.

              Given the above, this ticket could be closed as "won't fix".

              Show
              Joao Inacio (Inactive) added a comment - After investigating this it was found that there is no problem on the SolrSearchEngine indexing, but rather on the configuration and environment of long-running scripts. The major culprits of growing memory usage are: the usage of dev environment, in particular with stash tracking the usage of stash inMemory cache There is a documentation notice regarding the persistence cache at https://doc.ez.no/display/EZP/Persistence+cache+configuration: Note for "inMemory" cache with long running scripts Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen! In addition to the above, the flysystem IOMetadata handler does some caching as well, but the amount of memory used shouldn't have a big impact. Given the above, this ticket could be closed as "won't fix".
              Hide
              Permalink
              Yannick Roger (Inactive) added a comment -

              Please reopen if need feedback is provided.

              Show
              Yannick Roger (Inactive) added a comment - Please reopen if need feedback is provided.
              Hide
              Permalink
              Hannes Giesenow added a comment -

              So what is the suggested solution to solve the issue? Disabling the inMemory cache at all? Or changing the config every time the indexing runs?
              Wouldn't it be the best solution, when the script disables the inMemory cache by itself?

              Show
              Hannes Giesenow added a comment - So what is the suggested solution to solve the issue? Disabling the inMemory cache at all? Or changing the config every time the indexing runs? Wouldn't it be the best solution, when the script disables the inMemory cache by itself?
              Hide
              Permalink
              Nuno Oliveira (Inactive) added a comment -

              Dear Hannes Giesenow,

              You can do it either way, whichever suits your operations best. But it is a know fact that inMemory caching can have a big impact on long running scripts. We warn users about it here:

              https://doc.ez.no/display/EZP/Persistence+cache+configuration

              Note for "inMemory" cache with long running scripts

              Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen!

              You can see a similar warning on the StashBundle own page, in the "In-Memory" section:

              http://www.stashphp.com/Symfony.html

              By default, every cache service includes in-memory caching: during the lifetime of a single request, any values stored or retrieved from the cache service will be stored in memory, with the in-memory representation being checked before any other drivers. In some circumstances, however (such as long-running CLI batch scripts) this may not be desirable. In those cases, the in-memory driver can be disabled:

              stash:
              drivers: [ Apc ]
              inMemory: false
              Apc: ~

              Show
              Nuno Oliveira (Inactive) added a comment - Dear Hannes Giesenow, You can do it either way, whichever suits your operations best. But it is a know fact that inMemory caching can have a big impact on long running scripts. We warn users about it here: https://doc.ez.no/display/EZP/Persistence+cache+configuration Note for "inMemory" cache with long running scripts Use of inMemory caching with BlackHole or any other cache backend should not be used for long running scripts as it will over time return stale data, inMemory cache is not shared across requests/processes, so invalidation does not happen! You can see a similar warning on the StashBundle own page, in the "In-Memory" section: http://www.stashphp.com/Symfony.html By default, every cache service includes in-memory caching: during the lifetime of a single request, any values stored or retrieved from the cache service will be stored in memory, with the in-memory representation being checked before any other drivers. In some circumstances, however (such as long-running CLI batch scripts) this may not be desirable. In those cases, the in-memory driver can be disabled: stash: drivers: [ Apc ] inMemory: false Apc: ~
              Hide
              Permalink
              Hannes Giesenow added a comment -

              I don't want a warning, i want a working solution. In my opinion, the warning from stash should be considered and correctly be implemented inside the script.

              Show
              Hannes Giesenow added a comment - I don't want a warning, i want a working solution. In my opinion, the warning from stash should be considered and correctly be implemented inside the script.

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  Nuno Oliveira (Inactive)
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: