Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-26118

Cluster with php7 fails when creating images

    Details

    • Type: Story Story
    • Status: Closed
    • Priority: High High
    • Resolution: Fixed
    • Affects Version/s: 1.5.0, 1.4.1, 1.6.0-beta1
    • Fix Version/s: QA tracked issues
    • Component/s: Misc
    • Labels:
    • Environment:

      Ubuntu 16.04
      php 7.0
      cluster with redis
      Env: prod

      Description

      Having redis cluster with php7, when I try to create an image, I have the error:

      POST http://ezdfs1.ezp/api/ezp/v2/content/objects
      500 Internal Server Error
       
      {"ErrorMessage":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage"
      :"Internal Server Error","errorDescription":"A DBAL error occured while writing var\/site\/storage\/images
      \/3\/8\/1\/0\/183-1-eng-GB\/imagem04.jpg"}}
      

      Steps to Reproduce:
      • Access your installation as admin in admin interface
      • Create one image. Having your developer tools active, publish the image.
        On the publishing action, we can see a notification error and on developer tools, we can see the error referred above.
        If I go to my dfs var dir (/mnt/ezdfs/var/site/storage/images/3/8/1/0/183-1-eng-GB/imagem04.jpg), I can see that the image was created.

      If I create some object such Folders or articles, without attachments, then I have no errors.

      More info about my cluster environment:

      My cluster configuration is set with 4 servers as:

      • Server1(Varnish in round robin) -
      • Server2 (ezdfs1)(with ezplatform) -> 10.0.5.2
      • Server3 (ezdfs1)(with ezplatform) -> 10.0.5.3
      • Server4 (with MariaDB DB) - 10.0.5.4

      I have redis server on both server2 (10.0.5.2) and server3(10.0.5.3) configured as cluster (Master/Slave)

      Regarding sessions, I have in php.ini:

      session.save_handler = redis
      session.save_path = "tcp://10.0.5.2:6379" ;sessions are kept in first server
      

      And I have my ezplatform set as:

      ezpublish:
          # Repositories configuration, setup default repository to support solr if enabled
          repositories:
              default:
                  storage: ~
                  search:
                      engine: %search_engine%
                      connection: default
       
          # Siteaccess configuration, with one siteaccess per default
          siteaccess:
              list: [site]
              groups:
                  site_group: [site]
              default_siteaccess: site
              match:
                  URIElement: 1
       
          # System settings, grouped by siteaccess and/or siteaccess group
          http_cache:
              # As of 5.4 only use "http"
              # "single_http" and "multiple_http" are deprecated but will still work.
              purge_type: http
          system:
              default:
                  io:
                      metadata_handler: dfs
                      binarydata_handler: nfs
                      url_prefix: "storage"
              site_group:
                  # Pool to use for cache, needs to be differant per repository (database).
                  cache_pool_name: '%cache_pool%'
                  # These reflect the current installers, complete installation before you change them. For changing var_dir
                  # it is recommended to install clean, then change setting before you start adding binary content, otherwise you'll
                  # need to manually modify your database data to reflect this to avoid exceptions.
                  var_dir: var/site
                  # System languages. Note that by default, content, content types, and other data are in eng-GB locale,
                  # so removing eng-GB from this list may lead to errors or content not being shown, unless you change
                  # all eng-GB data to other locales first.
                  languages: [eng-GB]
                  http_cache:
                      # Fill in your Varnish server(s) address(es).
                      purge_servers: [http://192.168.2.201:6081]
                  session:
                      name: ~
       
      # new doctrine connection for the dfs legacy_dfs_cluster metadata handler.
      doctrine:
          dbal:
              connections:
                  dfs:
                      driver: pdo_mysql
                      host: 10.0.5.4
                      port: 3306
                      dbname: ezp
                      user: ezp
                      password: "ezp"
                      charset: UTF8
       
      # declare the handlers
      ez_io:
          binarydata_handlers:
              nfs:
                  flysystem:
                      adapter: nfs_adapter
          metadata_handlers:
              dfs:
                  legacy_dfs_cluster:
                      connection: doctrine.dbal.dfs_connection
       
      oneup_flysystem:
          adapters:
              nfs_adapter:
                  local:
                      # The last part, $var_dir$/$storage_dir$, is required for legacy compatibility
                      directory: "/mnt/ezdfs/$var_dir$/$storage_dir$"
       
      stash:
          caches:
              default:
                  drivers: [ Redis ]
                  Redis:
                      servers:
                          -
                              server: 10.0.5.2
                              port: 6379
                          -
                              server: 10.0.5.3
                              port: 6379
      

        Issue Links

          Activity

          Hide
          Bertrand Dunogier added a comment - - edited

          Not reproduced yet un single server with built-in PHP server, configured with DFS to a local directory.

          Could you try to apply https://github.com/ezsystems/ezpublish-kernel/pull/1805 (it's only a couple lines, trying to get it into the kernel), and paste the error you get from REST when trying to save the image ?

          P.S. Are you 100% sure that your database for DFS is reached ? I got a timeout from this layer with a bad IP address.

          Show
          Bertrand Dunogier added a comment - - edited Not reproduced yet un single server with built-in PHP server, configured with DFS to a local directory. Could you try to apply https://github.com/ezsystems/ezpublish-kernel/pull/1805 (it's only a couple lines, trying to get it into the kernel), and paste the error you get from REST when trying to save the image ? P.S. Are you 100% sure that your database for DFS is reached ? I got a timeout from this layer with a bad IP address.
          Hide
          Paulo Nunes (Inactive) added a comment -

          Tried setting up 1.6.0-beta1, with the patch you referred, I have in the response:

          {"ErrorMessage":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage"
          :"Internal Server Error","errorDescription":"A DBAL error occured while writing var\/site\/storage\/images
          \/4\/3\/2\/0\/234-1-eng-GB\/imagem07.jpg","Previous":{"_media-type":"application\/vnd.ez.api.ErrorMessage
          +json","ErrorMessage":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage"
          :"Internal Server Error","errorDescription":"An exception occurred while executing 'INSERT INTO ezdfsfile
          \n  (name, name_hash, name_trunk, mtime, size, scope, datatype)\n  VALUES (:name, :name_hash, :name_trunk
          , :mtime, :size, :scope, :datatype)\nON DUPLICATE KEY UPDATE\n  datatype=VALUES(datatype), scope=VALUES
          (scope), size=VALUES(size),\n  mtime=VALUES(mtime), expired=VALUES(expired)' with params [\"var\\\/site
          \\\/storage\\\/images\\\/4\\\/3\\\/2\\\/0\\\/234-1-eng-GB\\\/imagem07.jpg\", \"1c0ae360e67a8cae31cc0bb68731b049
          \", \"var\\\/site\\\/storage\\\/images\\\/4\\\/3\\\/2\\\/0\\\/234-1-eng-GB\\\/imagem07.jpg\", 1476887809
          , 7040, \"image\", \"image\\\/jpeg\"]:\n\nSQLSTATE[42S02]: Base table or view not found: 1146 Table 'ezp
          .ezdfsfile' doesn't exist","Previous":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","ErrorMessage"
          :{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage":"Internal
           Server Error","errorDescription":"SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ezp.ezdfsfile'
           doesn't exist","Previous":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","ErrorMessage"
          :{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage":"Internal
           Server Error","errorDescription":"SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ezp.ezdfsfile'
           doesn't exist"}}}}}}}}
          

          err..."ezdfsfile" table... in ezplatform?

          Are you 100% sure that your database for DFS is reached

          yes. Creating other objects , such folders, work and I have them referred on respective tables.

          Just in case, retried the same process with php5.6 with memcache and this problem

          Show
          Paulo Nunes (Inactive) added a comment - Tried setting up 1.6.0-beta1, with the patch you referred, I have in the response: {"ErrorMessage":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage" :"Internal Server Error","errorDescription":"A DBAL error occured while writing var\/site\/storage\/images \/4\/3\/2\/0\/234-1-eng-GB\/imagem07.jpg","Previous":{"_media-type":"application\/vnd.ez.api.ErrorMessage +json","ErrorMessage":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage" :"Internal Server Error","errorDescription":"An exception occurred while executing 'INSERT INTO ezdfsfile \n (name, name_hash, name_trunk, mtime, size, scope, datatype)\n VALUES (:name, :name_hash, :name_trunk , :mtime, :size, :scope, :datatype)\nON DUPLICATE KEY UPDATE\n datatype=VALUES(datatype), scope=VALUES (scope), size=VALUES(size),\n mtime=VALUES(mtime), expired=VALUES(expired)' with params [\"var\\\/site \\\/storage\\\/images\\\/4\\\/3\\\/2\\\/0\\\/234-1-eng-GB\\\/imagem07.jpg\", \"1c0ae360e67a8cae31cc0bb68731b049 \", \"var\\\/site\\\/storage\\\/images\\\/4\\\/3\\\/2\\\/0\\\/234-1-eng-GB\\\/imagem07.jpg\", 1476887809 , 7040, \"image\", \"image\\\/jpeg\"]:\n\nSQLSTATE[42S02]: Base table or view not found: 1146 Table 'ezp .ezdfsfile' doesn't exist","Previous":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","ErrorMessage" :{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage":"Internal Server Error","errorDescription":"SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ezp.ezdfsfile' doesn't exist","Previous":{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","ErrorMessage" :{"_media-type":"application\/vnd.ez.api.ErrorMessage+json","errorCode":500,"errorMessage":"Internal Server Error","errorDescription":"SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ezp.ezdfsfile' doesn't exist"}}}}}}}} err..."ezdfsfile" table... in ezplatform? Are you 100% sure that your database for DFS is reached yes. Creating other objects , such folders, work and I have them referred on respective tables. Just in case, retried the same process with php5.6 with memcache and this problem
          Hide
          Bertrand Dunogier added a comment -

          Base table or view not found: 1146 Table 'ezp.ezdfsfile' doesn't exist"
          

          Well, if the table doesn't exist, it's not gonna work is it ? It should probably be created (in my case, I've used the same database than ezplatform. Since the table is part of the default schema, it does exist.

          The fact that you can create other content types doesn't indicate that DFS works: if there is no image/binaryfile involved in this content, no Query is gonna be executed.

          Show
          Bertrand Dunogier added a comment - Base table or view not found: 1146 Table 'ezp.ezdfsfile' doesn't exist" Well, if the table doesn't exist, it's not gonna work is it ? It should probably be created (in my case, I've used the same database than ezplatform. Since the table is part of the default schema, it does exist. The fact that you can create other content types doesn't indicate that DFS works: if there is no image/binaryfile involved in this content, no Query is gonna be executed.
          Hide
          Paulo Nunes (Inactive) added a comment - - edited

          (...)Since the table is part of the default schema, it does exist.

          You mean this schema? Because I can't find the table in there.

          and cluster documentation don't refer anywhere to create it or to execute some script to create it.

          But this confuses me even more. Doing exactly the same installation/configuration process referred in description, with memcached (in php5.6), I don't have this problem. And 'ezdfsfile' table is not in the db as well... (my bad. Found out that in my php5.6 env, I was refering one db in parameters.yml and another one in ezplatform.yml:doctrine:dbal:connections:dfs:dbname that already had ezdfsfile table)

          Show
          Paulo Nunes (Inactive) added a comment - - edited (...)Since the table is part of the default schema, it does exist. You mean this schema ? Because I can't find the table in there. and cluster documentation don't refer anywhere to create it or to execute some script to create it. But this confuses me even more. Doing exactly the same installation/configuration process referred in description, with memcached (in php5.6), I don't have this problem. And 'ezdfsfile' table is not in the db as well... (my bad. Found out that in my php5.6 env, I was refering one db in parameters.yml and another one in ezplatform.yml:doctrine:dbal:connections:dfs:dbname that already had ezdfsfile table)
          Hide
          Paulo Nunes (Inactive) added a comment -

          The culprit it's the lack of ezdfsfile in the schema. I created the tables using legacy script, and the issue no longer occur when having it.

          Just to avoid spending time reading all the comments from this issue, i'll close it in favor of EZP-26482

          Show
          Paulo Nunes (Inactive) added a comment - The culprit it's the lack of ezdfsfile in the schema. I created the tables using legacy script , and the issue no longer occur when having it. Just to avoid spending time reading all the comments from this issue, i'll close it in favor of EZP-26482

            People

            • Assignee:
              Unassigned
              Reporter:
              Paulo Nunes (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: