Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-24842

As a Maintainer I want eZ Platform installer to use Doctrine Schema files

    Details

    • Story Points:
      5

      Description

      We have https://github.com/ezsystems/ezpublish-kernel/blob/master/data/mysql/schema.sql but no corresponding Postgres schema file - this should be added.
      Perhaps we can use https://github.com/ezsystems/ezpublish-kernel/blob/master/eZ/Publish/Core/Persistence/Legacy/Tests/_fixtures/schema.pgsql.sql as a source?

      As a result of many discussions and reviews of the prototype we decided to make this a part of 2.0 release and use Doctrine Schema files.
      After some investigation we've found out that there's no official schema file format as Doctrine DBAL Schema Tool is PHP API. (Not to be confused with Doctrine ORM which actually has defined Yaml format).
      We've decided to create our own Yaml format of a schema which uses Doctrine Schema Tool.

      The remaining thing to do is to make installer an API, so developers can call it from their own code without a need to call ezplatform:install command. However, of course the command will work as it worked before - there's no reason for BC break on that.

      The major BC break is related to replacing schema.sql with schema.yml file. It's important to make it extensible as developers tend to modify it for custom needs. This can be achieved by creating some schema file provider. TBD.

      There are no Sub-Tasks for this issue.

        Activity

        Show
        Andrzej Longosz added a comment - - edited PR: https://github.com/ezsystems/ezpublish-kernel/pull/1924
        Hide
        Andrzej Longosz added a comment - - edited

        We don't want to create another SQL script, so we decided to create schema in a generic way, using Doctrine. However we have to postpone this until Kernel 7.0.

        Reason: generic creation of schema is not enough to solve this. We also need to provide cleandata in a more generic way (via Public API?) to properly handle sequences (workaround for resetting them after cleandata import is not the way to do it).

        In short: too many workarounds due to BC on 6.x

        Show
        Andrzej Longosz added a comment - - edited We don't want to create another SQL script, so we decided to create schema in a generic way, using Doctrine. However we have to postpone this until Kernel 7.0 . Reason: generic creation of schema is not enough to solve this. We also need to provide cleandata in a more generic way (via Public API?) to properly handle sequences (workaround for resetting them after cleandata import is not the way to do it). In short: too many workarounds due to BC on 6.x
        Show
        Andrzej Longosz added a comment - PR: https://github.com/ezsystems/ezpublish-kernel/pull/1941
        Hide
        Bertrand Dunogier added a comment -

        From PM's perspective, it would be great if you could list benefits we get from that. You most likely went over those when talking about it, and it would help a lot of we had that knowledge.

        Also, please try to clarify what the status is as compared to the latest comments & pull-request. The latest comment hints that it is mostly cleanup & testing, but according to january's first planning, there is more work needed.

        Show
        Bertrand Dunogier added a comment - From PM's perspective, it would be great if you could list benefits we get from that. You most likely went over those when talking about it, and it would help a lot of we had that knowledge. Also, please try to clarify what the status is as compared to the latest comments & pull-request. The latest comment hints that it is mostly cleanup & testing, but according to january's first planning, there is more work needed.
        Hide
        Andrzej Longosz added a comment -

        What we've established so far by discussing it internally in December:

        • there will be a separate package ezsystems/ezplatform-installer providing:
          • bundle, ezplatform:install command and services (so installer can be injected via DI into another process)
          • logic parsing Yaml into SQL, including our custom improvements for PostgreSQL (missing expression index columns support) and MySQL (missing index column length support) implemented by extending Doctrine's PostgreSQLPlatform and MySqlPlatform. Note: we use Doctrine's Schema Tool, but for DBAL it is a PHP API only, so we've created our own format, similar to what I've seen for ORM, but more compact.
          • command and parser dumping existing schema to Yaml file (because why not)
          • in the future: maybe some command to update eZ Platform (just a random thought, didn't share it with anyone until now )
        • mentioned package would be a meta-repository dependency, not kernel, because it requires kernel
        • in kernel we would keep core schema and data (for now both core tables and external storage, in the future: each FieldType would provide its own schema file for external storage) - this is a little bit tricky, because I don't want to hardcode vendor/ezpublish-kernel/data/schema.yml path. Someone suggested schema and data files should be bundle Resources, so maybe it's a way to go.

        AFAIR (last time when I considered this) there's at least one difficulty with that solution: ideally we should get rid of all SQL schemas, including the ones for tests. Running integration tests on a database installed from Yaml schema would be a huge advantage.
        The solutions:

        • parts related to interpreting Yaml schema should remain in kernel
        • maybe interpreting Yaml schema should be another separate package, independent from kernel and reusable in any projects using Doctrine/DBAL?

        Other remaining things to do:

        • check if schema contains all changes done in past months (comparing schemas installed the old and new way, both for Postgres and MySQL).
        • test once again if app works (no extensive behat yet, so that's a risk)
        • make behat setup choose PostgreSQL database, controlled by Travis parameters, so we can at least track if installer works.
        Show
        Andrzej Longosz added a comment - What we've established so far by discussing it internally in December: there will be a separate package ezsystems/ezplatform-installer providing: bundle, ezplatform:install command and services (so installer can be injected via DI into another process) logic parsing Yaml into SQL, including our custom improvements for PostgreSQL (missing expression index columns support) and MySQL (missing index column length support) implemented by extending Doctrine's PostgreSQLPlatform and MySqlPlatform . Note: we use Doctrine's Schema Tool, but for DBAL it is a PHP API only, so we've created our own format, similar to what I've seen for ORM, but more compact. command and parser dumping existing schema to Yaml file (because why not) in the future: maybe some command to update eZ Platform (just a random thought, didn't share it with anyone until now ) mentioned package would be a meta-repository dependency, not kernel, because it requires kernel in kernel we would keep core schema and data (for now both core tables and external storage, in the future: each FieldType would provide its own schema file for external storage) - this is a little bit tricky, because I don't want to hardcode vendor/ezpublish-kernel/data/schema.yml path. Someone suggested schema and data files should be bundle Resources, so maybe it's a way to go. AFAIR (last time when I considered this) there's at least one difficulty with that solution: ideally we should get rid of all SQL schemas, including the ones for tests. Running integration tests on a database installed from Yaml schema would be a huge advantage. The solutions: parts related to interpreting Yaml schema should remain in kernel maybe interpreting Yaml schema should be another separate package, independent from kernel and reusable in any projects using Doctrine/DBAL? Other remaining things to do: check if schema contains all changes done in past months (comparing schemas installed the old and new way, both for Postgres and MySQL). test once again if app works (no extensive behat yet, so that's a risk) make behat setup choose PostgreSQL database, controlled by Travis parameters, so we can at least track if installer works.
        Hide
        Andrzej Longosz added a comment -

        Closing this one in favor of the fresh Story EZP-29938

        Show
        Andrzej Longosz added a comment - Closing this one in favor of the fresh Story EZP-29938

          People

          • Assignee:
            Unassigned
            Reporter:
            Gunnstein Lye
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 0 minutes
              0m
              Remaining:
              Remaining Estimate - 0 minutes
              0m
              Logged:
              Time Spent - 2 weeks, 5 hours
              2w 5h

                Agile