Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-24732

RichText field does not accept html5edit input with HTML entities

    Details

      Description

      Like   or " which are quite common when using a WYSIWYG editor but actually it should be possible to use any HTML entity.

      Step to reproduce: https://github.com/dpobel/TestCommandsBundle/blob/master/Command/CreateRichTextEntitiesCommand.php

        Issue Links

          Activity

          Hide
          Petar Spanja (Inactive) added a comment -

          [~damien.pobel@ez.no]

          The editor should actually force numeric entities. Such setting should exist for CK editor.
          Alternative is to define our DTD or to give up on a well formed XML as input by using loadHTML(), both of which I would rather not do.

          Show
          Petar Spanja (Inactive) added a comment - [~damien.pobel@ez.no] The editor should actually force numeric entities. Such setting should exist for CK editor. Alternative is to define our DTD or to give up on a well formed XML as input by using loadHTML() , both of which I would rather not do.
          Hide
          André Rømcke added a comment -

          > or to give up on a well formed XML

          but we are referring to the html5edit format here only right? using loadHtml, will this transform these to xml safe equivalents when storing/converting to xml?

          Show
          André Rømcke added a comment - > or to give up on a well formed XML but we are referring to the html5edit format here only right? using loadHtml, will this transform these to xml safe equivalents when storing/converting to xml?
          Hide
          Petar Spanja (Inactive) added a comment -

          It will, though loadHTML will also accept anything and load it into HTML document.
          Let me know if ensuring numerical entities from editor side is not an option, I can work something out with pre-processing at least.

          Show
          Petar Spanja (Inactive) added a comment - It will, though loadHTML will also accept anything and load it into HTML document. Let me know if ensuring numerical entities from editor side is not an option, I can work something out with pre-processing at least.
          Hide
          Damien Pobel (Inactive) added a comment -

          For the record, this prevents any usage of the RichText editor. For the potential users searching for the exception message, in PlatformUI the save draft REST request ends with a 406 with the message: Argument '$inputValue' is invalid: Could not create XML document: Entity 'nbsp' not defined or similar for others entities.

          Petar Spanja indeed CKEditor has an option to transform entities into numeric entities but my first quick test was not really successful, but I need to dig a bit more on how to pass CKEditor settings from AlloyEditor. That said, I dont' want to be picky but the format is called "html5edit" not "xhtml5edit". It's not obvious (nor documented anywhere AFAIK) that this should be a well formed XML document. And given how the browsers/rich text editors are working, I'm pretty sure we'll need a bit more flexible/relaxed parser at some point or to introduce some flexibility somewhere.

          Show
          Damien Pobel (Inactive) added a comment - For the record, this prevents any usage of the RichText editor. For the potential users searching for the exception message, in PlatformUI the save draft REST request ends with a 406 with the message: Argument '$inputValue' is invalid: Could not create XML document: Entity 'nbsp' not defined or similar for others entities. Petar Spanja indeed CKEditor has an option to transform entities into numeric entities but my first quick test was not really successful, but I need to dig a bit more on how to pass CKEditor settings from AlloyEditor. That said, I dont' want to be picky but the format is called "html5edit" not "xhtml5edit". It's not obvious (nor documented anywhere AFAIK) that this should be a well formed XML document. And given how the browsers/rich text editors are working, I'm pretty sure we'll need a bit more flexible/relaxed parser at some point or to introduce some flexibility somewhere.
          Hide
          Petar Spanja (Inactive) added a comment - - edited

          @[~damien.pobel@ez.no] For the record the format is called xhtml5edit, with namespace http://ez.no/namespaces/ezpublish5/xhtml5/edit and up until recently it even had it's own XSD schema. So I think it was clear from the beginning what the intention was.

          But not saying we can't implement this server-side, I just needed feedback from you.

          Show
          Petar Spanja (Inactive) added a comment - - edited @ [~damien.pobel@ez.no] For the record the format is called xhtml5edit , with namespace http://ez.no/namespaces/ezpublish5/xhtml5/edit and up until recently it even had it's own XSD schema. So I think it was clear from the beginning what the intention was. But not saying we can't implement this server-side, I just needed feedback from you.
          Hide
          Damien Pobel (Inactive) added a comment -

          you are of course right, I should not comment on JIRA before my first coffee I'm trying again on the CKEditor settings

          Show
          Damien Pobel (Inactive) added a comment - you are of course right, I should not comment on JIRA before my first coffee I'm trying again on the CKEditor settings
          Hide
          Damien Pobel (Inactive) added a comment -

          For whatever reason, the setting does not work through AlloyEditor I tried different things without luck and I open an issue in AlloyEditor tracker: https://github.com/liferay/alloy-editor/issues/359

          So Petar Spanja it would be nice to solve somehow server side, other RichText editor is unusable in a lots of different use cases. And I would vote for defining the entities in our DTD.

          Show
          Damien Pobel (Inactive) added a comment - For whatever reason, the setting does not work through AlloyEditor I tried different things without luck and I open an issue in AlloyEditor tracker: https://github.com/liferay/alloy-editor/issues/359 So Petar Spanja it would be nice to solve somehow server side, other RichText editor is unusable in a lots of different use cases. And I would vote for defining the entities in our DTD.
          Show
          Petar Spanja (Inactive) added a comment - Pull request https://github.com/ezsystems/ezpublish-kernel/pull/1480 Merged in https://github.com/ezsystems/ezpublish-kernel/commit/3c1ff2c73e83b94518ffb26dc4b45458f5809afc
          Hide
          Rui Silva (Inactive) added a comment -

          The PR doesn't work.
          I still get the error when passing html entities through the xml field, on running Damien's example code from a custom command.
          Sending back to input queue.

          Show
          Rui Silva (Inactive) added a comment - The PR doesn't work. I still get the error when passing html entities through the xml field, on running Damien's example code from a custom command. Sending back to input queue.
          Hide
          Petar Spanja (Inactive) added a comment -

          [~rui.silva@ez.no] you will need to define named entities in DTD as is required by XML. see example here: https://github.com/ezsystems/ezpublish-kernel/blob/master/eZ/Publish/Core/FieldType/Tests/RichText/Converter/Xslt/_fixtures/xhtml5/edit/029-htmlEntities.xml

          Handling XML-like input with named entities will be done in a separate issue ad is not a bug fix. Feel free to take the issue again.

          Show
          Petar Spanja (Inactive) added a comment - [~rui.silva@ez.no] you will need to define named entities in DTD as is required by XML. see example here: https://github.com/ezsystems/ezpublish-kernel/blob/master/eZ/Publish/Core/FieldType/Tests/RichText/Converter/Xslt/_fixtures/xhtml5/edit/029-htmlEntities.xml Handling XML-like input with named entities will be done in a separate issue ad is not a bug fix. Feel free to take the issue again.
          Hide
          Damien Pobel (Inactive) added a comment -

          This issue is referenced in several places, so please keep this issue opened until the issue is really fixed, as said before, having to define the entities is not an option in a real usage scenario and the solution merged here is more workaround.

          Show
          Damien Pobel (Inactive) added a comment - This issue is referenced in several places, so please keep this issue opened until the issue is really fixed, as said before, having to define the entities is not an option in a real usage scenario and the solution merged here is more workaround.
          Hide
          Petar Spanja (Inactive) added a comment -

          New improvement issue handling RichText input to XML normalization.

          Show
          Petar Spanja (Inactive) added a comment - New improvement issue handling RichText input to XML normalization.
          Hide
          Damien Pobel (Inactive) added a comment -

          Petar Spanja now that EZP-25027 is implemented, what is missing so that HTML entities are accepted when saving a RichText containing an HTML entities ?

          Show
          Damien Pobel (Inactive) added a comment - Petar Spanja now that EZP-25027 is implemented, what is missing so that HTML entities are accepted when saving a RichText containing an HTML entities ?
          Hide
          Damien Pobel (Inactive) added a comment -

          it seems to work actually, I guess I was not up to date when I tested a while ago, so sending to QA.

          Show
          Damien Pobel (Inactive) added a comment - it seems to work actually, I guess I was not up to date when I tested a while ago, so sending to QA.
          Hide
          Rui Silva (Inactive) added a comment -

          Tested and approved by QA for master.

          Show
          Rui Silva (Inactive) added a comment - Tested and approved by QA for master.
          Hide
          Damien Pobel (Inactive) added a comment - - edited

          Reopening because this was actually hidden by a bug in PlatformUI. The following curl command allows to reproduce the issue:

          curl 'http://ez.loc/api/ezp/v2/content/objects' -H 'X-CSRF-Token: r6LpxwMcsXNMck_sdGGmym2GeV0RB75IH6GI0TTBslk' \
          -H 'Content-Type: application/vnd.ez.api.ContentCreate+json' \
          -H 'Accept: application/vnd.ez.api.Content+json' \
          -H 'Cookie: is_logged_in=true; privacyCookieAccepted=1; eZSESSID=oi1ikoj6n4hurhok2bcakb8oo7' \
          --data-binary '{"ContentCreate":{"ContentType":{"_href":"/api/ezp/v2/content/types/18"},"mainLanguageCode":"eng-GB","LocationCreate":{"ParentLocation":{"_href":"/api/ezp/v2/content/locations/1/2/90"},"sortField":"PATH","sortOrder":"ASC"},"Section":null,"alwaysAvailable":false,"remoteId":null,"modificationDate":"2015-11-30T14:57:28.374Z","fields":{"field":[{"fieldDefinitionIdentifier":"title","fieldValue":"Test"},{"fieldDefinitionIdentifier":"body","fieldValue":{"xml":"<section xmlns=\"http://ez.no/namespaces/ezpublish5/xhtml5/edit\"><p>&nbsp;</p></section>"}},{"fieldDefinitionIdentifier":"publication_date","fieldValue":{"timestamp":1448893263}},{"fieldDefinitionIdentifier":"tags","fieldValue":[]}]}}}'
          

          {
              "ErrorMessage": {
                  "_media-type": "application\/vnd.ez.api.ErrorMessage+json",
                  "errorCode": 406,
                  "errorMessage": "Not Acceptable",
                  "errorDescription": "Argument '$inputValue' is invalid: Could not create XML document: Entity 'nbsp' not defined",
                  "trace": "#0 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/RichText\/Type.php(167): eZ\\Publish\\Core\\FieldType\\RichText\\Type->loadXMLString('<section xmlns=...')\n#1 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/FieldType.php(383): eZ\\Publish\\Core\\FieldType\\RichText\\Type->createValueFromInput('<section xmlns=...')\n#2 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/RichText\/Type.php(303): eZ\\Publish\\Core\\FieldType\\FieldType->acceptValue('<section xmlns=...')\n#3 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/Repository\/Values\/ContentType\/FieldType.php(191): eZ\\Publish\\Core\\FieldType\\RichText\\Type->fromHash(Array)\n#4 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Common\/Input\/FieldTypeParser.php(96): eZ\\Publish\\Core\\Repository\\Values\\ContentType\\FieldType->fromHash(Array)\n#5 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Server\/Input\/Parser\/ContentCreate.php(169): eZ\\Publish\\Core\\REST\\Common\\Input\\FieldTypeParser->parseValue('ezrichtext', Array)\n#6 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Common\/Input\/ParsingDispatcher.php(83): eZ\\Publish\\Core\\REST\\Server\\Input\\Parser\\ContentCreate->parse(Array, Object(eZ\\Publish\\Core\\REST\\Common\\Input\\ParsingDispatcher))\n#7 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Common\/Input\/Dispatcher.php(117): eZ\\Publish\\Core\\REST\\Common\\Input\\ParsingDispatcher->parse(Array, 'application\/vnd...')\n#8 \/home\/dp\/dev\/PlatformUIBundle\/Controller\/Rest\/ContentController.php(116): eZ\\Publish\\Core\\REST\\Common\\Input\\Dispatcher->parse(Object(eZ\\Publish\\Core\\REST\\Common\\Message))\n#9 \/home\/dp\/dev\/PlatformUIBundle\/Controller\/Rest\/ContentController.php(32): EzSystems\\PlatformUIBundle\\Controller\\Rest\\ContentController->isUserCreateRequest(Object(Symfony\\Component\\HttpFoundation\\Request))\n#10 [internal function]: EzSystems\\PlatformUIBundle\\Controller\\Rest\\ContentController->createContent(Object(Symfony\\Component\\HttpFoundation\\Request))\n#11 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/HttpKernel.php(147): call_user_func_array(Array, Array)\n#12 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/HttpKernel.php(68): Symfony\\Component\\HttpKernel\\HttpKernel->handleRaw(Object(Symfony\\Component\\HttpFoundation\\Request), 1)\n#13 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/DependencyInjection\/ContainerAwareHttpKernel.php(69): Symfony\\Component\\HttpKernel\\HttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#14 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/Kernel.php(196): Symfony\\Component\\HttpKernel\\DependencyInjection\\ContainerAwareHttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#15 \/home\/dp\/dev\/ezplatform\/web\/index.php(66): Symfony\\Component\\HttpKernel\\Kernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request))\n#16 {main}",
                  "file": "\/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/RichText\/Type.php",
                  "line": 218
              }
          }
          

          Show
          Damien Pobel (Inactive) added a comment - - edited Reopening because this was actually hidden by a bug in PlatformUI. The following curl command allows to reproduce the issue: curl 'http://ez.loc/api/ezp/v2/content/objects' -H 'X-CSRF-Token: r6LpxwMcsXNMck_sdGGmym2GeV0RB75IH6GI0TTBslk' \ -H 'Content-Type: application/vnd.ez.api.ContentCreate+json' \ -H 'Accept: application/vnd.ez.api.Content+json' \ -H 'Cookie: is_logged_in=true; privacyCookieAccepted=1; eZSESSID=oi1ikoj6n4hurhok2bcakb8oo7' \ --data-binary '{"ContentCreate":{"ContentType":{"_href":"/api/ezp/v2/content/types/18"},"mainLanguageCode":"eng-GB","LocationCreate":{"ParentLocation":{"_href":"/api/ezp/v2/content/locations/1/2/90"},"sortField":"PATH","sortOrder":"ASC"},"Section":null,"alwaysAvailable":false,"remoteId":null,"modificationDate":"2015-11-30T14:57:28.374Z","fields":{"field":[{"fieldDefinitionIdentifier":"title","fieldValue":"Test"},{"fieldDefinitionIdentifier":"body","fieldValue":{"xml":"<section xmlns=\"http://ez.no/namespaces/ezpublish5/xhtml5/edit\"><p>&nbsp;</p></section>"}},{"fieldDefinitionIdentifier":"publication_date","fieldValue":{"timestamp":1448893263}},{"fieldDefinitionIdentifier":"tags","fieldValue":[]}]}}}' { "ErrorMessage": { "_media-type": "application\/vnd.ez.api.ErrorMessage+json", "errorCode": 406, "errorMessage": "Not Acceptable", "errorDescription": "Argument '$inputValue' is invalid: Could not create XML document: Entity 'nbsp' not defined", "trace": "#0 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/RichText\/Type.php(167): eZ\\Publish\\Core\\FieldType\\RichText\\Type->loadXMLString('<section xmlns=...')\n#1 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/FieldType.php(383): eZ\\Publish\\Core\\FieldType\\RichText\\Type->createValueFromInput('<section xmlns=...')\n#2 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/RichText\/Type.php(303): eZ\\Publish\\Core\\FieldType\\FieldType->acceptValue('<section xmlns=...')\n#3 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/Repository\/Values\/ContentType\/FieldType.php(191): eZ\\Publish\\Core\\FieldType\\RichText\\Type->fromHash(Array)\n#4 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Common\/Input\/FieldTypeParser.php(96): eZ\\Publish\\Core\\Repository\\Values\\ContentType\\FieldType->fromHash(Array)\n#5 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Server\/Input\/Parser\/ContentCreate.php(169): eZ\\Publish\\Core\\REST\\Common\\Input\\FieldTypeParser->parseValue('ezrichtext', Array)\n#6 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Common\/Input\/ParsingDispatcher.php(83): eZ\\Publish\\Core\\REST\\Server\\Input\\Parser\\ContentCreate->parse(Array, Object(eZ\\Publish\\Core\\REST\\Common\\Input\\ParsingDispatcher))\n#7 \/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/REST\/Common\/Input\/Dispatcher.php(117): eZ\\Publish\\Core\\REST\\Common\\Input\\ParsingDispatcher->parse(Array, 'application\/vnd...')\n#8 \/home\/dp\/dev\/PlatformUIBundle\/Controller\/Rest\/ContentController.php(116): eZ\\Publish\\Core\\REST\\Common\\Input\\Dispatcher->parse(Object(eZ\\Publish\\Core\\REST\\Common\\Message))\n#9 \/home\/dp\/dev\/PlatformUIBundle\/Controller\/Rest\/ContentController.php(32): EzSystems\\PlatformUIBundle\\Controller\\Rest\\ContentController->isUserCreateRequest(Object(Symfony\\Component\\HttpFoundation\\Request))\n#10 [internal function]: EzSystems\\PlatformUIBundle\\Controller\\Rest\\ContentController->createContent(Object(Symfony\\Component\\HttpFoundation\\Request))\n#11 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/HttpKernel.php(147): call_user_func_array(Array, Array)\n#12 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/HttpKernel.php(68): Symfony\\Component\\HttpKernel\\HttpKernel->handleRaw(Object(Symfony\\Component\\HttpFoundation\\Request), 1)\n#13 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/DependencyInjection\/ContainerAwareHttpKernel.php(69): Symfony\\Component\\HttpKernel\\HttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#14 \/home\/dp\/dev\/ezplatform\/vendor\/symfony\/symfony\/src\/Symfony\/Component\/HttpKernel\/Kernel.php(196): Symfony\\Component\\HttpKernel\\DependencyInjection\\ContainerAwareHttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#15 \/home\/dp\/dev\/ezplatform\/web\/index.php(66): Symfony\\Component\\HttpKernel\\Kernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request))\n#16 {main}", "file": "\/home\/dp\/dev\/ezpublish-kernel\/eZ\/Publish\/Core\/FieldType\/RichText\/Type.php", "line": 218 } }
          Hide
          Damien Pobel (Inactive) added a comment -

          Note: this is supposed to be fixed thanks to https://jira.ez.no/browse/EZP-25027

          Show
          Damien Pobel (Inactive) added a comment - Note: this is supposed to be fixed thanks to https://jira.ez.no/browse/EZP-25027
          Hide
          Bertrand Dunogier added a comment - - edited
          Show
          Bertrand Dunogier added a comment - - edited Fixed in https://github.com/ezsystems/ezpublish-kernel/pull/1529 (merged to master@80bf1d0 ).
          Hide
          Rui Silva (Inactive) added a comment -

          Tested and approved by QA for master.

          Show
          Rui Silva (Inactive) added a comment - Tested and approved by QA for master.

            People

            • Assignee:
              Unassigned
              Reporter:
              Damien Pobel (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 7 hours, 5 minutes
                7h 5m