At the moment, custom tags in the xhtml5edit RichText format are represented by eztemplate (for block custom tag) and eztemplateinline (for inline custom tag) containing an ezcontent tag. Unfortunately, browsers are confused when a non HTML tag is used in the middle of an HTML snippet. So this should be changed so that the markup generated to represent a custom tag is (X)HTML valid.
For instance, at the moment, a potential ezquote block custom tag is represented with the following code in xhtml5edit:
this would become:
In the case of an inline custom tag the 2 divs becomes 2 spans.
For the record, pretty much the same was done for embed in https://github.com/ezsystems/ezpublish-kernel/pull/1435.