It seems that Admin UI v2 is always starting the session for anonymous user, which effectively breaks the Varnish cache.

It looks like the following service is causing trouble:

EzSystems\EzPlatformAdminUi\Notification\FlashBagNotificationHandler:
        arguments:
            $flashBag: "@=service('session').getFlashBag()"