Currently eZClusterFileHandler::cleanupEmptyDirectories() is being called every time a file is deleted.
Although the operation itself doesn't take too long to execute, the frequency with which this method is invoked is somewhat excessive.

Often happens that a file is removed and a new one is recreated in the very same place (cache or image alias regeneration, for instance).

If it is not viable to split between file deletions that require cleanupEmptyDirectories and those that do not, it might be possible to defer the actual removal operation to a cronjob.