It is possible to assign limitations to users, but it is not possible to unassign one single limitation from a user.

The only possible way to unassign limitations from users is by using $roleService->unassignRoleFromUser which removes all the existing limitations for the given user.

A feature to unasign a single limitation from a user would be a great implementation.