Hello Nico,
I have just completed the learning path 'Quality Control and Testing' (Link) and I noticed the same behavior. According to the step description, the rollback should prevent all changes on my entities, but new entries are added and others are deleted after the tests run.
I used Mendix Studio Pro 8.5.1, UnitTest 8.0.0 and ObjectHandling 3.0.0.
This issue has since been resolved by Mendix Support