You could do an XPath Retrieve on the same reference set and compare this list with the list you have from a retrieve from association (which is the newest changed version). From there you can use things like List Intersect to find the changes and apply your validation.
You could also create a custom microflow that handles the removal and do the test there. That way you do not have to do a commit.
I must say this is just really inconvenient, why can't this just pass 2 parameters. A list of entities that are being changed and a parameter Add or Deduct.