The simplest solution is to create additional attribute: “Selected” of the type Boolean, with default as False, in the entity from where you are selecting the values. Once its selected, update this value to True.
In your reference set selection, then your Xpath only needs to have “Selected=false’
In case, you wouldn’t like to add this attribute, then your XPath will need to check for the association between the two entities exists or not for this particular record. This might get a bit tricky.