Since mendix will always store a copy of the objects provided to a published webservice you should either map the imported data to a different object or you should remove the validation and create validation microflows that are triggered during the webservice call. That way the object is allowed to be stored during the webservice call. After it has been stored you can throw a custom exception with help of the community commons.
If I understand you correct, you're getting a number of entities, let's say "Person" objects, and they all have a "PersonalId" attribute that you'd like to make unique. Then, for each "Person" object you receive via the webservice, you want to either create it (if the PersonalId can't be found in the DB) or retrieve the object from the DB and update it.
The current "retrieve" option in the webservice mapping form doesn't support this. This is due to the way the data is structured in the runtime, and will be fixed in 4.0 if I'm not mistaken. The current correct way to implement this behavior is to use a find by Microflow. In this microflow you should search for the attribute, if found create etc. Do not map the attribute in the mapping, only set it in the Microflow if you're creating a new object.