Matt,
In short, yes! Take a look at the "Selectable Objects" tab of the reference set selector. That's where you'll input some XPath to constrain the selection.
Now, since you want to limit a salesman to particular colors, you'll want to make sure you have an association Salesman and a Color. Car sales should be changed to have an association with the Color entity as well, and you should remove the "Color" attribute on CarSales.
In your Selectable Objects XPath, you'd want to write something like this:
[Module.Cars_Color/Module.Color/Module.Color_Salesman = '[%CurrentUser%]']
This XPath would limit the selection of cars to only ones with a color that is also associated to the current user's account. Hope that helps!
If at all possible, use constrained by on selectable objects tab. This is also applied and refreshed when other elements change and can be used to make reference selectors depend on each other like the make and model of a car.