Removing Duplicates from a list

I am trying to create a list of objects from an imported JSON. I am sure that there are going to be duplicate entries coming from the JSON to be filled in the list and I need to avoid adding duplicate items while creating the list. The uniqueness of an item in the list is determined by multiple (3 to be precise) fields. What operations can I perform to avoid adding duplicates?
3 answers

Even though you sadly do need an extra concatenated attribute for this I think the removing of duplicates from a list can also be achieved by doing a union of the list on itself. That is a bit less cumbersome than the iteration mentioned in the linked answer.


Well you could use the same method as described in

To do this..

  1. First make a new attribute per entry and append the three fields you want to ID on in there.
  2. After having set that for the whole list, use the method in the linked question.


It's not really pretty though, the union-operator sounds like a better option but I can't think of a way how to use it correctly here.


Give the

  • Quick selection of duplicate records (with respect to certain values);

of Appstore app “ListUtils” a try. It is available in 7.23

like it says in their documentation-tab:

Example 4: List clean-up, removing redundant items

You may want to clean-up a list by removing duplicate items (i.e., items with the same attribute value). This can be achieved by using the Filter function with the following filter to get the unneeded redundant items:

<xsl:for-each-group select="record" group-by="@lastname">
  <xsl:if test="count(current-group()) > 1">
    <xsl:for-each select="current-group()">
      <xsl:if test="position() > 1">
        <xsl:copy-of select="."/>