6

Here is the way I am using to return duplicate elements.. But I am facing most dangerous performance issues like browser close etc when my array have large number of items with long texts..

var arr = [9, 9, 111, 2, 3, 4, 4, 5, 7];
var sorted_arr = arr.sort();
var results = [];
for (var i = 0; i < arr.length - 1; i++) {
  if (sorted_arr[i + 1] == sorted_arr[i]) {
     results.push(sorted_arr[i]);
  }
}
alert(results);

Please suggest me a best way of doing this

Exception
  • 7,721
  • 19
  • 82
  • 133
  • possible duplicate http://stackoverflow.com/questions/840781/easiest-way-to-find-duplicate-values-in-a-javascript-array – Muhammad Saifuddin Nov 29 '11 at 16:40
  • Are the array keys only integers? Also, how long is your real array and the items in it? – hugomg Nov 29 '11 at 16:40
  • 2
    @Saifuddin Holy crap it's even the same array. This leads me to believe it's a question about consumption, rather than the technique. – Dave Newton Nov 29 '11 at 16:41
  • 2
    Not a duplicate question as much as a question on a solution of another question. Also, this algorithm has a potential bug: it will return N-1 copies of K if the input contains N copies of K. This bug is currently hidden because the test case has no more than two of an item. – Mike DeSimone Nov 29 '11 at 16:57

4 Answers4

6

i don't get exactly what you want, but if you need to return duplicates you could use a cache object. this works with number or string or whatever.

var arr = [9, 9, 111, 2, 3, 4, 4, 5, 7];
var cache = {};
var results = [];
for (var i = 0, len = arr.length; i < len; i++) {
  if(cache[arr[i]] === true){
      results.push(arr[i]);
   }else{
       cache[arr[i]] = true;
   }

}
console.log(results);//returns an array with 9 and 4

Of course you can do other things like deleting multiple items etc. etc.

EDIT - i've written a blog entry on how to remove duplicates from an array

Nicola Peluchetti
  • 74,514
  • 30
  • 136
  • 188
6

If you have array filter, you also have indexOf and lastIndexOf, and you can return the duplicates without doing the sort.

var results, arr= [9, 9, 111, 2, 3, 4, 4, 5, 4, 7];

if(arr.filter){
    results= arr.filter(function(itm, i){
        return arr.lastIndexOf(itm)== i && arr.indexOf(itm)!= i;
    });
}

else// use your loop method

alert(results)

/*  returned value: (Array)
9,4
*/
kennebec
  • 98,993
  • 30
  • 103
  • 125
2

Assuming Nicola's solution doesn't work for you (since it uses about as much memory as the original solution: two elements stored per element in the input, worst-case), you can use the slower process of repeatedly searching your input.

This requires the Array.indexOf method from ECMAScript 5. A lot of browsers have it. For alternatives, see How do I check if an array includes an object in JavaScript?.

var arr = [9, 9, 111, 2, 3, 4, 4, 5, 7];
var results = [];
for (var i = 0, len = arr.length - 1; i < len; i++) {
  if((results.indexOf(arr[i]) == -1) && (arr.indexOf(arr[i], i + 1) != -1)) {
      results.push(arr[i]);
   }
}
console.log(results);

This uses no more memory than the input arr plus the output results, but it's an O(N^2) algorithm and doesn't have to modify arr.

Community
  • 1
  • 1
Mike DeSimone
  • 40,052
  • 10
  • 71
  • 95
1

Your method relies on a sort, which may or may not be one reason you run out of space/time.

The canonical way to remove duplicates is to keep a hash map of the keys (an object in JS). The object keys you get back won't necessarily be in the order you want; you don't specify if you want the results ordered as well, but they are now.

You could null out the original array, since you no longer require it; when it gets collected is up to the JS engine though.

You could remove duplicates "in place" by keeping a "current index" into the sorted array, and increment it only when you move a non-duplicated element "down" from the counter index, then truncate the array that you return.

Combining the last two techniques should mean that in general you'll only have a single array with a valid reference.

Edit Example. Setting length explicitly, as .slice() creates a new array.

var have = {};
var arr = [9, 9, 111, 2, 3, 4, 4, 5, 7];
arr = arr.sort();

for (var rIdx = 0, i = 0; i < arr.length; i++) {
    if (have[arr[i]]) {
        arr[rIdx++] = arr[i];
    } else {
        have[arr[i]] = true;
    }
}

arr.length = rIdx;
console.log(arr);
Dave Newton
  • 156,572
  • 25
  • 250
  • 300