Optimize Linq Contains method

Question

I need a list from OldGuids that are not in the NewGuids, so I used the Contains method, the problem is that it is already running more then half hour, is there a faster way? or how much longer would it take about?

Dim OldGuids As New List(Of Guid) ' 18 million rows
Dim NewGuids As New List(Of Guid) ' 6 million rows
Dim Filtered = From n In OldGuids Where Not NewGuids.Contains(n)

April 1st? It is taken: http://stackoverflow.com/questions/1705008/simple-proof-that-guid-is-not-unique — Hans Passant, Apr 01 '12 at 15:19

score 3 · Answer 1 · answered Apr 01 '12 at 15:18

3

You should use HashSet<Guid>s.

You can then write OldGuids.IntersectWith(NewGuids)

answered Apr 01 '12 at 15:18

SLaks

837,282
173
1,862
1,933

`IntersectWith` doesn't help here. You missed a `not` in the question. – CodesInChaos Apr 01 '12 at 15:36
The method that should be used is `ExceptWith`, not `IntersectWith` – Meta-Knight Apr 01 '12 at 16:12

CodesInChaos · Accepted Answer · 2012-04-01T15:56:19.323

2

Filtered = OldGuids.Except(NewGuids)

(Note that this will only return unique elements, if you want to preserve duplicates this is not what you want).

var newGuidSet=new HashSet<Guid>(newGuids);
Filtered = OldGuids.Where(g=>!newGuidSet.Contains(g));

edited Apr 01 '12 at 15:56

answered Apr 01 '12 at 15:28

CodesInChaos

103,479
23
206
257

will it be much faster then contains? – Ezi Apr 01 '12 at 15:34
It will be fast, but take about 400MB or RAM. – CodesInChaos Apr 01 '12 at 15:36
I get system out of memory exception, there is plenty of space on the hard drive – Ezi Apr 01 '12 at 15:42
I've added a second solution which requires a bit less memory. If that isn't enough, switch to 64bit and download more RAM. – CodesInChaos Apr 01 '12 at 15:56
Making `NewGuids` a `HashSet` is enough for your specific use, and saves a bit of memory. – CodesInChaos Apr 01 '12 at 15:59

Optimize Linq Contains method

2 Answers2