It depends on the situation. Is this the tenth time Smith has pointed you at new evidence, and the other nine times you investigated the new evidence only to find it irrelevant or actually harmful to Smiths case? Investigation can be endless. If you suspend judgement until all possible evidence is in, then you might never be able to come to a conclusion.
Furthermore, some beliefs are more important than others and require greater investigation. I hear a particular tune playing outside. Given my experience of the neighborhood, I conclude that the ice cream truck is driving by outside. Now, it doesn't have to be the ice cream truck; other vehicles and devices can play that tune, but I'm justified in drawing a conclusion without going to look, in part because there is no reason to think it might be anything else, and partly because it's not important enough an issue to investigate thoroughly.
If someone were to tell me that they don't think it sounds like the ice cream truck, I might go and take a look out of curiosity. If someone's life depended on me being right about my conclusion, I'd go take a look.
So, there are at least three reasons not to gather additional evidence to draw a conclusion, even if additional evidence is available:
- You have reason to doubt that the additional evidence will be of use.
- You have practical constraints such as a deadline and have to conclude your investigation without covering all possible evidence.
- Given the circumstances, the matter is not important enough to warrant further investigation.