It really depends on what you're trying to achieve. If you want the modulation to be sudden and jarring, then you might choose to modulate to a key area (or new mode) with minimal common tones.
But if you're simply going for ease of modulation, then yes: by and large, the easiest modulations are effected with two key areas (or modes) that share the most common tones.
If you're in F Lydian, for instance, you could modulation to A Dorian pretty easily since you're only adding a single sharp.
With that said, in my experience it's easier for students to modulate when there is at least one new accidental. Modulations where there are no new pitches (like F Lydian to G Mixolydian) can often be really tricky to write effectively.
Lastly, this rule—that more common tones makes it an easier modulation—has one main caveat. The so-called "truck driver modulation," that just moves up or down by half or whole step, is very easy to do without any preparation at all. You just move transpose the music into the new key; it's really as simple as that. It's a little jarring, but it's so popular (dare I say overused...) these days that it no longer sounds surprising to us.