A natural 20 is always a hit, but it's only a critical hit if it would otherwise have (numerically) hit the target.
The Rules Compendium does spell this out, as it's a consolidation of the 4e core books plus errata, but it's also included in the Player's Handbook.
Player's Handbook, pages 276 and 278
Automatic Hit: If you roll a natural 20 (the die shows a 20), your attack automatically hits.
Critical Hit: If you roll a natural 20 (the die shows a 20), your attack might be a critical hit (page 278).
When you roll a natural 20 and your total attack roll is high enough to hit your target’s defense, you score a critical hit, also known as a crit.
Natural 20: If you roll a 20 on the die when making an attack roll, you score a critical hit if your total attack roll is high enough to hit your target’s defense. If your attack roll is too low to score a critical hit, you still hit automatically.
Only attack rolls have this critical exception. A natural 20 doesn't have anything special for skill checks, saving throws, or death saving throws. Death saves usually don't have modifiers though, and a 20+ lets you spend a healing surge, so natural 20 will usually get you that.