You are right that the use of "can" (or "be able to") is more extended in English than "poder" is in Spanish.
I'd say that "poder" can be omitted when the verb in Spanish already suggests an ability. Actually, it does happen that "can" is sometimes redundant in English and can be replaced with the present tense of the main verb:
- I can't understand = I don't understand.
With verbs implying physical perception or some mental process, "poder" would be a little too emphatic in Spanish, meaning something like: Even if I try to, I can't.
- No te oigo.
- No puedo oírte. (Sorry but, no matter how hard I try, I can't hear you.)
(Please notice that, even if its use is extended, "escuchar" is equivalent to "listen to" and involves a voluntary action, so it would be most impolite to tell somebody that you don't want to hear them.)
- No te veo.
- No puedo verte. (It's too dark and, even if I try to make you out, I just can't.)
To express that effort, we can also use other verb phrases like lograr + infinitive / conseguir + infinitive / alcanzar a + infinitive.
- No logro / consigo / alcanzo a verte.
- No logro / consigo / alcanzo a oírte.
Sometimes, instead of "poder" we use "saber", and that is when a specific skill is being dealt with, as in:
- No sé nadar (I can't swim).