Everyone has an accent or dialect, and some even have a speech impediment like a lisp. There is no perfect way to pronounce a language. As long as you can put together the right syllables in the right order (with the right intonation), you can be understood.
It would be no different if you were an American traveling to small-town Ireland or Scotland or some Cockney dialect in England. All regions slur their words into something that works and evolves for them.
Therefore, you should not be discouraged from learning Spanish simply because you cannot notice the subtle differences in similar sounding words. Furthermore, the context of the speech should allow you to make the best guess as to what they are saying. For instance,
Pongo la comida en mi boca. If you hear the sentence and think for some odd reason that the speaker said Pongo la comida en mi poca, then you need to practice your Spanish more and more. Context is everything.
I come from a very redneck, mountainous, Appalachian area in the United States so I have heard A LOT of mispronunciations throughout my life, some of which I doubt any foreigner would catch onto. For example, I often hear the old-timers around here say Nary for Not even, Ni siquiera. https://www.google.com/search?q=nary -- which could easily be confused for Merry, something completely different.
The more regions you visit (Spanish speaking) and the more dialects you communicate to, the more you will recognize the subtleties and differences, like how Castillian Spanish in Spain makes a 'th' sound for the 'S' sound.