I believe that your father could potentially be putting your newborn at risk, yes.
Here's why:
According to a 2-year survey, 64% of travelers to 3rd world countries develop some sort of illness. 26% of those travelers were ill once they returned from the trip, with 56% of those travelers' illnesses starting after the return (9.3% of all travelers).
Thus, roughly 1/10 of people who travel to 3rd world countries will return to their home country with an illness that may be contagious, but isn't currently presenting any symptoms.
Nearly half of those reported illnesses were diarrhea, which is contagious. Even if you flush, and wash your hands, diarrhea contaminates many other surfaces by aerosol.
If your father visits and has an episode of diarrhea, he risks spreading the illness to any member of the household. If you or your wife catches it, then there's a high likelihood that the rest of the house will as well. You may not actually get ill until after your father's visit, because of incubation times, just as he may not be sick until after he returns.
Since your father is specifically visiting a hospital, his risk of catching something is actually greater.
Typically, an infant is going to be exposed to some contagions no matter what. However, I can't think of any other situations where I could concretely say there's as high as a 1 in 10 chance that a visitor may be bringing contagions that no one in my household has been exposed to.
All that being said, there's no telling how severe any spread illness would be, or that you could tell it apart from any of the myriad of illnesses your infant could catch anyway. However, from personal experience, while infant illnesses may not negatively impact the overall health of your child, they certainly impact other parts of your life. Feeding and sleeping schedules are often immediately affected and can take much longer than the baby's health to get back to normal. When you're already sleep-deprived from having a newborn in the house, losing even more of your own sleep to take care of your sick child is brutal.
I don't think it's unreasonable to ask someone who's coming from overseas (even without the stop in a developing country) to wait until after the 6-week mark to visit your newborn.
You're not being selfish by trying to keep your child healthy. Your father is going out of his way to visit sick children! He of all people should understand the importance of illness prevention.
You're not saying he can't visit, you're just asking him to respect your timetable about when he can visit. He may be doing morally-positive charity work, but morally-positive behavior isn't immune to negative natural consequences (unfortunately).