The first author Neil Z Miller is the director of the Thinktwice Global Vaccine Institute, which is decidedly anti-vaccination as a short look at their website will confirm. He also published a series of books on vaccination. This does not mean the paper is necessarily biased, but it is an undisclosed conflict of interest which is not a good sign.
They are also not providing any evidence for causation, the linear regression graphs in your question is essentially the whole analysis the authors performed.
There is an excellent deconstruction of this specific study by David Gorkski on the Scienced Based Medicine blog. I recommend to read the whole article, I will only summarize a few points here.
David Gorski also notes the conflict of interest that I observed when I researched the first author. One aspect he point out is that the authors used only the data for one year (2009) and only for countries with IMR lower than the United States.
Miller and Goldman only looked at one
year’s data. There are many years
worth of data available; if such a
relationship between IMR and vaccine
doses is real, it will be robust,
showing up in multiple analyses from
multiple years’ data. Moreover, the
authors took great pains to look at
only the United States and the 33
nations with better infant mortality
rates than the U.S. There is no
statistical rationale for doing this,
nor is there a scientific rationale.
Again, if this is a true correlation,
it will be robust enough to show up in
comparisons of more nations than just
the U.S. and nations with more
favorable infant mortality rates.
Basically, the choice of data analyzed
leaves a strong suspicion of cherry
picking.
When I saw the graph on the right with the grouped data I was suspicious as I could not see any reason to arbitrarily group the data. It looked like a cheap way to make the plot look better, not like an analysis that would actually provide more insight. David Gorsky shares my suspicion and notes
More dubiously, for some reason the
authors, not content with an weak and
not particularly convincing linear
relationship in the raw data, decided
to do a little creative data
manipulation and divide the nations
into five groups based on number of
vaccine doses, take the means of each
of these groups, and then regraph the
data. Not surprisingly, the data look
a lot cleaner, which was no doubt why
this was done, as it was a completely
extraneous analysis. As a rule of
thumb, this sort of analysis will
almost always produce a much
nicer-looking linear graph, as opposed
to the “star chart” in Figure 1.
As pointed out by Catharina from the Just the Vax blog, the paper also contains an error regarding the german vaccination schedule. The German Childhood Vaccination Schedule recommends additionally Hepatitis B from birth on, as well as MMR and Chickenpox vaccinations starting at 11 months.
There are other studies that examined the association of SIDS (sudden infant death syndrome) and vaccinations, a meta analysis concluded that vaccinations help to prevent SIDS
Immunisations are associated with a
halving of the risk of SIDS. There are
biological reasons why this
association may be causal, but other
factors, such as the healthy vaccinee
effect, may be important.
Immunisations should be part of the
SIDS prevention campaigns.
US infant mortality rate
There is a report from the CDC addressing the high infant mortality rate in the United States: Behind International Rankings of Infant Mortality: How the United States Compares with Europe.
Infant mortality rates for preterm (less than 37 weeks of gestation)
infants are lower in the United States
than in most European countries;
however, infant mortality rates for
infants born at 37 weeks of gestation
or more are higher in the United
States than in most European
countries.
One in 8 births in the United States were born preterm, compared with 1 in
18 births in Ireland and Finland.
If the United States had Sweden’s distribution of births by gestational
age, nearly 8,000 infant deaths would
be averted each year and the U.S.
infant mortality rate would be
one-third lower.
The main cause of the United States’ high infant mortality rate when
compared with Europe is the very high
percentage of preterm births in the
United States.
The conclusion is that the higher rate of preterm infants explains a large part of the higher infant mortality rate, but not the whole discrepancy between Europe and the United States.
The following figure shows the IMR comparison if you exclude births earlier than 22 weeks, the US rate is significantly lower, but still higher than in most european countries.

However, infant mortality rates for
infants born at 37 weeks of gestation
or more are generally higher in the
United States than in European
countries.
The report does not speculate what the source of the remaining difference between Europe and US infant mortality rate could be.
Conclusion
The whole paper looks more like a fishing expedition to me than a thorough and objective analysis. They used an arbitrarily limited subset of the available data and did not correct for any potential confounding factors. This looks suspiciously like they played around with the data until they found the correlation they searched for, especially given the known bias of the authors.