There was fairly little mixing between European settlers and the first peoples in what is today USA and Canada. There was extensive mixing in what is today Mexico (modern day: 65% Mestizo, 17.5% Amerindian, 16.5% White, 1% Black, Asian, or other, source: wiki: Mexico. Argentina had very little mixing (modern day: 97% European, 3% Mestizo, source: wiki: Argentina ).
Are there existing theories, literature?
Here are my rough ideas, (disclaimer: not backed up with research!)
Gender balance of settler community
If a settler community was considered a family destination (like the USA), then European men and women moved and had children with each other. Settler communities that were composed primarily of men would mate with women from the first peoples.
Religious conversion and development of new national identity
Shared religion and integration into a shared national community would make mating more likely. There was large-scale conversion to Catholicism among first peoples in Mexico. There was not large-scale conversion to the varieties of Christianity among first peoples in USA/Canada. Co-religionists would make more feasible mating partners. What is more, co-religionists could be more likely to develop a shared sense of national identity (e.g., as "Mexicans") than a people following very different religions. Having different senses of national identity would make people less likely to mate (e.g., in Canada and USA).