This answer provides some historical context, but does not completely answer the question.
Baden & Württemberg experienced large emigrations in the 1820s to 1850s, to Hungary (recruited by Austria-Hungary to re-settle lands recovered from the Ottomans, in the Banat and Transylvania) and to North and South America. The aftermath of the Napoleonic wars and several years of bad harvests are usually given as causes. These were not expulsions, but desperation is not exactly voluntary either (as evidenced by some of the published diary excerpts).
Some who went to Hungary became disenchanted and returned, but, having exhausted their resources were not always welcomed back:
Example: "Zipf Sebastian Billigheim Krautheim 1723 IV 65 ausgewandert, kömt wieder zurück, wird der geringste Unterhalt nicht gestattet." (Kurmainzische Auswanderer nach Ungarn). However, these returnees were individuals and families, not whole villages.
Many periodical articles and books deal with these emigrations, usually from the viewpoint of the end destination or a particular individual. (Check your favourite bibliographic or library catalog site for examples). Google "Baden-Württemberg emigration" and similar search terms, for websites, some with name databases.
The Zipf surname is still heavily concentrated in southwest Germany: Geogen shows that ~10% of all the Zipf households in Germany today live in Ortenaukreis, Baden-Württemberg, with another concentration in Main-Tauber-Kreis and the neighbouring Neckar-Odenwald-Kreis.