I know Ancient Latin was subjected to a phenomenon called "rhotacism", which changed some s into r. However, I can't help but ask myself why it happened.
Why did rhotacism happen? Did it influence all words containing s, or not? If not, which types of words were affected?
It's often difficult to say "why" a sound change happened, so I'll focus on your other questions. Rhotacism in Latin happened via a series of sound changes. It only affected inherited *s in specific environments.
The first step towards rhotacism is believed to have been voicing of single "s" to the sound [z] in certain contexts: generally, between vowels. Phonologically, this change can be seen as an assimilation of the consonant to the surrounding voiced segments. This kind of voicing assimilation is common historically; for example, it also occured between Classical Latin and many Romance languages. Double ss remained unvoiced.
In some related languages, such as Oscan, /z/ remained as such between vowels and did not undergo rhotacism. We know this because we have Oscan inscriptions written in the Latin alphabet that use the letter Z for this sound (Buck 1904). (However, my understanding is that we don't have evidence of the letter Z being used this way in Latin inscriptions: see the related question Why are there no native Latin words with a Z?)
The next step of rhotacism (z > r) is a little less common across languages, but we do have some other examples. The process also applied historically in Germanic; one place where you can see the result in English is forlorn, which is historically related to the verb lose. Rhotacism is thought to have occured in Old Latin, and to have been complete by the 4th century BCE.
As a result of these changes, original intervocalic *s ultimately became r in Classical Latin. In the ancestor of Latin, *s seems to have also been voiced to [z] after the voiced resonant /r/, so another related sound change is original *rs to Classical Latin rr. One word that shows this change is terra "earth," from Proto-Indo-European *ters- (“dry”). The cluster rs that occurs in Classical Latin is a simplification of earlier *rtt or *rts; this is why most words with rs also have rt in some forms or in related words (such as ars "art," genitive artis). There also seems to have been voicing of *s after the voiced lateral resonant /l/, but the end outcome of that cluster in Classical Latin was geminate ll.
These two environments (between vowels and after r) are the main cases where Latin r regularly came from earlier *s.
There seem to be some words with r followed by a consonant where the r was originally intervocalic *s, but the following vowel was lost (as in veternus).
There also seem to be some words where *s developed into z and then r directly before a voiced consonant; in fact, the voicing in these words seems to have occurred earlier than the intervocalic voicing. For example, Proto-Indo-European *-sg- seems to be the ancestor of rg in mergo "to immerse, engulf" and probably in virga "twig, switch, rod" (de Melo). This also applied before the semivowel v [w]; thanks to TKR for pointing this out in a comment.
A strange example that looks like it underwent rhotacization between Old and Classical Latin is the word carmen, which Lewis and Short say used to be casmen. This is odd because my understanding is that before a nasal like m, s usually did not undergo rhotacism, but was dropped instead, lengthening a preceding vowel if there was one. De Vaan (2008) says that the r in the form carmen is not derived from a change of -sn- to -rn-, but instead from dissimilation of -nm- to -rm- in a hypothetical form *canmen.
Once the sound r became established in certain words, their derivatives might come to have it in other environments. For example, as Joonas Ilmavirta mentions, some words such as honor developed final r instead of s after a vowel; it's believed this is due to the presence of r in other related forms (like honoris).
Single intervocalic s did exist in Classical Latin; we believe it was pronounced [s]. In many words, it corresponds to inherited *ss, which was regularly shortened in some environments (for example, after a diphthong or long vowel). Other words with intervocalic s have been explained as borrowings from other languages, such as Greek or Italic languages such as Oscan that did not undergo rhotacism. Whatever the reason, their presence indicates that rhotacism was not active as an automatic sound change in the Classical era (Gorman).
Peck summarizes the change as follows:
In Latin rhotacism is a regular law, for though there are many apparent exceptions to it, examination shows that in most of these cases the s was not originally intervocalic or else that the word is of (a) foreign or (b) late origin. [...]
The unvoiced s first passes into the voiced s (z) and thence to lingual r, for the position of the vocal organs in pronouncing z is substantially the same as that required for r.
References:
Why questions in linguistics are the hardest because we can only speculate. That being said, there is plenty of evidence to make an educated guess.
Lundquist 2016 defines rhotacism as “the replacement of a non [r] sound with [r].”
Inervocalic s (a letter, not a sound!) is found “only on the very earliest inscriptions” (Weiss 2009/2011),
e.g. IOVESAT, VALESIOSIO, ESED (all circa 500 BCE); NUMASIOI (700 BCE?).
We don't know for sure how intervocalic s was pronounced in Old Latin but there is good reason to believe it was a voiced fricative, [z], cf. Sihler 1995, who writes that rhotacism in Latin was “doubtless through the medium of [z].”
Thus, we reconstruct the change as
*s > *z > r
Not everyone agrees on whether it happened in Proto-Italic or just affected the Latino-Faliscan branch only. Weiss 2009/2011 argues it was "an areal feature that affected Latin, Faliscan, and Umbrian" (Umbrian belongs to the Sabellic branch).
On the other hand, ancient grammarians were aware of this change too - Varro is the most cited example. e.g.
In multis verbis in quo antiqui dicebant S, postea dicunt R (L.7.27).
As for dating, most evidence comes from texts. For example, Cicero talks about princeps L. Papirius Mugillanus "qui censor cum L. Sempronio Atratino fuit" and we learn about him that he "primum Papisius est vocari desitus."
Another example: in 312 BC "Idem Appius Claudius... R litteram invenit, ut pro Valesiis Valerii essent, et pro Fusiis Furii." Weiss 2010 interprets this passage the following way, "in redacting the senatorial roll in his capacity as censor in 312 BCE Claudius rewrote these nomina, formerly spelled with s, with the letter r."
Taking such evidence into consideration, we assume that rhotacism must have happened before the middle of the fourth century BC,
cf. Niederman 1906 “Or, si nous considérons que, de tous les mots d’une langue, ce sont les noms propres qui se transforment le plus lentement […], nous ne risquerons guère de nous tromper en affirmant que, dans les noms communs, le rhotacisme était un fait accompli des l’année 350 av. J.-C. environ” (p. 74).
Naturally, scholars have always wanted to explain rhotacism phonetically (why does it happen?). As Garrett and Johnson 2011 astutely observe, historical linguistics textbook classify sound changes rather superficially, arguing that “an explanatory classification of surface patterns should reflect a typology of causes” (i.e it should be articulatory or acoustically oriented). They explain rhotacism with the help of aerodynamic constraints (a bias against voiced fricatives); in other words, voiced fricatives tend to become glides.
Cf. Solé 1992, who argues that z > r is due to “hardware constraints which are built into the physical system used for speech” or Painter 2012, who concludes that “rhotacism is the result of the listener reanalyzing phonetic realizations of a target /z/ in the speech signal as belonging to /r/.”
Naturally, rhotacism didn't happen simultaneously in all contexts - that is why there are numerous exceptions - below is my summary of exceptions, based on Baldi 2002: 285-290; Leumann 1977 and Sihler 1995, §172-173.
Leveling, e.g. arbor;
Borrowings (they were borrowed either later or from non-rhotacizing donor languages), e.g. basis, casa, Musa, rosa etc.;
Words that originally had ss: causa, formosa, quaeso, visus, usus etc.;
Next syllable begins with r, e.g. Caesar, miser (but: soror, aurora) etc;
Derivationally transparent compounds: e.g. desino, nisi etc.
A pretty common view to rhotacism is to treat it a multi-stage, diachronic process (starting with Touratier 1975; also see Baldi 1994). The best description of rhotacism I'm aware of is Roberts 2012. His model has five stages, starting with stage 0 - he calls his a phonetic tendency - when "co-articulatory pressures cause intervocalic voiceless consonants to tend to be realised as voiced" and concluding with stage L, when rhotacism became "a systematic property of the lexicon, subject to extension by analogy" and when rhotacism was "no longer productive over new VsV sequences, such as in loans or those created by the degemination of ss."
Sihler 1995 observes that rhotacism may seem “extreme” phonetically but it is attested in many languages: in West Germanic languages and in Old Norse (Ringer and Taylor 2014, pp. 82-88), in some Greek dialects (Elean, Euboean, Eretrian etc.; Lundquist 2016) or in some Spanish dialects (Solé 1992).
Baldi, Philip. 1994. Some Thoughts on Latin Rhotacism. General Linguistics 34: 209-216.
Baldi, Philip. 2002. The Foundations of Latin. Berlin: De Gruyter.
Garrett, Andrew, and Keith Johnson. 2011. Phonetic bias in sound change. UC Berkeley Phonology Lab Annual Report.
Lundquist, Jesse. 2016. "Rhotacism." Encyclopedia of Ancient Greek Language and Linguistics. Brill Online.
Leumann, Manu. 1977. Lateinische Laut- und Formenlehre. München: C.H. Bech.
Niedermann, Max. 1906. Précis de phonétique historique du latin. Paris: C. Klincksieck.
Painter, Robert Kenneth. 2012. Acoustic and perceptual explanations for rhotacism in Latin and Germanic. Doctoral dissertation.
Ringe, Donald A., and Ann Taylor. 2014. The development of Old English. Oxford: Oxford University Press.
Roberts, Philip J. 2012. Latin Rhotacism: A Case Study in the Life cycle of Phonological Processes. Transactions of the Philological Society, 110(1): 80–93.
Sihler, Andrew L. 1995. New comparative grammar of Greek and Latin. Oxford: Oxford University Press.
Solé, M.J. 1992. Experimental Phonology: the case of rhotacism. In W. U. Dressler, H.C. Luschützky, O. E. Pfeiffer & J. R. Rennison (eds.), Phonologica 1988, Cambridge: Cambridge University Press, pp. 259-271.
Touratier, C. 1975. Rhotacisme synchronique du latin classique et rhotacisme diachronique. Glotta 53.246-281.
Weiss, Michael L. 2009/2011. Outline of the historical and comparative grammar of Latin. Ann Arbor: Beech Stave Press.
The reason for rhotacism is I believe ascribed to the fact that there was only one sibilant in latin - /s/, and as such it was not pronounced exactly [s] as in today's English or French. These two languages have more sibilants and in the terms of position, they distinguish two - /s/ and /š/. Latin, with only one position, is thought to have /s/ phoneme similar to the Castilian Spanish one (again only one sibilant), which is pronounced apically (with the tip of the tongue) and sounds perceptively as something between the two.
As explained in detail in other answers, the /s/ became sonorous in certain positions, so it was pronounced as apical [z] (perceptively, this would be something between [z] and [ž]). This sound is not very far from the way /r/ is pronounced by the British. It is unknown how exactly PIE or early Latin /r/ was pronounced but there is comparative evidence which suggest it might have been slightly retroflex and not a vibrant, so if that were the case, there would be ample opportunity for the latin sonorous apical /s/ to be confused with the /r/ and essentially merge in the same phoneme.
This is a partial answer. I can only tell which words are affected, not why this phenomenon happens.
My understanding is that s turned into an r between two vowels: amase > amare, honose > honore, corposis > corporis and so on. It has also changed in some word-final positions by analogy to declined forms: honos > honor but corpus did not become corpor. Notice that this only applies to a single s between to vowels (and occasional word-final s's), not to an initial or double s. In classical Latin we still say sum, amavisti and amavisse, not rum, amavirti and amavirre.
