Symmetry has become a central concept in mathematics. The Euclidean concept of similarity is an example of symmetry, but similarity was not a subject of study in itself.
Q: How did symmetry come to take centre stage and become the subject of study in itself? In particular, what results led mathematicians to focus their attention on the concept of symmetry.
The traditional answer to questions on symmetry would be to point to Felix Klein's Erlangen Program as a way of systematizing the study of symmetry by focusing on the symmetries of a manifold as the essential feature thereof. Earlier sources include of course Galois' group theory, which can always be interpreted as a study of symmetry. In a multicultural vein, you would find the Mayas and the symmetries on their clay pots; modern historians are good at this sort of thing though I personally am sceptical; see this review.
In my opinion, there are two separate questions being asked here. The expectation that the answer should concern 18th-19th century was not formulated in the statement of the question(s), although one can say that this is when mathematicians started to `` focus their attention on the concept of symmetry." At any rate, imposing such a time frame leaves out many important developments. So let me supplement Mikhail's answer with earlier examples and possible motivations (from traditional Western culture).
So "How did symmetry come to take centre stage and become the subject of study in itself?" Two motivations emerged in antiquity: the study of nature and aesthetics, sometimes intertwined. In Plato's dialogue "Timaeus" (to which I linked in an earlier comment) an early cosmological model (so to speak) is offered. Plato seeks to explain the order and beauty observed in the nature. His explanation is that humans, animals, heavenly bodies etc. are orderly and beautiful because they are made out of orderly and beautiful `"particles" by the divine Craftsman (Demiurge). These "particles" are four Platonic solids: tetrahedron, octahedron, icosahedron and cube, corresponding tho the four "elements" considered by earlier philosophers. The dodecahedron can be thought to approximate the shape of the universe, which is a sphere that moves in a circular motion (most symmetric, hence most ``perfect"). Formation of physical matter and phase transitions reduce to the composition, decomposition and exchange of triangles into which the faces of the Platonic solids can be divided. Hence the study of nature and universe can be based on the study of symmetric solids and triangles (I am just summarizing Plato, not formulating my own credo, in case someone wonders).
The Roman architect Vitruvius https://en.wikipedia.org/wiki/Vitruvius emphasized the importance of symmetry and proportion in architecture, which to him was imitation of nature and therefore should reflect the patterns of the cosmic order. Moreover, architecture should be useful to the human, whose body is the ultimate work of art. Vitruvius's ten books on architecture inspired early modern architects,especially Leonardo da Vinci and Andrea Palladio. The picture of the Vitruvian Man by da Vinci is a well-known image, https://en.wikipedia.org/wiki/Vitruvian_Man
And long before Galois theory there was substantial interest in symmetric functions, especially in relation with roots of algebraic equations. The names and results of Vieta, Girard and Newton are best known. Many more details can be found in the following article: H. Gray Funkhouser:A short account of the history of symmetric functions of roots of equations, American Mathematical Monthly, 37 (7) (1930), 357–365: http://www.jstor.org/stable/2299273?origin=crossref&seq=5#page_scan_tab_contents
On my opinion, there are two things which triggered this process in 19th cetury:
Galois theory (and the work of his predecessors, like Lagrange and Cauchy). They introduced the notion of group.
Work on crystallography (second part of 19th century) where groups are explicitly used to develop a physical theory.
Of course the study of symmetry is much older than that but only in 19th century, because of the evolution of the abstract notion of group it became a central part of mathematics.
EDIT. Then in the early 20th century group theory was applied with the great success to understanding quantum mechanics and elementary particles, and from that time it was clearly realized that symmetry also plays the fundamental role in the physical laws.
As a side remark: I recommend warmly the book: "From Summetria to Symmetry: The Making of a Revolutionary Scientific Concept" by Hon\Goldstein, which deals with the emergence of the concept of symmetry till the beginning of the 19th cent.
