It was lingual fun.
The trend developed in the middle of the 15th century and one of the first such lists occurs in The Bokys of Haukyng and Huntyng; and also of coot-armuris better known as Boke of Seynt Albans or The Book of St. Albans printed 1486.
That it also contains such entries as "a doctrine of doctors", "a disworship of Scots" and "a gaggle of women", "a sentence of judges" and "a fighting of beggars" shows that this was something people had fun with from the early days of the form.
It provides a form of idle learning; the pointlessness of knowing such collective nouns is where the charm lies. It is telling that such uses largely died away, but made a revival in the 19th century.
Likely they originated in the earlier distinctions of e.g. using flock for sheep and goats but herd for cattle and deer, and was then taken to further lengths.