As in my question, the following dictionary definitions suggest that the term camp follower has devolved into a derogatory term synonymous with prostitution.
Webster:
a civilian (as a prostitute) who follows a military unit to attend or exploit military personnel
Wiktionary offers a fence-sitting definition albeit one leaning towards the derogatory a wee bit:
A civilian who works for a military organization, often a prostitute.
While there are plenty of references in Google Books that only include prostitutes as a subset of camp followers, there are a number of others such as the ones below that either equate the two or suggest that they are commonly equated with each other.
From A to Z of Women in World History by Erika A Kuhlman, PH.D.:
The tradition of camp followers dates back at least as far as the Middle Ages (800–1300). Most camp followers were women who wanted to join their husbands in battle. Typically, camp followers performed wifely duties at camp, such as cooking meals, washing clothes, obtaining water supplies, and treating wounds. Generally, wives took care of their own families (they often brought their young children with them), not of the whole company. The term camp follower also represents the women who travelled with the military and worked as prostitutes.
From Women In Early America: Struggle, Survival, And Freedom In A New World by Dorothy A. Mays:
To modern ears, the term camp follower implies a derogatory connotation far from the actual experience of camp followers during the American Revolution.
From Social Disorganisation by Rajendra Kumar Sharma1:
Camp followers, in the broad sense, are simply prostitutes who gather and operate where large numbers of soldiers, and other servicemen are stationed, and who serve principally the sexual needs of those males, although civilians are seldom discriminated against.
From Perspectives on Sex Crime & Society by David W. Selfe, Vincent Burke:
Following emergence from the Dark Ages, and the commencement of feudalism in medieval Europe, prostitution once more began to thrive. This was seen especially in the rise of the so called 'camp followers'—large groups of prostitutes who travelled with pilgrims, went on Crusades and marched with a variety of armies cross Europe. These camp followers, like a macrocosm of the single prostitute, were fulfilling the functional needs of a variety of temporarily misplaced males.
From Sex in China: Studies in Sexology in Chinese Culture by Fang Fu Ruan:
Other scholars suggest that institutionalized prostitution in China began in the Western Han dynasty, when the famous Emperor Wu (reigning 140–87 B.C.) recruited female camp followers for his armies; these women were called "ying-chi" ("camp harlots") (Chen, 1928; van Gulik, 1961).
From Encyclopedia of prostitution and sex work: A-N edited by Melissa Hope Ditmore:
"Hog ranch" is a term that was used particularly in Wyoming and parts of Colorado for combination saloons/dance halls/brothels serving military outposts in the 19th century. A limited number of women were permitted association with the daily workings of the camp, and most camp followers (prostitutes) were excluded from the post proper.
From Prostitution and Sex Work by Melissa Hope Ditmore:
Camp followers, including laundresses and seamstresses, typically sold sex or offered situations akin to serial marriage at the garrisons.
From Hotel Ritz: Comparing Mexican and US Street Prostitutes by David J. Bellis:
Various terms have been used to label commercial sex workers, including harlot, hooker (after the "camp followers" of General Joseph Hooker during the Civil War), prostitute, whore, stroller, and streetwalker.
From Maneuvers: Intl. Politics of Militarizing Women's Lives by Cynthia H. Enloe:
The commander's periodic purge required the discrediting of women who followed and serviced the troops, for it was far easier for commanders to send the women out of camp if they could be portrayed as rootless, promiscuous, parasitic, and generally a drag on the military's discipline and battle readiness. Thus camp follower was commonly equated with prostitute.
Ugo Pirro's 1958 novel is about a WWII Lieutenant Martino and his men who are assigned to lead a group of prostitutes through the mountainous ways to serve in brothels for Italian soldiers in Albania. It is titled "The Camp Followers":

The book was adapted into a film in 1965 and released to English audiences as "The Camp Followers".
In summary, even if the term camp followers is not ubiquitously synonymous with prostitutes, there's a significant sub-section of the English-speaking world which does. The presence of entries in respectable dictionaries further confirms this fact.
1Curiously, the exact same passage also appears in a book titled Urban Sociology by an N. Jayapalan.