You are mistaken, the verb losgehen simply means to start. But be careful:
… und dann ging das Theater wieder los.
… and then the hassle reran.
But your question was about the fixed phrase
Los geht’s!
Consider
Auf geht’s!
No one would assume this is related to the verb aufgehen, and it's only coincidence Los geht’s! is actually related to losgehen. There's not much grammar in German prefixes. Prefixes are prepositions or adverbs who married a verb but sometimes those tiny words are just what they used to be – prepositions or adverbs.
Was hast du mit der Tür gemacht?
Auf habe ich die Tür gemacht.
Same here. The question isn't about whether you opened or closed the door but what you have done to it. For example, it may be bent by your ill attempt to open a double-winged, locked door. If you wonder, this deadpan style of answering is typical German humor – to overcome an awkward mistake or shortcoming.
That auf isn't a prefix, but an adverb and it is put at the same position as was in the question by purpose: to make it the topic.
Outside of such a situation, everyone would just say:
Ich habe die Tür aufgemacht.
The meaning is the same.