As others have indicated, you cannot rely on a word's form to deduce it's meaning. Often there is a correspondence. Words are often formed by combining parts in a predictable way. But there are many paths that words can take to arrive in English.
From etymonline:
aver (v.) late 14c., from Old French averer "verify," from Vulgar Latin *adverare "make true, prove to be true," from Latin ad- "to" (see ad-) + verus "true" (see very). Related: Averred; averring.
avert (v.) c. 1400, from Old French avertir (12c.), "turn, direct; avert; make aware," from Vulgar Latin *advertire, from Latin avertere "to turn away, to drive away," from ab- "from, away" (see ab-) + vertere "to turn" (see versus). Related: Averted; averting.
averse (adj.) mid-15c., "turned away in mind or feeling," from Old French avers and directly from Latin aversus "turned away, turned back," past participle of avertere (see avert). Originally and usually in English in the mental sense, while avert is used in a physical sense.
aversion (n.)
"a turning away from," 1590s; figurative sense of "mental attitude of repugnance" is from 1650s, from Middle French aversion and directly from Latin aversionem (nominative aversio), noun of action from past participle stem of aversus "turned away, backwards, behind, hostile," itself past participle of avertere (see avert)
As you can see, there is a reason for the formation of the words, but it isn't necessarily obvious from the English spelling. Notably, aver doesn't belong in the same group as aversion, averse, avert.