Greetings, thread-commenters of the past, and whoever lands on this page in the future from a Google search for "Firefox randomly sets always-on-top" just as I did!
If it's any help, I still, after many years, 3 versions of Windows and a gazillion versions of Firefox, haven't figured out why FF periodically goes to AoT mode (it's not my extensions, of which I just have the usual ones.. uBlock Origin, Ghostery etc) but I do have a neat & robust little workaround that I can share with you all.
I wrote this short AutoIt script. It lurks in the system tray, and every 10 seconds it checks to see if the FF window exists, and if so, sets it to Not-Always-On-Top. If the window doesn't exist (i.e. Firefox isn't running), it pops up a brief system-tray tip to let you know the script is still active.
(If you don't have it, the AutoIt3 compiler to turn this into an EXE is available for download here:
https://www.autoitscript.com/site/autoit/downloads/)
Here's my script:
Opt("WinTitleMatchMode",2)
While (true)
If (WinSetOnTop("Mozilla Firefox","",0)) = 0 Then TrayTip("Unset 'FF Window On Top'", "Didn't find Firefox.",2,18)
Sleep(10000)
WEnd
Note: the ,2,18 values at the end of the third line are, respectively, the length of time for the "didn't find Firefox" tray-tip to linger, and the icon-type (yellow warning triangle in this case). You may wish to tweak them to make the script less/more intrusive.
The "Mozilla Firefox" value in the 3rd line is the unchanging part of the FF window title - at least, on my en-US version of FF 79.0 - but I can imagine that this, too, might need to be changed for non-English or other variants of Firefox.
The (10000) value on the 4th line is the interval in milliseconds between checks - i.e., with this value the script will check-and-reset the AoT-ness of Firefox every 10 seconds. Although the CPU load of this script is miniscule, I still instinctively feel it would be a bad idea to have it obsessively checking Firefox too often!
I hope this helps someone else with this issue.