I did some research and found that most power banks (and laptop batteries) use 18650 Li-Ion batteries (cells).
So finally I bought a 18650 battery case (search for "18650 power bank case") which I'm using with a 18650 cell from a disassembled old laptop battery. Currently this one:

Product link and image source: 2600mAh 5V 1A USB Power Bank Case Charger DIY Box For iPhone
Small enough to mount on a bicycle helmet
Its shape fits well to bicycle helmets
It's not perfect, but it seems to be the best option currently.
Mine weights 67 grams (including battery).
Possible/easy to mount on a helmet
With self-adhesive velcro it works fine.
At least 1000mAh capacity
18650 batteries usually have 2600+mAh capacity.
No need to power on, always on (I don't want two handle two power buttons, the one on the camera is enough)
I have not tested it thoughtfully but mine does not have any power button and seems fine.
Easy to mount/unmount (unmount for shorter rides, or using more than one for longer rides)
Velcro solves that.