Moonbeam triggers on entering or starting the turn in the area
If you move a creature out of the area and back in, it is entering the area for the first time that turn and will take damage. So yes, you can use this method to damage them.
There is no sentence that states you can not have been in the area before when it comes to entering. After you moved them back in, they then will likely take damage another time at the start of their own turn.
I think this is not overpowered -- you are using your action to grapple them, which can fail, for a chance to deal 2d10 damage to them, instead of directly attacking them. At least in tier 2, an attack with your primary weapon is likely going to deal a similar amount of damage.
The Sage Advice Compendium has some additional language, although it does not directly adress your point:
Entering such an area of effect needn’t be voluntary,
unless a spell says otherwise. You can, therefore, hurl a
creature into the area with a spell like thunderwave. We
consider that clever play, not an imbalance, so hurl away!
Keep in mind, however, that a creature is subjected to
such an area of effect only the first time it enters the area
on a turn. You can’t move a creature in and out of it to
damage it over and over again on the same turn.
In summary, a spell like moonbeam affects a creature
when the creature passes into the spell’s area of effect and
when the creature starts its turn there.
P.S. Note that the rule for Grappling (page 195 PHB) says:
Moving a Grappled Creature. When you move, you
can drag or carry the grappled creature with you, but
your speed is halved, unless the creature is two or more
sizes smaller than you.
Drag is defined in the dictionary as
pull (someone or something) along forcefully, roughly, or with difficulty.
So you cannot push them in front of you, you need to pull them after you, or you would need to be strong enough to carry them. If you pull them (and possibly even if you carry them), it may be that your DM will rule you need to enter the area yourself to move them there, thereby also taking the damage yourself (although, dragging them sideways along may work).