Should you be worried? I don't think so, especially if she's empathetic in other situations.
Empathy for squishable things is tough at that age, because squishing feels good, and having power over something - the power of life and death, in this case, even if she doesn't fully comprehend death and it's finality - is fascinating to a preschooler.
As adults, an insect's suffering and death is much more meaningful because we understand it. She really doesn't.
That doesn't mean you should not attempt to change her behavior, though. I would. Here are a few ideas:
Make sure she has a rich emotional and physical vocabulary. Ask her what she thinks the insect felt when she did that. Ask her what she felt when she squished it. Ask her what she thinks you felt when she squished it. Ask her what she thinks you felt when she agreed to be gentle with the insect but was not. The first step to recognizing emotions is to have a name for them.* (Don't add a value judgement to her words, though. You want to understand her thinking, not to make her feel bad, and besides, if she knows you don't approve of something, she's less likely to be truthful about it, preferring to give you the answer she thinks you want.)
Stop giving her insects to play with. If she asks for it, remind her that she hurt the insect the last time you gave one to her. Model the behavior you want her to emulate: gentle handling and setting the insect free.
Ask her what kind of adventures the insect in your hand is going to have. Maybe make some up. That might help her to relinquish the idea of squishing it to death.
Read stories with insects as the main characters. If you can't find many (which would be surprising), make some up. Anthropomorphizing insects might give her more empathy for them.
Give her a caterpillar to raise. It's easy, fun, short, and exciting to see a butterfly emerge. Caring for something - even a plant - is an empathy building exercise.
What I strongly urge you not to do is ask her if she's good when she has just done something wrong. That is more likely to make her feel unlovable. Kids that age are very prone to bifurcation: they are either good or bad, and it's kind of like hearing ten compliments and one insult: you'll remember the sting of the insult long after you've forgotten the compliments. We aren't good or bad based on one action.
*There are lots of age appropriate emotion vocabularies on the internet. Aim a bit higher than the age listed; I think many aim a bit low.