Keeping your storm windows closed in the summer will result in net energy savings due to reduced ventilation.
EDIT: model updated to eliminate cooling load during the winter.
Using eQUEST, a building energy simulation software based on research from the U.S. Department of Energy, I built a simple model of a roughly 500 m2, two-story home with 15% glass on the southern (equator-facing) side:

I then modeled three different changes caused by closing the storm windows to see how they affected total energy use for space cooling and space heating:
- Modest improvement in U-value of the southern-facing windows. Change from 0.50 to 0.38 (U-value is roughly the inverse of R-value). This would tend to reduce the rate at which thermal energy passes through the windows (so heat stays inside in winter, and outside in summer).
- Modest increase in solar heat gain coefficient. Change from 0.4 to 0.5. This is the hothouse or greenhouse effect, meaning that more of the sun's energy will be turned into heat inside the house.
- Significant reduction in ventilation. I assumed that the baseline ventilation rate is about 0.50 m3 per person, meaning that for every person in the house, 0.50 m3 of air is exhausted per minute. The closure of the storm windows is modeled as a reduction to about 0.25 m3/minute.
Here's the resulting effect on energy use for cooling:

Note that the measures are cumulative, so the final measure (in gray) includes the effects from the first two.
During the summer, the reduced ventilation reduces the need for the AC. The interesting thing is that in the shoulder seasons (particularly April and October), this increases the need for AC. Thus during the spring and fall it makes sense to open the windows in the morning and evening when the ambient air is cool.
And the same chart for heating:

Here all three measures result in energy savings, as you'd expect. Note that the scale here is an order of magnitude larger -- meaning that keeping the storm windows closed saves a bit of energy in the summer, and a lot of energy in the winter.
Of course, as the saying goes, all models are wrong, but some are useful. Your best bet would be to test this yourself over a period of a few days, if you've got access to look at your electric meter reading periodically.