I have a similar issue. I've just upgraded a 1.7.02 site to 1.8.1.0 and noticed an immediate degradation of speed - page load times up from < 1 second to 5+ seconds.
This site is running on a dedicated server, with .htaccess optimisations as recommended, using a cdn, etc. The server is not the issue, since 1.7 was blisteringly fast - all page types were routinely loading around 750 ms to 1 second.
However, as a test, I swapped out the template I was using for the default Magento template & the speed improvement was like, wow! Everything was as fast, if not faster than the 1.7 version. Restoring our original template resulted in page load times of 5+ seconds again.
So, I'm assuming there's something's not quite right with our template (eventho' the template designer released a 1.8 version), especially the product page, where load times are slowest. Needless to say, the template designer (who has been dodging responsibility to date) has been notified!
Therefore, my suggestion is to do as I did - see if running the default Magento template makes any noticeable difference. If so, then the template may be the origin of your problem.