Ok, in htaccess you should have a mod_rewrite section that starts with <IfModule mod_rewrite.c>
Soon after that there should be a line that says RewriteEngine on
After that you need to put the following:
RewriteCond %{HTTPS} off
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
Because the secure url and unsecure base urls are the same in Magento admin it is redirecting all traffic to https but it is redirecting traffic to direct URLs to the base URL as you are experiencing. The rule in htaccess will make sure that they are just redirected to the same url but over https.
Please try this on a dev site first.