Unfortunately, the only general solution to this problem is to give your users the https:// only and make sure that they expect to use that only. It is ultimately the responsibility of the user to check that they are using SSL/TLS, as they expect.
Other solutions are vulnerable to man-in-the-middle attacks, even if the website only accepts SSL/TLS connections. Attackers could intercept the traffic to http://example.com (as requested by the user, even if example.com isn't even listening on that port) and replace it by making their own connection to https://example.com, proxy-ing it back to the user.
There was an OWASP rule against automatic redirections because of this. It was removed, probably be cause redirections are not a bad way to mitigate the risk (especially against passive eavesdroppers), but don't solve the fundamental problem.
There are various techniques you can use to guide the user to the HTTPS site, and it's not a bad idea to use them (although it won't protect them against active MITM attackers).
Firstly, if you don't have anything that should be served in plain HTTP at all on the webserver, turn off port 80 (e.g. remove Listen 80 in Apache Httpd's configuration). The users will have to use https:// at all times, which may be inconvenient.
Secondly, in your Apache Httpd configuration section for a particular path (either Location or Directory), use the SSLRequireSSL directive: it will require usage of SSL/TLS (even if you've configured it on an alternative port in fact). Other web servers probably have similar directives.
Thirdly, you can use an redirection, either using mod_rewrite or within your code (if it's an application). Something like this should do, for a specific location (see the HTTPS special variable; you can use 302 too, but 301 is better if this is to be more permanent):
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^(samples/.*)$ https://example.com/$1 [R=301,L]
More importantly, make sure that all the links to that secure section use https://. Never rely on the automatic redirection to do the job for you. For this reason, I'd recommend not to use it at all during the development phase.
However, I have noticed that I can still access the website
non-securely, ie. by using http instead of https.
This also sounds like you're using the same configuration for both http and https. If you're using Apache Httpd, I would suggest splitting the configuration into two distinct VirtualHosts: one for port 80 and one for port 443. They don't have to have exactly the same configuration: just don't put what's only for HTTPS in the HTTP virtual host at all.
A way to mitigate the problems mentioned above is to use HTTP Strict Transport Security, for browsers that support it (it applies to the entire host as far as I know). The very first connection may still be exposed if https:// isn't used without the redirection, but it's possible to have a pre-loaded list of sites expecting https:// anyway (and enabled for HSTS).
https://mail.google.com. If, as a user, you see it work withhttp://mail.google.com, there probably is a MITM proxying the requests to the genuinehttps://mail.google.com. Unfortunately, Gmail can't do much about that if the users themselves don't check. Same principle as in real life: if Alice wants to talk to Bob, but talks to Chuck (who claims to be Bob) instead w/o verifying the ID, Bob won't know about this conversation and won't be able to do anything about it. It's Alice's responsibility. – Bruno Apr 12 '12 at 13:25