First, you should create a dynamic route (something like "games/glossary/<slug>") and point it to a template. A dynamic route means you won't have to create a template for every letter. Note the use of the <slug> token, which – although not entirely semantic – should do nicely in this case.
In your template, you'll need to pull the letter from the URL. Note that it's also a good idea to test if the variable is a single character, and in the a-z range (i.e. a valid letter, and not some other entity).
{% set letter = slug is defined and slug|length == 1 and slug matches '/^[a-z]/i' ? slug %}
If there is no valid letter in the slug variable, it's a good idea to throw a 404:
{% if not letter %}
{% exit 404 %}
{% endif %}
Next, you can actually use search syntax for the title parameter (specifically, a wildcard *), in order to pull all entries with a title starting with the relevant letter:
{% set entries = craft.entries.section('games').title(letter~'*').limit(null).order('title asc') %}
If you want to throw a 404 whenever there aren't any entries for any given letter, it's simply a matter of testing the length property for entries:
{% if not entries|length %}
{% exit 404 %}
{% endif %}
If you need pagination in your template, it'll work out of the box as usual:
{% paginate craft.entries.section('games').title(letter~'*').limit(10).order('title asc') as pageInfo, pageEntries %}
{% if not pageEntries|length %}
{% exit 404 %}
{% endif %}
{% for entry in pageEntries %}
...