As Groody suggested, there's a lot you can do to make the room good, but there are two more steps you can do for handling that can aid in preserving them.
UV light is the enemy
UV light, like from a window or an incandescent lamp, is the enemy of pulp paper. If you ever put a cheap newspaper into the sun for a few weeks, you'll notice that the paper got quite stiffer and after a few months the paper color will change. This is the paper chemistry working, accelerated by the sunlight and warmth. So, we should reduce that exposure to the minimum.
Because of this aging from UV light, you should replace the lighting with LED, which does not produce UV light and also is more friendly for the environment, as such bulbs draw less power.
Further, you should store the books in a way that is protected from sunlight - for example by just using a windowless room (like the basement) for your gaming den, placing the shelf on a wall that doesn't get hit by daylight or by using a cabinet door that prevents daylight to fall onto the books.
The best preservation is not to handle them (carelessly)
The sad part is, that whenever you handle a book, you harm it. So the first part of preservation is to limit access to it.
Wear gloves
I have had the honor to get access to files from about 1900 in a couple of archives. Some of these papers started to yellow and were in various active states of very slow decay, despite being stored in a dry and lightless cellar on proper shelving. Most of those I was still allowed to handle with bare hands, but there was one volume of old newspapers that had suffered from some additional damage over the last century. For this volume, I was handed a pair of white gloves to prevent fingerprints from accelerating the aging of the paper.
A book of collected weekly newsletters from that time was in an even worse condition and I was denied access to this at all unless I could make a very good case to even see it. Instead, I was given a digital copy of the contents by the librarian. This brings us to the other handling damage reduction method:
Digitizing helps to access without handling
When you have a rare book, you could digitize it to have the information available while not having to remove the book from its storage. This could either be done yourself using a scanner or camera and shooting lots of photos with a tripod will make a rather huge file. It is generally better to use a special camera machine (Some libraries have these, or you might hire a professional digitizing service) to create a digital copy of the work. The biggest hiccup is, that many services and machines do not run an OCR on the resulting picture and this generates picture PDFs. These are not searchable, making them sometimes clunky, and the quality can vary in the same PDF.
Also, remember to stay within the law: you may only create copies for some purposes, for example archiving, but not others, like distributing. Check your copyright laws.
However, Digitizing can also be as simple as acquiring a file-sized optimized and almost always OCR-treated version of the book from a reputable reseller. You wouldn't realize at first, but services like DriveThruRPG have, by offering PDF versions of the old books, kept old games from which the books are near-extinct quite alive and accessible.
