Early gaming consoles were quite simple machines (from today's point of view). So simple in fact that often the games were the initial "entry point" for the CPU on boot and the games had to set up the hardware first. For example, the game cartridge of a Sega MegaDrive/Genesis had to setup the interrupt vectors, initialize the graphic and sound chips and do some other low-level tasks before the actual game code could start. As far as I know, virtually every functionality had to be provided by the game, though I can imagine the console vendors like Nintendo and Sega probably provided SDKs which implemented some common tasks.
By contrast, modern gaming consoles run full Operating Systems, and games are merely applications and can build on functionality provided by the OS and its libraries (citation needed).
When did this transition from "no OS" to "some/full OS" happen? The PlayStation 1 must already have had an OS since its games come on CDs: these have a filesystem, the console must be able to set itself up, load code from the CD and provide a game functionality to access the CD's filesystem. Was there an intermediate step, like a ROM similar to a PC BIOS that provided some tools at a time when the game still was the first thing the CPU booted into?
For the sake of this question, "Operating System" can be used loosely. It should satisfy at least these simple conditions:
- A game is not the primary entry point of the CPU. First, some software (ROM, flash, whatever) is running which sets up the hardware and then starts the game (directly or after user selects it in a UI).
- Utility functions are provided by the OS to games for common operations. For example, memory management, reading files, rendering support for fonts, playing sound samples in common formats, … (Not all must be present, but there must be some useful functionality that abstracts hardware access or simplifies common tasks.)
So an IBM PC BIOS would already qualify: it's what's run after the CPU starts up, initializes hardware, then searches a medium, loads code from it and hands execution to that code. It provides APIs to access disks and some video (text) services. But not much else, so it's probably the absolute bare minimum to consider for this question. I'm more interested in the earliest systems that provide more functionality than that.