CPU Reset (CPU 0) and triple-fault when setting SS register to 0 in bootloader

Question

I am attempting to create a custom operating system (for educational purposes) for which I am also creating the bootloader (multiboot 2). I am attempting to create a 64-bit system.

After entering long mode, I want to "clean up" by loading all data segment registers with 0. To do this, I have the following code:

long_mode_start:
    mov ax, 0
    mov ss, ax
    mov ds, ax
    mov es, ax
    mov fs, ax
    mov gs, ax
    ret

The moment mov ss, ax is executed, the boot process fails and the CPU is reset (eventually leading up to a triple-fault). I'm using QEMU to emulate a X86_64 system. In the logs I get the following (I'm not sure how to fully interpret the logs): CPU Reset (CPU 0). For full log file, see: https://github.com/WJJongkind/TicTacTOS/blob/develop/log.txt

I'm really not sure what I am doing wrong in my code. The full assembly file that sets up long mode and paging can be found here: https://github.com/WJJongkind/TicTacTOS/blob/develop/Bootloader/boot.asm

Note: I'm very much a beginner if it comes to assembly language & OS programming, so my comments in the code are probably not 100% accurate.

What causes the CPU to reset when setting the SS register value?

You should no do that in long mode. The segment registers need to refer to valid segmen selectors. You canno "clean them up." — fuz, Apr 29 '22 at 10:09
@fuz Then the tutorial I'm following for all this is wrong... ouch. Thanks for pointing it out. I'll dig into it a bit further to see how to properly make bootloaders. If you have any suggestions for places to look then I'd be glad to hear them :) — ImJustACowLol, Apr 29 '22 at 11:41
It's probably not. Just don't confuse real mode boot sectors with protected mode multiboot bootloaders. The two are quite different as are the modes they operate in. — fuz, Apr 29 '22 at 12:09
I tried running the code from the tutorial's github page; that one also does not work. I ended up using the code provided on wiki.osdev.org's page on long mode with some modifications (checks if 64-bit mode is even supported). — ImJustACowLol, Apr 29 '22 at 13:11
@fuz the code for the tutorial that I followed can be found on this github page: https://github.com/davidcallanan/os-series/tree/ep2 Relevant files are in src/impl/x86_64/boot/ (all of the asm files). In main64.asm, replace `call kernel_main` with either `hlt` or a statement to print OK to the screen. It should never reach that line anyways. — ImJustACowLol, Apr 29 '22 at 16:48
`ds` and `es` can be the null selector (`0`), as can `fs` and `gs`, so your first comment is an overstatement, @fuz. That's what Linux does, as you can see with a debugger for a user-space process. But SS needs a valid descriptor for some reason, maybe to set the stack address width to 64, even though that's the only valid choice when CS selects a 64-bit code segment. — Peter Cordes, Apr 30 '22 at 04:20

Peter Cordes · Answer 1 · 2022-05-29T05:16:07.193

Correction, this answer was based on a misreading of the manual: the pseudo-code it contains only applies to 32-bit protected mode. See the linked duplicate where Michael Petch's answer covers this in much more detail.

This code will fault if executed in 32-bit protected mode, but not in 64-bit long mode at CPL=0. Or if you are executing this code in 32-bit mode before switching to long mode, then yes you should expect it to fault on the spot for SS, unlike later when a load or store tries to use it for the other segments.

Old answer, lightly edited to point out that it's answering the wrong question: 64-bit user space or 32-bit mode.

ds and es can be the null selector (0) in 64-bit mode (unlike 32-bit PM), as can fs and gs.
That's what Linux does, as I can confirm with GDB for a user-space process.

But SS needs to select a valid descriptor for some reason, maybe to set the stack address width to 64, even though that's the only valid choice when CS selects a 64-bit code segment. (Correction, no, 64-bit mode allows SS=0 when CPL<3, as long as CPL==RPL.)

Surprised QEMU doesn't give you more of a log about what exception triggered the reset. I think Bochs could do that, at least if you single-stepped with its built-in debugger.

According to Intel's manual (Operation section for mov-to-Sreg in protected mode) it should be a #GP(0), and then double-fault and then triple-fault, assuming you don't have IDT entries for those. Which is fine if you have a simulator that shows you exception details.

PROTECTED MODE ONLY; there is no pseudo-code in the manual for 64-bit mode

DEST ← SRC;
Loading a segment register while in protected mode results in special checks and actions, as described in the
following listing. These checks are performed on the segment selector and the segment descriptor to which it
points.
IF SS is loaded
    THEN
        IF segment selector is NULL
            THEN #GP(0); FI;
        IF segment selector index is outside descriptor table limits
        OR segment selector's RPL ≠ CPL
        OR segment is not a writable data segment
        OR DPL ≠ CPL
            THEN #GP(selector); FI;
        IF segment not marked present
            THEN #SS(selector);
            ELSE
                SS ← segment selector;
                SS ← segment descriptor; FI;
FI;

For details on what can cause exceptions in 64-bit mode, see that later section of the same manual entry: https://www.felixcloutier.com/x86/mov#64-bit-mode-exceptions

CPU Reset (CPU 0) and triple-fault when setting SS register to 0 in bootloader

1 Answers1