26

It is highly recommended when creating a 64-bit kernel (for x86_64 platform), to instruct the compiler not to use the 128-byte Red Zone that the user-space ABI does. (For GCC the compiler flag is -mno-red-zone).

The kernel would not be interrupt-safe if it is enabled.

But why is that?

Peter Cordes
  • 286,368
  • 41
  • 520
  • 731
mmk
  • 575
  • 5
  • 13
  • 1
    Related: http://stackoverflow.com/questions/38042188/where-exactly-is-the-red-zone-on-x86-64 and http://stackoverflow.com/questions/37941779/why-do-we-need-stack-allocation-when-we-have-a-red-zone have answers explaining what the red zone is all about for code that can use it. – Peter Cordes Jun 26 '16 at 22:24

3 Answers3

16

Quoting from the AMD64 ABI:

The 128-byte area beyond the location pointed to by %rsp is considered to be reserved and shall not be modified by signal or interrupt handlers. Therefore, functions may use this area for temporary data that is not needed across function calls. In particular, leaf functions may use this area for their entire stack frame, rather than adjusting the stack pointer in the prologue and epilogue. This area is known as the red zone.

Essentially, it's an optimization - the userland compiler knows exactly how much of the Red Zone is used at any given time (in the simplest implementation, the entire size of local variables) and can adjust the %rsp accordingly before calling a sub-function.

Especially in leaf functions, this can yield some performance benefits of not having to adjust %rsp as we can be certain no unfamiliar code would run while in the function. (POSIX Signal Handlers might be seen as a form of a co-routine, but you can instruct the compiler to adjust the registers before using stack variables in a signal handler).

In the kernel space, once you start thinking about interrupts, if those interrupts make any assumptions about %rsp, they will likely be incorrect - there is no certainty with regards to the utilization of the Red Zone. So, you either assume all of it is dirty, and needlessly waste stack space (effectively running with a 128-byte guaranteed local variable in every function), or, you guarantee that the interrupts make no assumptions about %rsp - which is tricky.

In user space, context switches + 128-byte overallocation of stack handle it for you.

qdot
  • 6,025
  • 5
  • 42
  • 93
  • 7
    It's not just space-saving. It's actually impossible to implement the normal 128-byte red-zone safely, because interrupts always clobber the 16 bytes below `%rsp` before any code from the interrupt handler even runs. – Peter Cordes Jun 26 '16 at 21:51
  • 1
    @qdot, could you please explain what you mean by 128-byte overallocation? It means that if amd64 ABI did not have "red zone" concept, the lowest address stack could grow upon would be 128 bytes higher? – Bulat M. Sep 19 '16 at 05:09
  • POSIX signal are delivered to handlers by the kernel, not the hardware. The kernel simply respects the ABI's red-zone when delivering signals that don't use `sigaltstack`. The relevant code in the kernel isn't compiler-generated. That's why normal functions can be registered as signal handlers; they don't need any special `__attribute__` to compile specially. – Peter Cordes Jul 01 '21 at 06:12
  • And on Linux at least, libc doesn't have to silently substitute a wrapper function for the real address in sigaction(2). It only tells the kernel what return address to pass to that user-space function call, getting it to return to a special libc function that uses [`sigreturn(2)`](https://man7.org/linux/man-pages/man2/sigreturn.2.html). (That man page describes the Linux mechanism where the kernel puts the thread's register state onto the user-space stack.) – Peter Cordes Jul 01 '21 at 06:15
  • Also, on x86-64, local variables go below the return address, so reserving 128 bytes of space for a dummy local variable wouldn't help. Having a return address above that would clobber a red-zone. (Unlike on ISAs with a link register where a normal function call gets its return address in a register, not stack memory. Although interrupts on most ISAs still implicitly use a stack.) – Peter Cordes Jul 01 '21 at 06:16
16

It is possible to use red-zone in kernel-type contexts. The IDTentry can specify a stack index (ist) of 0..7, where 0 is a bit special. The TSS contains a table of these stacks. 1..7 are loaded, and used for the initial registers saved by the exception/interrupt, and do not nest. If you partition the various exception entries by priorities (eg. NMI is the highest and can happen at any time) and treat these stacks as trampolines, you can safely handle red zones in kernel-type contexts. That is, you can subtract 128 from the saved stack pointer to get a usable kernel stack before enabling interrupts or code which can cause exceptions.

The zero index stack behaves in a more conventional manner, pushing the stack,flags,pc,error on the existing stack when there is no privilege transition.

The code in the trampoline has to be careful (duh, it is a kernel) not to generate other exceptions while it sanitizes the machine state, but provides a nice, safe spot to detect pathological kernel nesting, stack corruption, etc... [ sorry to respond so late, noticed this while searching for something else].

mevets
  • 9,553
  • 19
  • 30
  • 1
    Please upvote this person some more. This is why the red zone was introduced in the ABI - it's universally usable *if* you actually use the 64-bit TSS and IST mechanism, specifically created to make this work. – dascandy Aug 12 '19 at 09:15
15

In kernel-space, you're using the same stack that interrupts use. When an interrupt happens, the CPU pushes a return address and RFLAGS. This clobbers 16 bytes below rsp. Even if you wanted to write an interrupt-handler that assumed the full 128 bytes of the red-zone were valuable, it would be impossible.

You could maybe have a kernel-internal ABI that had a small red-zone from rsp-16 to rsp-48 or something. (Small because kernel stack is valuable, and most functions don't need very much red-zone anyway.)

Interrupt handlers would have to sub rsp, 32 before pushing any registers. (and restore it before iret).

This idea won't work if an interrupt handler can itself be interrupted before it runs sub rsp, 32, or after it restores rsp before an iret. There would be a window of vulnerability where valuable data is at rsp .. rsp-16.

Another practical problem with this scheme is that AFAIK gcc doesn't have configurable red-zone parameters. It's either on or off. So you'd have to add support for a kernel flavour of red-zone to gcc / clang if you wanted to take advantage of it.

Even if it was safe from nested interrupts, the benefits are pretty small. The difficulty of proving it's safe in a kernel might make it not worth it. (And as I said, I'm not at all sure it can be implemented safely, because I think nested interrupts are possible.)

(BTW, see the tag wiki for links to the ABI documenting the red-zone, and other stuff.)

Community
  • 1
  • 1
Peter Cordes
  • 286,368
  • 41
  • 520
  • 731
  • A bit unsure about why it won't work cf. the sentence "This idea won't work if an interrupt handler can itself be interrupted before it runs sub rsp, 32, or after it restores rsp before an iret. There would be a window of vulnerability where valuable data is at rsp .. rsp-16.". Wouldn't the "second interrupt" handler do the sub rsp,32'ing too, thus protecting the assumed red zone of the original interrupted code? Is it because there will be multiple nested return-addresses+RFLAGS pushed (by the CPU itself) which could eventually overwrite the red zone or? – Morty Jun 23 '20 at 11:13
  • @Morty: not if a 2nd or 3rd nested interrupt is handled by hardware *before* software can run `sub rsp,32`. An exception / interrupt frame is more than 16 bytes: at least RIP, CS, RFLAGS, and for synchronous exceptions an exception-type code, IIRC. And if nested can happen, double-nested can theoretically happen, so even `sub rsp, 2*max_single_frame` is in theory not enough, and neither is any arbitrary size. – Peter Cordes Jun 23 '20 at 15:39