-1

I'm trying to call a C function using x64 ASM with specific parameters. Calling the function itself works, and the correct arguments are supplied too, but later in the function an access violation will occur, and I'm not sure what causes this. My code looks as follows:

int __stdcall TestFunction(void *arg1, unsigned long arg2, void *arg3)
{
    MessageBoxW(NULL, L"Called", L"Test", MB_OK); //Causes access violation: "Access violation reading location 0xFFFFFFFFFFFFFFFF"
    return 0;
}


int main() 
{

    //Shellcode to call the test function
    BYTE shellCode[] = "\x48\xB9\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"       //mov rcx, ...
                       "\x48\xC7\xC2\x01\x00\x00\x00"                   //mov rdx, 1h
                       "\x49\xC7\xC0\x00\x00\x00\x00"                   //mov r8, 0
                       "\x48\xB8\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"       //mov rax, ...
                       "\xFF\xD0";                                      //call rax

    //Allocate executable page(s)
    PVOID alloc = VirtualAllocEx(GetCurrentProcess(), NULL, sizeof(shellCode), MEM_RESERVE | MEM_COMMIT, PAGE_EXECUTE_READWRITE);

    //Replace dummy addresses with actual addresses in the shellcode
    *(PDWORD_PTR)(shellCode + 2) = (DWORD_PTR)GetModuleHandleW(L"kernel32.dll");
    *(PDWORD_PTR)(shellCode + 26) = (DWORD_PTR)TestFunction;

    //Copy shellcode to allocated memory
    memcpy(alloc, shellCode, sizeof(shellCode));

    //Create a new thread that will start execution at the start address of the allocated executable memory
    CreateThread(NULL, 0, (PTHREAD_START_ROUTINE)alloc, NULL, 0, NULL);
}

I'm rather confident that it is not a problem with the call to MessageBoxW or CreateThread, but that it's a problem with my assembly code. How can I fix this error?

Peter Cordes
  • 286,368
  • 41
  • 520
  • 731
Louis Bernard
  • 150
  • 4
  • 16
  • 4
    Not that familiar with Windows calling conventions, but it looks like you didn't provide the [shadow space](https://stackoverflow.com/questions/30190132/what-is-the-shadow-space-in-x64-assembly). – Nate Eldredge May 16 '22 at 17:32
  • Are registers C and D different sizes? – Robert Harvey May 16 '22 at 17:35
  • @RobertHarvey: That part looks okay to me. rcx and rdx are both 64 bits, but two different mov instructions are being used: opcode b9 takes a 64-bit immediate, while opcode c7 takes a 32-bit immediate which is sign-extended to 64 bits. – Nate Eldredge May 16 '22 at 17:37
  • Ah, understood. It's been awhile. :) – Robert Harvey May 16 '22 at 17:38
  • 3
    Also, what happens when your shellcode is done? Execution just derails into nothingness after `call rax` instead of returning from your thread's procedure. – CherryDT May 16 '22 at 17:38
  • About your crash: I tested it myself and it seems your stack is not aligned, it's off by 8 bytes after the system's `call` to your assembly function and you don't realign it there as compiler-generated code normally would. (And once you fixed this, you might want to wrap your `CreateThread` in `WaitForSingleObject` so your program won't terminate immediately without giving the message box time to appear.) – CherryDT May 16 '22 at 17:47
  • Your machine code is a lot longer than it needs to be; looks like you're using an assemble that doesn't optimize properly. e.g. `mov rcx, -1` should assemble to 7-byte `mov r/m64, sign_extended_imm32`. And `mov rdx, 1` should assemble to 5-byte `mov edx, 1`. Zeroing R8 should be `xor r8d, r8d` (3 bytes). If you wanted to really optimize for size (avoiding 0s), do RCX=-1 with `lea rcx, [rdx-2]` (4 bytes) after getting RDX=1 perhaps with `push 1` / `pop rdx`. Or use `or rcx, -1` (4B) as a starting point. [Tips for golfing in x86/x64 machine code](//codegolf.stackexchange.com/a/132985) – Peter Cordes May 16 '22 at 18:16
  • That's not a correctness problem, though; that's covered by the linked duplicates. Alternatively, you could avoid messing with RSP entirely by using `jmp` as a tail-call, and then you wouldn't need a `ret` after it. Oh, I see two of those `mov r64, imm64` instructions were placeholders for 8-byte addresses. That's fine then, but the `0` and `1` should be more compact. – Peter Cordes May 16 '22 at 18:18

0 Answers0