Assembly - calling conventions - callees

The callee has to do the other side of the handshake I discussed last time. But it also has to do a little more housekeeping. There are two registers which point into the stack, esp points to the top of the stack and is kept up to date by push/pop. ebp is the base pointer and points to the bottom of the current stack frame. It must be set when we create a new stack frame, such as when we enter a function. Since ebp is a callee-preserved register we must save the old value of ebp to the stack and later restore it.

cdecl

void __declspec(naked) copy(...)
{
  __asm {
    //prologue
    push ebp
    mov ebp, esp
    push esi
    //copy
    mov eax, [ebp+12]
    mov esi, [ebp+8]
    cmp eax, esi
    je finish
    mov ecx, [ebp+16]
start:
    mov dl, BYTE PTR [esi+ecx]
    mov BYTE PTR [eax+ecx], dl
    loop start
    mov dl, BYTE PTR [esi]
    mov BYTE PTR [eax], dl
finish:
    //epilogue
    pop esi
    pop ebp
    ret
  }
}

I've elided the arguments to the function (since we are doing the calling now, they are irrelevant). The __declspec(naked) declaration means that the compiler will not generate prologue and epilogue for the function. Our epilogue saves ebp to the stack, moves esp to ebp and saves esi to the stack. In the epilogue, esi is restored (it is the only register we need to preserve), then restore ebp. Because there are no push/pops in the body of the function, the stack pointer is correct. The body of the function is pretty much unchanged, the only difference is that we access the arguments by offset from ebp (i.e., in the stack), rather than by name.

Normally, we would store the arguments on the stack, somewhere between ebp and esp, but here we never take them out of the registers, so no need. We would often allocate memory on the stack (also between ebp and esp) for local variables, but we don't need to do that either.

fastcall

Here, we get the arguments in registers, we could copy them to the stack, but we don't need to, so we'll just use them. Because we don't pass anything on the stack, there is nothing to tidy up. We could avoid a couple of moves by using different registers, but I haven't.

  //prologue
  push ebp
  mov ebp, esp
  push edi
  //fillWithCount
  mov edi, ecx
  mov ecx, edx
  dec ecx
start:
  mov BYTE PTR [edi+ecx], cl
  loop start
  mov BYTE PTR [edi], cl
  //epilogue
  pop edi
  pop ebp
  ret

stdcall

Here the arguments are on the stack, and we must restore it on exit.

  //prologue
  push ebp
  mov ebp, esp
  push edi
  push ebx
  //fill
  mov edi, [ebp+8]
  mov ebx, [ebp+12] 
  mov ecx, [ebp+16]
  dec ecx
start:
  mov BYTE PTR [edi+ecx], bl
  loop start
  mov BYTE PTR [edi], bl
  //epilogue
  pop ebx
  pop edi
  pop ebp
  ret 12

We use the arguments in the same way as for the cdecl function. The difference is that the ret instruction takes an argument, the number of bytes to pop from the stack in order to tidy up. We could also do this manually:

pop edx
add esp, 12
jmp edx