Checking if an address is encoded on the end of a bytes array

Question

I have a bytes array. Within the last 20 bytes, there is an address stored.

I am trying to compare the address at the end of this array to the address of a msg.sender, and am wondering if there is a hyper-efficient way of doing so (code golf people, I'm looking at you).

So far, here is the solution I have come up with.

function toBytes(address a) constant returns (bytes b){
  assembly {
      let m := mload(0x40)
      mstore(add(m, 20), xor(0x140000000000000000000000000000000000000000, a))
      mstore(0x40, add(m, 52))
      b := m
  }
}

function checkAddress(bytes data, address toCheck) constant returns (bool){
    bytes memory addBytes = toBytes(toCheck);
    for(uint i = data.length - 20; i < data.length; i++) {
        if (data[i] != addBytes[i - data.length + 20]) {
            return false;
        }
    }
    return true;
  }

checkAddress takes the byte array, and the address to compare it to, converts the address to a bytes array, and then compares it, byte by byte, to the last 20 bytes of the byte array. As of now, this works.

(As a side note, the toBytes function was taken from the stack exchange here: How to convert an address to bytes in Solidity?)

However, I feel like there should be a significantly more efficient way of doing this check, possibly using assembly. Something like casting the address to a uint, maybe using Nick Johnson's string library to grab the last 20 bytes of the array, and converting that to a uint also, and then comparing.

Any directions/tips would be greatly appreciated. Thanks for the help!