What's the actual difference between RSA encrypting and Descrypting and RSA Signing?

Question

this question arised because I'm working with Windows' CNG library and it has 4 functions for RSA:

1. BCryptEncrypt (private key, message)
2. BCryptDecrypt (public key, message)
3. BCryptSignHash (private key, hashed message)
4. BCryptVerifySignature (public key, signature, hashed message)

So I was wondering, is there any difference between "Encrypting + Decrypting" and "Signing + Verifying"? I mean, if I hash the message and then encrypt it, wouldn't that be the same as Signing it? or Am I missing some logic that is behind that function? (talking about RSA in general, not about how windows' CNG work).

The only visible difference I see is:

• For Encryption and Decryption I can use OAEP and PKCS1 padding.
• For Signing and Verifying I can use PKCS1 and PSS padding.

But again, is this because of how those functions are made in this library? or because the "operation" and the output are done in different ways depending on if it is Signing or Encryption?

If they are different, then why do we need the Signing part? Isn't encrypting with private and decrypting with public secure enough to "sign" it?

Answer 1

So I was wondering, is there any difference between "Encrypting + Decrypting" and "Signing + Verifying"? I mean, if I hash the message and then encrypt it, wouldn't that be the same as Signing it?

First of all RSA or any public key is not preferred for encryption, we prefer a hybrid-cryptosystem where a key is transferred/exchanged with public-key cryptography and the message is encrypted with a symmetric encryption scheme with the transferred/exchanged key.

RSA encryption to be secure requires a padding scheme like PKCS#v1.5 or OAEP. The letter is preferable since there is a security proof. The modulus size prevents you from encrypting arbitrary sizes so you need to encrypt a message dividing part like in block ciphers then perform the slow encryption many times.

RSA signature with PSS requires hashing not only for the security but for the message size. There is also RSA Full Domain Hash (FDH) Signature that hashes and signs and it has existentially unforgeable under adaptive chosen-message attacks.

If you hash a message then encrypt, then except you, nobody can verify the signature. You need to use your private key for signatures even for RSA-FDH. Keep in mind that it is not advisable to use a key for both encryption and signatures.

So the main difference in encryption vs signature is the used key and padding requirements. If you encrypt a message without padding you will have many attacks like it is malleable and see more here Twenty Years of Attacks on the RSA Cryptosystem

But again, is this because of how those functions are made in this library? or because the "operation" and the output are done in different ways depending on if it is Signing or Encryption?

Yes, the operations are done differently since they require different padding to be secure.

If they are different, then why do we need the Signing part?

Because they are different so one needs different functions. Even if they are the same one still needs these functions to remove the ambiguity. Think about there is only encrypt and decrypt functions and the developers need to tell everyone to use decryption for signature and encryption for verification. Some people will use the encryption for the signature that will reveal unwanted information. Unfortunately, this is due to the RSA that has both encryption and signature.

Isn't encrypting with private and decrypting with public secure enough to "sign" it?

No, you need hash then sign for RSA-FDH and hash and apply PSS then sign.

signature = RSA-FDH(Hash(message))

of

signature = RSA-PSS(Hash(message))

Answer 2

Encryption/Decryption and Signing/Verifying satisfies two different aspects of the information security triad. Refer to the CIA triad for more information.

• Encryption/Decryption ( Confidentiality)

Encryption makes sure that the content of a message we send through an unprotected medium, stays unknown, even though it falls into the wrong hands. Thus ensuring the confidentiality of the message.

At senders end,

cipher_msg = encrypt(key, plain_msg)

At the receivers end,

plain_msg = decrypt(key, cipher_msg)

Without the knowledge of the key, a third party cannot retrieve the contents of the message even though they manage to get the encrypted message. Hence, encryption allows us to send messages securely via an unprotected medium.

• Signing/Verifying (Integrity)

What do you expect when you sign a check? And what does your sign mean to the cashier at the bank when that check is being cashed? Your signature guarantees the authenticity of the check or basically whatever the document it is hanging under. In the context of information security, signing and verifying is used to ensure the integrity of a message sent via an unprotected medium. The sender does not care about the message being readable by everyone, instead, they want to make sure that there is a way for the reader to verify that the message is absolutely sent by the said person and it has not been tampered with. The process is simple.

At the sender's end,

signature = rsa(hash_of_the_message, private_key)

Then send the message and the signature via the unprotected medium.

At the receiver's end,

original_msg_hash = rsa(signature, public_key)

msg_hash = hash(message)

is_authentic = compare(msg_hash, original_msg_hash)

These steps are merged into a single function.

is_authentic = verify(signature, public_key, msg_hash)

If the verification fails, it essentially means that the integrity of the message cannot be guaranteed.