Wednesday, August 31, 2011

Using OpenSSL AES GCM

While I've been doing work on crypto for many years, I hadn't needed to use OpenSSL's EVP functions much at all.  Much of my life was PKCS#11 were mechanism parameters are passed with the Init functions.  Using GCM with from the Aug 15th 2011 OpenSSL gate was my encounter with passing parameters OpenSSL style.   Something I couldn't find documentation when searching.

The easy one to figure out what using EVP_CIPHER_CTX_ctrl functions to get the authentication tag in and out of the operation.  It was obvious that some function had to do it, and it was pretty easy to find while looking through the code.  After getting that setup, I was doing encrypt operations fine, but decrypt would fail during the EVP_DecryptFinal. Looking through the code it was the tag checking that was failing, but why?  I did not have any additional authentication data (AAD), so a simple decrypt should work for which it just encrypted.  After thinking it was a bug for quite a while I finally came across the FIPS POST code and saw the answer.

EVP requires that the app after doing EVP_*Init() must provide the AAD in the form of an EVP_*Update().  For example in pseudo code:
    EVP_EncryptInit(ctx, algorithm, key, iv);
    EVP_EncryptUpdate(ctx, NULL, &len, aad, aad_len);
    EVP_EncryptUpdate(ctx, ct, &len, pt, pt_len);
    EVP_EncryptFinal(ctx, ct + ct_len, &len);
    EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, 16, ct + ct_len)


It has been my observation that even if you are not using an AAD, you must send a zero length buffer to the EVP_EncryptUpdate() for the TAG to be correct during decrypt. If you do not, the TAG will be random.

This is as of the Aug 15th 2011 code in the gate of OpenSSL. A binary release of this is probably headed for OpenSSL 1.0.1