Implementing a resilient AES-256 GCM decryptor in Rust
Introduction
This post is about a decryption problem I recently worked upon. The solution is nice and simple (almost trivial) to begin with but becomes more interesting when an additional requirement is added. All the implementation and testing code for this can be found on my GitHub here - prnvbn/rest-resilient-aes-256-gcm
A simple problem
Decrypt an encrypted stream of data.
The encryption algorithm used is AES-256 GCM.
This on its own is fairly straightforward. Some basic knowledge of cryptography and just being aware of OpenSSL’s existence is enough to come up with a working solution.
A simple solution
The simplest way to implement AES 256 GCM decryption (in Rust) is to just use the openssl crate with Rust bindings for the OpenSSL library (written in C).
All that needs to be done is have a struct implement the following trait and maybe expand the error enum for it1.
// file: src/decryption/mod.rs
#[derive(Debug, Error)]
pub enum DecryptError {
}
pub trait Decryptor {
fn decrypt(&mut self, ciphertext: &mut [u8]) -> Result<(), DecryptError>;
}
A simple implementation for this is given below
// file: src/decryption/open.rs
use crate::decryption::{DecryptError, Decryptor};
use openssl::symm::{Cipher, Crypter, Mode};
const WORK_BUF_SZ: usize = 2048;
pub struct AesGcmDecryptor {
decryptor: Crypter,
work_buf: [u8; WORK_BUF_SZ],
}
impl AesGcmDecryptor {
pub fn new(key: [u8; 32], iv: [u8; 16]) -> Result<Self, DecryptError> {
let decryptor = Crypter::new(Cipher::aes_256_gcm(), Mode::Decrypt, &key, Some(&iv))
.map_err(DecryptError::InitError)?;
Ok(AesGcmDecryptor {
decryptor,
work_buf: [0; WORK_BUF_SZ],
})
}
}
impl Decryptor for AesGcmDecryptor {
fn decrypt(&mut self, ciphertext: &mut [u8]) -> Result<(), DecryptError> {
match self.decryptor.update(ciphertext, &mut self.work_buf) {
Ok(s) => {
ciphertext.copy_from_slice(&self.work_buf[..s]);
Ok(())
}
Err(e) => Err(DecryptError::DecryptionError(e)),
}
}
}
To verify our implementation, let’s use some contrived input and ensure that this worked. The following is a simple program that pretty prints the encrypted cipher text and the decrypted plain text2
// file: src/main.rs
use base64::{engine::general_purpose, Engine};
use pretty_hex::*;
use sdbx::decryption::{open::AesGcmDecryptor, Decryptor};
fn main() {
// test data generated from - https://www.devglan.com/online-tools/aes-encryption-decryption
let mut ciphertext = general_purpose::STANDARD
.decode("wsDHBX9rmTv7OmuLycMEA3pjfeDo5aeTTFJICVaQ7h7AxTnItKi2TKSWt+NG22Y=")
.unwrap();
let iv: [u8; 16] = *b"oneinitialvector";
let secret: [u8; 32] = *b"encrypt data securely for safety";
println!("Encrypted data:");
println!("{}", pretty_hex(&ciphertext.as_slice()));
let mut decryptor = AesGcmDecryptor::new(secret, iv).expect("failed to create decrytpro ");
let _ = decryptor
.decrypt(&mut ciphertext)
.expect("failed to decrypt");
println!("Decrypted data:");
println!("{}", pretty_hex(&ciphertext.as_slice()));
}
The output of the above program is as follows
Encrypted data:
Length: 47 (0x2f) bytes
0000: c2 c0 c7 05 7f 6b 99 3b fb 3a 6b 8b c9 c3 04 03 .....k.;.:k.....
0010: 7a 63 7d e0 e8 e5 a7 93 4c 52 48 09 56 90 ee 1e zc}.....LRH.V...
0020: c0 c5 39 c8 b4 a8 b6 4c a4 96 b7 e3 46 db 66 ..9....L....F.f
Decrypted data:
Length: 47 (0x2f) bytes
0000: 73 75 70 65 72 20 73 65 63 72 65 74 20 6d 65 73 super secret mes
0010: 73 61 67 65 20 74 6f 20 65 6e 63 72 79 70 74 92 sage to encrypt.
0020: 71 7f 76 cd 00 ef d3 51 bf 35 81 c4 ae e1 c2 q.v....Q.5.....
A heisenbug
While the above solution worked well initially, the decryptor began intermittently failing for certain data streams, only recovering when a new encrypted stream was started. This inconsistent behavior made it challenging to identify the root cause making this issue a heisenbug. To under this issue, let’s examine AES-256 GCM more closely.
AES-256 is a block cipher that encrypts fixed-size blocks of data.
GCM (Galois/Counter Mode) is an authenticated encryption mode that converts the AES block cipher a stream-like mode of operation. GCM uses a counter (CTR) mode internally to track the number of bytes that have been processed giving AES stream cipher-like properties.
This counter makes it absolutely crucial that are decryptor doesn’t skip bytes while decrypting as doing so would mean that the internal counter of the decryptor is out of sync and all following decryption will fail.
Fixing the heisenbug
Since the OpenSSL bindings abstract away the counter state, we implement our own Rust native AES decryptor using some low level crates like aes and ctr3 to keep track of the internal counter state.
An implementation for a new packet loss resilient decryptor is provided below4
// file: src/decryption/resilient.rs
use aes::{
cipher::{
generic_array::{typenum::U16, GenericArray},
BlockEncrypt, KeyInit, KeyIvInit, StreamCipher
},
Aes256, Block,
};
use crate::decryption::{DecryptError, Decryptor};
use ctr::Ctr32BE;
use ghash::{universal_hash::UniversalHash, GHash};
pub struct ResilientDecryptor {
cipher: Ctr32BE<Aes256>,
counter: u32,
}
const NONCE_BITS: u64 = 16_u64 * 8;
impl ResilientDecryptor {
pub fn new(key: [u8; 32], iv: [u8; 16]) -> Result<Self, DecryptError> {
let key = GenericArray::from(key);
let cipher = aes::Aes256::new(&key);
// initialize cipher
let mut ghash_key = ghash::Key::default();
cipher.encrypt_block(&mut ghash_key);
let ghash = GHash::new(GenericArray::from_slice(&ghash_key));
let first_block = Self::gen_first_block(&iv, &ghash);
let mut cipher = Ctr32BE::<Aes256>::new(&key, &first_block);
// tag mask handling
let mut tag_mask = Block::default();
cipher.apply_keystream(&mut tag_mask);
Ok(Self {
cipher: cipher,
counter: 16,
})
}
fn gen_first_block(iv: &[u8], ghash: &GHash) -> GenericArray<u8, U16> {
let mut ghash = ghash.clone();
ghash.update_padded(iv);
let mut block = Block::default();
block[8..].copy_from_slice(&NONCE_BITS.to_be_bytes());
ghash.update(&[block]);
ghash.finalize()
}
}
impl Decryptor for ResilientDecryptor {
fn decrypt(&mut self, ciphertext: &mut [u8]) -> Result<(), DecryptError> {
self.cipher.apply_keystream(ciphertext);
self.counter += ciphertext.len() as u32;
Ok(())
}
}
In addition to decrypting the bytes, this decryptor also maintains a counter which tracks the number of bytes that have been decrypted. Using this counter we can extend the above implementation to allow the client of our decryption library to arbitrarily move the counter:
use aes:: cipher:: StreamCipherSeek; // add this import
impl ResilientDecryptor {
// ...
pub fn forward(&mut self, num_bytes: u32) {
self.cipher.seek(self.counter + num_bytes);
self.counter += num_bytes;
}
// ...
}
Using this newly added forward method, upon facing packet loss the internal counter of the decryption algorithm can be seeked appropriately to account for the missing bytes5. An example for this is shown in the main function provided below:
fn main() {
// test data generated from - https://www.devglan.com/online-tools/aes-encryption-decryption
let mut ciphertext = general_purpose::STANDARD
.decode("wsDHBX9rmTv7OmuLycMEA3pjfeDo5aeTTFJICVaQ7h7AxTnItKi2TKSWt+NG22Y=")
.unwrap();
let iv: [u8; 16] = *b"oneinitialvector";
let secret: [u8; 32] = *b"encrypt data securely for safety";
println!("Encrypted data:");
println!("{}", pretty_hex(&ciphertext.as_slice()));
let mut fst_part = ciphertext[..12].to_vec();
let mut decryptor = ResilientDecryptor::new(secret, iv).expect("failed to create decryptor");
let _ = decryptor
.decrypt(&mut fst_part)
.expect("failed to decrypt first part");
println!("First part decrypted:");
println!("{}", pretty_hex(&fst_part.as_slice()));
decryptor.forward(8);
let mut snd_part = ciphertext[20..].to_vec();
let _ = decryptor
.decrypt(&mut snd_part)
.expect("failed to decrypt first part");
println!("Second part decrypted:");
println!("{}", pretty_hex(&snd_part.as_slice()));
}
The output for the above main function shows that as long as the counter is correctly forwarded to the number of “lost” bytes, the decryptor can still decrypt successfully!
Encrypted data:
Length: 47 (0x2f) bytes
0000: c2 c0 c7 05 7f 6b 99 3b fb 3a 6b 8b c9 c3 04 03 .....k.;.:k.....
0010: 7a 63 7d e0 e8 e5 a7 93 4c 52 48 09 56 90 ee 1e zc}.....LRH.V...
0020: c0 c5 39 c8 b4 a8 b6 4c a4 96 b7 e3 46 db 66 ..9....L....F.f
First part decrypted:
Length: 12 (0xc) bytes
0000: 73 75 70 65 72 20 73 65 63 72 65 74 super secret
Second part decrypted:
Length: 27 (0x1b) bytes
0000: 20 74 6f 20 65 6e 63 72 79 70 74 92 71 7f 76 cd to encrypt.q.v.
0010: 00 ef d3 51 bf 35 81 c4 ae e1 c2 ...Q.5.....
Notes
I usually create a custom Error enum for most structs using thiserror. I quite like this pattern and it makes error handling in Rust quite pleasant to work with. ↩︎
I wrote this as a test but have converted it to a main program for this post, the tests can be found in the linked GH repo. ↩︎
The aes crate has a security warning which I am turning a blind eye towards but you may not want to ↩︎
Figuring out what traits to import was extremely annoying! The aes package has very similar traits that implement slightly different things and the only way to discover them is to read the source code for the crate. ↩︎
It is highly unlikely that packet loss can be quantified in terms of number of missing bytes, so in reality I ended up trying out a a range of forwards steps based on heuristics. ↩︎