LibCrypto: Make PKSystem methods return a ByteBuffer directly

It used to be that the caller would supply a buffer to write the output
to. This created an anti-pattern in multiple places where the caller
would allocate a `ByteBuffer` and then use `.bytes()` to provide it to
the `PKSystem` method. Then the callee would resize the output buffer
and reassign it, but because the resize was on `Bytes` and not on
`ByteBuffer`, the caller using the latter would cause a bug.

Additionally, in pretty much all cases the buffer was pre-allocated
shortly before.
This commit is contained in:
devgianlu 2024-12-25 22:04:38 +01:00 committed by Ali Mohammad Pur
commit 0fc02d4d00
Notes: github-actions[bot] 2025-01-13 16:02:16 +00:00
9 changed files with 69 additions and 112 deletions

View file

@ -345,18 +345,11 @@ bool Context::verify_certificate_pair(Certificate const& subject, Certificate co
Crypto::PK::RSAPrivateKey dummy_private_key;
Crypto::PK::RSAPublicKey public_key_copy { issuer.public_key.rsa };
auto rsa = Crypto::PK::RSA(public_key_copy, dummy_private_key);
auto verification_buffer_result = ByteBuffer::create_uninitialized(subject.signature_value.size());
if (verification_buffer_result.is_error()) {
dbgln("verify_certificate_pair: Unable to allocate buffer for verification");
return false;
}
auto verification_buffer = verification_buffer_result.release_value();
auto verification_buffer_bytes = verification_buffer.bytes();
MUST(rsa.verify(subject.signature_value, verification_buffer_bytes));
auto verification_bytes = MUST(rsa.verify(subject.signature_value));
ReadonlyBytes message = subject.tbs_asn1.bytes();
auto pkcs1 = Crypto::PK::EMSA_PKCS1_V1_5<Crypto::Hash::Manager>(kind);
auto verification = pkcs1.verify(message, verification_buffer_bytes, subject.signature_value.size() * 8);
auto verification = pkcs1.verify(message, verification_bytes, subject.signature_value.size() * 8);
return verification == Crypto::VerificationConsistency::Consistent;
}