mirror of
https://github.com/ceph/ceph
synced 2026-08-02 07:03:18 +00:00
Merge pull request #67246 from mheler/wip-rgw-sse-gcm
rgw: Add AES-256-GCM support for RGW server-side encryption Reviewed-by: Casey Bodley <cbodley@redhat.com>
This commit is contained in:
commit
54a58396ff
@ -110,8 +110,16 @@ for a given bucket:
|
||||
}
|
||||
|
||||
Other commands and APIs will report object and bucket sizes based on their
|
||||
uncompressed data.
|
||||
uncompressed data.
|
||||
|
||||
The ``size_utilized`` and ``size_kb_utilized`` fields represent the total
|
||||
size of compressed data, in bytes and kilobytes respectively.
|
||||
The ``size_utilized`` and ``size_kb_utilized`` fields represent the actual
|
||||
bytes stored on disk, in bytes and kilobytes respectively. This value reflects
|
||||
the effects of both compression and encryption:
|
||||
|
||||
* With compression only: size after compression
|
||||
* With AEAD encryption only (e.g., AES-256-GCM): size after encryption
|
||||
(includes authentication tag overhead)
|
||||
* With both compression and encryption: size after compression then encryption
|
||||
|
||||
See :ref:`radosgw-encryption` for details on encryption algorithms.
|
||||
|
||||
|
||||
@ -233,6 +233,7 @@ Server-side Encryption Settings
|
||||
===============================
|
||||
|
||||
.. confval:: rgw_crypt_s3_kms_backend
|
||||
.. confval:: rgw_crypt_sse_algorithm
|
||||
|
||||
Barbican Settings
|
||||
=================
|
||||
|
||||
@ -18,6 +18,72 @@ Object Gateway stores that data in the Ceph Storage Cluster in encrypted form.
|
||||
|
||||
.. note:: Server-side encryption keys must be 256-bit long and base64 encoded.
|
||||
|
||||
Encryption Algorithm
|
||||
====================
|
||||
|
||||
.. versionadded:: Umbrella
|
||||
|
||||
The Ceph Object Gateway supports two AES-256 encryption algorithms for
|
||||
server-side encryption:
|
||||
|
||||
**AES-256-CBC** (Cipher Block Chaining)
|
||||
The legacy encryption algorithm. This mode is compatible with older Ceph
|
||||
releases and is the default for backward compatibility. CBC mode encrypts
|
||||
data but does not provide built-in integrity verification.
|
||||
|
||||
**AES-256-GCM** (Galois/Counter Mode)
|
||||
A modern authenticated encryption algorithm that provides both
|
||||
confidentiality and integrity protection. GCM mode detects any tampering
|
||||
or corruption of encrypted data. This is the recommended algorithm for
|
||||
new deployments.
|
||||
|
||||
The encryption algorithm for new objects can be configured with::
|
||||
|
||||
rgw crypt sse algorithm = aes-256-cbc # default, for backward compatibility
|
||||
rgw crypt sse algorithm = aes-256-gcm # recommended for new deployments
|
||||
|
||||
.. note:: This setting only affects newly encrypted objects. Existing objects
|
||||
are always decrypted using the algorithm that was used when they
|
||||
were encrypted, regardless of the current setting. This allows
|
||||
CBC-encrypted and GCM-encrypted objects to coexist in the same
|
||||
cluster.
|
||||
|
||||
.. important:: When upgrading from an older Ceph release, keep the default
|
||||
``aes-256-cbc`` setting until all RGW instances have been
|
||||
upgraded. Once all instances support GCM, you can enable
|
||||
``aes-256-gcm`` for new uploads.
|
||||
|
||||
GCM Encryption Format
|
||||
---------------------
|
||||
|
||||
AES-256-GCM encrypts data in 4 KB (4096 byte) chunks. Each chunk produces
|
||||
4112 bytes of ciphertext (4096 bytes of encrypted data plus a 16-byte
|
||||
authentication tag).
|
||||
|
||||
.. list-table:: GCM Size Calculation Example
|
||||
:header-rows: 1
|
||||
:widths: 40 30 30
|
||||
|
||||
* - Description
|
||||
- Size
|
||||
- Notes
|
||||
* - Original plaintext
|
||||
- 10,000 bytes
|
||||
- User's data
|
||||
* - Number of chunks
|
||||
- 3
|
||||
- ⌈10000 ÷ 4096⌉
|
||||
* - Authentication tags
|
||||
- 48 bytes
|
||||
- 3 chunks × 16 bytes
|
||||
* - **Encrypted on disk**
|
||||
- **10,048 bytes**
|
||||
- plaintext + tags
|
||||
|
||||
The storage overhead is approximately 0.4% (16 bytes per 4 KB). S3 API
|
||||
responses always report the original plaintext size, so this overhead is
|
||||
transparent to clients.
|
||||
|
||||
Customer-Provided Keys
|
||||
======================
|
||||
|
||||
|
||||
@ -63,6 +63,11 @@ which means that Ceph will not be able to utilize QAT hardware features for
|
||||
crypto operations based on the OpenSSL crypto plugin. As a result, one QAT plugin
|
||||
based on native QAT API is added into the crypto framework.
|
||||
|
||||
.. note:: QAT acceleration currently supports only AES-256-CBC encryption mode.
|
||||
The AES-256-GCM encryption mode (see :ref:`radosgw-encryption`) uses ISA-L
|
||||
or OpenSSL acceleration instead. QAT support for GCM may be added in a
|
||||
future release.
|
||||
|
||||
2. QAT Support for Compression
|
||||
|
||||
As mentioned above, QAT support for compression is based on the QATzip library
|
||||
|
||||
1
qa/suites/rgw/crypt/aes/.qa
Symbolic link
1
qa/suites/rgw/crypt/aes/.qa
Symbolic link
@ -0,0 +1 @@
|
||||
../.qa/
|
||||
1
qa/suites/rgw/crypt/aes/default.yaml
Normal file
1
qa/suites/rgw/crypt/aes/default.yaml
Normal file
@ -0,0 +1 @@
|
||||
|
||||
5
qa/suites/rgw/crypt/aes/gcm.yaml
Normal file
5
qa/suites/rgw/crypt/aes/gcm.yaml
Normal file
@ -0,0 +1,5 @@
|
||||
overrides:
|
||||
ceph:
|
||||
conf:
|
||||
client:
|
||||
rgw crypt sse algorithm: aes-256-gcm
|
||||
1
qa/suites/rgw/multisite/aes/.qa
Symbolic link
1
qa/suites/rgw/multisite/aes/.qa
Symbolic link
@ -0,0 +1 @@
|
||||
../.qa/
|
||||
1
qa/suites/rgw/multisite/aes/default.yaml
Normal file
1
qa/suites/rgw/multisite/aes/default.yaml
Normal file
@ -0,0 +1 @@
|
||||
|
||||
5
qa/suites/rgw/multisite/aes/gcm.yaml
Normal file
5
qa/suites/rgw/multisite/aes/gcm.yaml
Normal file
@ -0,0 +1,5 @@
|
||||
overrides:
|
||||
ceph:
|
||||
conf:
|
||||
client:
|
||||
rgw crypt sse algorithm: aes-256-gcm
|
||||
@ -3294,6 +3294,25 @@ options:
|
||||
services:
|
||||
- rgw
|
||||
with_legacy: true
|
||||
- name: rgw_crypt_sse_algorithm
|
||||
type: str
|
||||
level: advanced
|
||||
desc: Default encryption algorithm for server-side encryption
|
||||
long_desc: Specifies the default AES encryption algorithm to use for new objects
|
||||
encrypted with SSE-C, SSE-KMS, SSE-S3, or RGW-AUTO modes. Valid values are
|
||||
aes-256-cbc (legacy, compatible with older RGW versions) and aes-256-gcm
|
||||
(recommended, provides authenticated encryption). Existing encrypted objects
|
||||
are always decrypted using the algorithm specified in their metadata,
|
||||
regardless of this setting.
|
||||
default: aes-256-cbc
|
||||
services:
|
||||
- rgw
|
||||
see_also:
|
||||
- rgw_crypt_require_ssl
|
||||
enum_values:
|
||||
- aes-256-cbc
|
||||
- aes-256-gcm
|
||||
with_legacy: true
|
||||
- name: rgw_crypt_sse_s3_backend
|
||||
type: str
|
||||
level: advanced
|
||||
|
||||
@ -30,6 +30,14 @@ class CryptoAccel {
|
||||
|
||||
static const int AES_256_IVSIZE = 128/8;
|
||||
static const int AES_256_KEYSIZE = 256/8;
|
||||
|
||||
/**
|
||||
* GCM constants (IV size is 12 bytes; distinct from CBC's 16-byte IV).
|
||||
* NIST SP 800-38D recommends 96-bit (12-byte) IVs for GCM.
|
||||
*/
|
||||
static const int AES_GCM_IV_SIZE = 96/8; // 12 bytes
|
||||
static const int AES_GCM_TAGSIZE = 16; // 128-bit auth tag
|
||||
|
||||
virtual bool cbc_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_256_IVSIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
@ -46,5 +54,30 @@ class CryptoAccel {
|
||||
const unsigned char iv[][AES_256_IVSIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
optional_yield y) = 0;
|
||||
|
||||
virtual bool gcm_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
unsigned char* tag,
|
||||
optional_yield y) = 0;
|
||||
virtual bool gcm_decrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
const unsigned char* tag,
|
||||
optional_yield y) = 0;
|
||||
virtual bool gcm_encrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) = 0;
|
||||
virtual bool gcm_decrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
const unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) = 0;
|
||||
};
|
||||
#endif
|
||||
|
||||
@ -15,6 +15,10 @@
|
||||
#include "crypto/isa-l/isal_crypto_accel.h"
|
||||
|
||||
#include "crypto/isa-l/isa-l_crypto/include/aes_cbc.h"
|
||||
#include "crypto/isa-l/isa-l_crypto/include/aes_gcm.h"
|
||||
#include "include/compat.h" // for ceph_memzero_s
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
|
||||
bool ISALCryptoAccel::cbc_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_256_IVSIZE],
|
||||
@ -43,3 +47,90 @@ bool ISALCryptoAccel::cbc_decrypt(unsigned char* out, const unsigned char* in, s
|
||||
aes_cbc_dec_256(const_cast<unsigned char*>(in), const_cast<unsigned char*>(&iv[0]), keys_blk.dec_keys, out, size);
|
||||
return true;
|
||||
}
|
||||
|
||||
/*
|
||||
* Thread-local GCM key cache to avoid re-running aes_gcm_pre_256() for
|
||||
* repeated keys. Key material is securely wiped on thread exit.
|
||||
*/
|
||||
struct gcm_key_cache_t {
|
||||
unsigned char last_key[CryptoAccel::AES_256_KEYSIZE];
|
||||
alignas(16) gcm_key_data cached_gkey;
|
||||
|
||||
~gcm_key_cache_t() {
|
||||
ceph_memzero_s(last_key, sizeof(last_key), sizeof(last_key));
|
||||
ceph_memzero_s(&cached_gkey, sizeof(cached_gkey), sizeof(cached_gkey));
|
||||
}
|
||||
};
|
||||
|
||||
static inline const gcm_key_data* get_cached_gcm_key(const unsigned char* key)
|
||||
{
|
||||
static thread_local std::unique_ptr<gcm_key_cache_t> cache;
|
||||
|
||||
if (!cache)
|
||||
cache = std::make_unique<gcm_key_cache_t>();
|
||||
|
||||
if (memcmp(cache->last_key, key, CryptoAccel::AES_256_KEYSIZE) != 0) {
|
||||
aes_gcm_pre_256(key, &cache->cached_gkey);
|
||||
memcpy(cache->last_key, key, CryptoAccel::AES_256_KEYSIZE);
|
||||
}
|
||||
|
||||
return &cache->cached_gkey;
|
||||
}
|
||||
|
||||
bool ISALCryptoAccel::gcm_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
unsigned char* tag,
|
||||
optional_yield y)
|
||||
{
|
||||
if (!out || !in) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const gcm_key_data* gkey = get_cached_gcm_key(&key[0]);
|
||||
alignas(16) struct gcm_context_data gctx;
|
||||
|
||||
aes_gcm_enc_256(gkey, &gctx,
|
||||
reinterpret_cast<uint8_t*>(out),
|
||||
reinterpret_cast<const uint8_t*>(in),
|
||||
static_cast<uint64_t>(size),
|
||||
const_cast<unsigned char*>(&iv[0]),
|
||||
reinterpret_cast<const uint8_t*>(aad),
|
||||
static_cast<uint64_t>(aad_len),
|
||||
tag, AES_GCM_TAGSIZE);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ISALCryptoAccel::gcm_decrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
const unsigned char* tag,
|
||||
optional_yield y)
|
||||
{
|
||||
if (!out || !in) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const gcm_key_data* gkey = get_cached_gcm_key(&key[0]);
|
||||
alignas(16) struct gcm_context_data gctx;
|
||||
|
||||
unsigned char computed_tag[AES_GCM_TAGSIZE];
|
||||
aes_gcm_dec_256(gkey, &gctx,
|
||||
reinterpret_cast<uint8_t*>(out),
|
||||
reinterpret_cast<const uint8_t*>(in),
|
||||
static_cast<uint64_t>(size),
|
||||
const_cast<unsigned char*>(&iv[0]),
|
||||
reinterpret_cast<const uint8_t*>(aad),
|
||||
static_cast<uint64_t>(aad_len),
|
||||
computed_tag, AES_GCM_TAGSIZE);
|
||||
|
||||
if (memcmp(computed_tag, tag, AES_GCM_TAGSIZE) != 0) {
|
||||
memset(out, 0, size); // Clear output on auth failure
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
@ -38,5 +38,33 @@ class ISALCryptoAccel : public CryptoAccel {
|
||||
const unsigned char iv[][AES_256_IVSIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
|
||||
bool gcm_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
unsigned char* tag,
|
||||
optional_yield y) override;
|
||||
|
||||
bool gcm_decrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
const unsigned char* tag,
|
||||
optional_yield y) override;
|
||||
|
||||
bool gcm_encrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
|
||||
bool gcm_decrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
const unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
};
|
||||
#endif
|
||||
|
||||
@ -103,3 +103,128 @@ bool OpenSSLCryptoAccel::cbc_decrypt(unsigned char* out, const unsigned char* in
|
||||
nullptr, // Hardware acceleration engine can be used in the future
|
||||
EVP_aes_256_cbc(), AES_DECRYPT);
|
||||
}
|
||||
|
||||
bool OpenSSLCryptoAccel::gcm_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
unsigned char* tag,
|
||||
optional_yield y)
|
||||
{
|
||||
using pctx_t = std::unique_ptr<EVP_CIPHER_CTX, decltype(&::EVP_CIPHER_CTX_free)>;
|
||||
pctx_t pctx{ EVP_CIPHER_CTX_new(), EVP_CIPHER_CTX_free };
|
||||
|
||||
if (!pctx) {
|
||||
derr << "failed to create evp cipher context for GCM encrypt" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
int outlen;
|
||||
|
||||
if (EVP_EncryptInit_ex(pctx.get(), EVP_aes_256_gcm(), nullptr, nullptr, nullptr) != EVP_SUCCESS) {
|
||||
derr << "EVP_EncryptInit_ex failed for GCM" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
if (EVP_CIPHER_CTX_ctrl(pctx.get(), EVP_CTRL_GCM_SET_IVLEN, AES_GCM_IV_SIZE, nullptr) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM IV length" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
if (EVP_EncryptInit_ex(pctx.get(), nullptr, nullptr, &key[0], &iv[0]) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM key/IV" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Add AAD (pass nullptr for out to process AAD only)
|
||||
if (aad_len > 0) {
|
||||
if (EVP_EncryptUpdate(pctx.get(), nullptr, &outlen, aad,
|
||||
static_cast<int>(aad_len)) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM AAD" << dendl;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (EVP_EncryptUpdate(pctx.get(), out, &outlen, in,
|
||||
static_cast<int>(size)) != EVP_SUCCESS) {
|
||||
derr << "EVP_EncryptUpdate failed for GCM" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
int final_len;
|
||||
if (EVP_EncryptFinal_ex(pctx.get(), out + outlen, &final_len) != EVP_SUCCESS) {
|
||||
derr << "EVP_EncryptFinal_ex failed for GCM" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
if (EVP_CIPHER_CTX_ctrl(pctx.get(), EVP_CTRL_GCM_GET_TAG, AES_GCM_TAGSIZE, tag) != EVP_SUCCESS) {
|
||||
derr << "failed to get GCM tag" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OpenSSLCryptoAccel::gcm_decrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
const unsigned char* tag,
|
||||
optional_yield y)
|
||||
{
|
||||
using pctx_t = std::unique_ptr<EVP_CIPHER_CTX, decltype(&::EVP_CIPHER_CTX_free)>;
|
||||
pctx_t pctx{ EVP_CIPHER_CTX_new(), EVP_CIPHER_CTX_free };
|
||||
|
||||
if (!pctx) {
|
||||
derr << "failed to create evp cipher context for GCM decrypt" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
int outlen;
|
||||
|
||||
if (EVP_DecryptInit_ex(pctx.get(), EVP_aes_256_gcm(), nullptr, nullptr, nullptr) != EVP_SUCCESS) {
|
||||
derr << "EVP_DecryptInit_ex failed for GCM" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
if (EVP_CIPHER_CTX_ctrl(pctx.get(), EVP_CTRL_GCM_SET_IVLEN, AES_GCM_IV_SIZE, nullptr) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM IV length" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
if (EVP_DecryptInit_ex(pctx.get(), nullptr, nullptr, &key[0], &iv[0]) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM key/IV" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
if (EVP_CIPHER_CTX_ctrl(pctx.get(), EVP_CTRL_GCM_SET_TAG, AES_GCM_TAGSIZE,
|
||||
const_cast<unsigned char*>(tag)) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM expected tag" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Add AAD (must match encryption)
|
||||
if (aad_len > 0) {
|
||||
if (EVP_DecryptUpdate(pctx.get(), nullptr, &outlen, aad,
|
||||
static_cast<int>(aad_len)) != EVP_SUCCESS) {
|
||||
derr << "failed to set GCM AAD" << dendl;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (EVP_DecryptUpdate(pctx.get(), out, &outlen, in,
|
||||
static_cast<int>(size)) != EVP_SUCCESS) {
|
||||
derr << "EVP_DecryptUpdate failed for GCM" << dendl;
|
||||
return false;
|
||||
}
|
||||
|
||||
int final_len;
|
||||
if (EVP_DecryptFinal_ex(pctx.get(), out + outlen, &final_len) != EVP_SUCCESS) {
|
||||
// Authentication failure - tag mismatch
|
||||
derr << "GCM authentication failed - tag mismatch" << dendl;
|
||||
memset(out, 0, size); // Clear output on auth failure
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
@ -39,5 +39,33 @@ class OpenSSLCryptoAccel : public CryptoAccel {
|
||||
const unsigned char iv[][AES_256_IVSIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
|
||||
bool gcm_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
unsigned char* tag,
|
||||
optional_yield y) override;
|
||||
|
||||
bool gcm_decrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
const unsigned char* tag,
|
||||
optional_yield y) override;
|
||||
|
||||
bool gcm_encrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
|
||||
bool gcm_decrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
const unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
};
|
||||
#endif
|
||||
|
||||
@ -42,5 +42,30 @@ class QccCryptoAccel : public CryptoAccel {
|
||||
const unsigned char iv[][AES_256_IVSIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
optional_yield y) override;
|
||||
|
||||
bool gcm_encrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
unsigned char* tag,
|
||||
optional_yield y) override { return false; }
|
||||
bool gcm_decrypt(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char (&iv)[AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* aad, size_t aad_len,
|
||||
const unsigned char* tag,
|
||||
optional_yield y) override { return false; }
|
||||
bool gcm_encrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
bool gcm_decrypt_batch(unsigned char* out, const unsigned char* in, size_t size,
|
||||
const unsigned char iv[][AES_GCM_IV_SIZE],
|
||||
const unsigned char (&key)[AES_256_KEYSIZE],
|
||||
const unsigned char* const aad[], const size_t aad_len[],
|
||||
const unsigned char tag[][AES_GCM_TAGSIZE],
|
||||
optional_yield y) override { return false; }
|
||||
};
|
||||
#endif
|
||||
|
||||
@ -36,6 +36,7 @@
|
||||
#include "rgw_rest_conn.h"
|
||||
#include "rgw_cr_rados.h"
|
||||
#include "rgw_cr_rest.h"
|
||||
#include "rgw_crypt.h"
|
||||
#include "rgw_datalog.h"
|
||||
#include "rgw_putobj_processor.h"
|
||||
#include "rgw_lc_tier.h"
|
||||
@ -3538,8 +3539,28 @@ int RGWRados::Object::Write::_do_write_meta(uint64_t size, uint64_t accounted_si
|
||||
ldpp_dout(rctx.dpp, 0) << "ERROR: complete_atomic_modification returned r=" << r << dendl;
|
||||
}
|
||||
|
||||
/**
|
||||
* For AEAD encryption (GCM), set bucket index sizes:
|
||||
* index_size = encrypted bytes on disk (for size_utilized)
|
||||
* index_accounted_size = plaintext bytes (for quota/listings)
|
||||
*
|
||||
* This is consistent with compression behavior where:
|
||||
* index_size = compressed bytes, index_accounted_size = uncompressed bytes
|
||||
*
|
||||
* For non-AEAD (CBC or unencrypted), both remain as passed in.
|
||||
*/
|
||||
uint64_t index_size, index_accounted_size;
|
||||
{
|
||||
index_size = size;
|
||||
index_accounted_size = accounted_size;
|
||||
uint64_t original_size = 0;
|
||||
if (rgw_get_aead_original_size(rctx.dpp, attrs, &original_size)) {
|
||||
index_accounted_size = original_size;
|
||||
}
|
||||
}
|
||||
|
||||
tracepoint(rgw_rados, complete_enter, req_id.c_str());
|
||||
r = index_op->complete(rctx.dpp, poolid, epoch, size, accounted_size,
|
||||
r = index_op->complete(rctx.dpp, poolid, epoch, index_size, index_accounted_size,
|
||||
meta.set_mtime, etag, content_type,
|
||||
storage_class, meta.owner,
|
||||
meta.category, meta.remove_objs, rctx.y,
|
||||
@ -5157,14 +5178,29 @@ int RGWRados::copy_obj(RGWObjectCtx& src_obj_ctx,
|
||||
src_attrs.erase(RGW_ATTR_OBJ_REPLICATION_TIMESTAMP);
|
||||
src_attrs.erase(RGW_ATTR_OBJ_REPLICATION_STATUS);
|
||||
|
||||
// drop encryption attributes
|
||||
// will be generated by copy_obj_data() if encryption is requested
|
||||
/**
|
||||
* Drop encryption key-related attributes — they will be regenerated by
|
||||
* copy_obj_data() if encryption is requested.
|
||||
*
|
||||
* CRYPT_ORIGINAL_SIZE and CRYPT_PARTS are intentionally preserved here.
|
||||
* In the zero-copy path (!copy_data), the object's raw bytes are not
|
||||
* rewritten, so the encryption state (ciphertext layout, sizes) is
|
||||
* unchanged and these size attributes remain valid. In the copy_data
|
||||
* path, they are erased separately once the data is rewritten as a
|
||||
* single stream.
|
||||
*
|
||||
* CRYPT_PART_NUMS must be erased because copy writes data as a single
|
||||
* stream (part 0), so stale part numbers from a multipart source would
|
||||
* cause wrong key derivation.
|
||||
*/
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_KEYSEL);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_CONTEXT);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_MODE);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_KEYID);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_KEYMD5);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_DATAKEY);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_SALT);
|
||||
src_attrs.erase(RGW_ATTR_CRYPT_PART_NUMS);
|
||||
|
||||
set_copy_attrs(src_attrs, attrs, attrs_mod);
|
||||
attrs.erase(RGW_ATTR_ID_TAG);
|
||||
@ -5260,6 +5296,10 @@ int RGWRados::copy_obj(RGWObjectCtx& src_obj_ctx,
|
||||
|
||||
if (copy_data) { /* refcounting tail wouldn't work here, just copy the data */
|
||||
attrs.erase(RGW_ATTR_TAIL_TAG);
|
||||
// Data is rewritten as a single stream; drop stale multipart boundaries
|
||||
attrs.erase(RGW_ATTR_CRYPT_PARTS);
|
||||
attrs.erase(RGW_ATTR_CRYPT_PART_NUMS);
|
||||
attrs.erase(RGW_ATTR_CRYPT_PREFETCH_ALIGN);
|
||||
return copy_obj_data(dest_obj_ctx, owner, dest_bucket_info, dest_placement, read_op, obj_size - 1, dest_obj,
|
||||
mtime, real_time(), attrs, olh_epoch, delete_at, petag, dp_factory, dpp, y);
|
||||
}
|
||||
@ -7138,6 +7178,26 @@ int RGWRados::get_obj_state_impl(const DoutPrefixProvider *dpp, RGWObjectCtx *oc
|
||||
s->fake_tag = true;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* For AEAD encryption: adjust accounted_size to original plaintext size.
|
||||
* This ensures Content-Length headers and range requests use plaintext size.
|
||||
* Helpers return false for non-AEAD modes (including CBC), so this is a no-op
|
||||
* outside of AEAD.
|
||||
* Must be after manifest handling since manifest->get_obj_size() returns
|
||||
* encrypted size and re-sets accounted_size.
|
||||
*/
|
||||
if (!compressed && s->accounted_size == s->size) {
|
||||
uint64_t original_size = 0;
|
||||
if (rgw_get_aead_original_size(dpp, s->attrset, &original_size)) {
|
||||
s->accounted_size = original_size;
|
||||
} else {
|
||||
uint64_t decrypted_size = 0;
|
||||
if (rgw_get_aead_decrypted_size(dpp, s->attrset, s->size, &decrypted_size)) {
|
||||
s->accounted_size = decrypted_size;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (iter = s->attrset.find(RGW_ATTR_PG_VER); iter != s->attrset.end()) {
|
||||
const bufferlist& pg_ver_bl = iter->second;
|
||||
if (pg_ver_bl.length()) {
|
||||
@ -7943,6 +8003,19 @@ int RGWRados::Object::Read::prepare(optional_yield y, const DoutPrefixProvider *
|
||||
}
|
||||
|
||||
for (auto& iter : src_attrset) {
|
||||
/**
|
||||
* Skip object-level encryption attributes when reading individual parts.
|
||||
* These attrs describe the complete multipart object, not this part:
|
||||
* - ORIGINAL_SIZE: would cause Content-Length mismatch
|
||||
* - PARTS: contains sizes of all parts, not applicable to single part
|
||||
* - PART_NUMS: maps part indices to S3 part numbers for the full object
|
||||
*/
|
||||
if (iter.first == RGW_ATTR_CRYPT_ORIGINAL_SIZE ||
|
||||
iter.first == RGW_ATTR_CRYPT_PARTS ||
|
||||
iter.first == RGW_ATTR_CRYPT_PART_NUMS) {
|
||||
ldpp_dout(dpp, 4) << "skip crypt attr for part read: " << iter.first << dendl;
|
||||
continue;
|
||||
}
|
||||
ldpp_dout(dpp, 4) << "copy crypt attr: " << iter.first << dendl;
|
||||
if (astate->attrset.find(iter.first) == astate->attrset.end()) {
|
||||
astate->attrset[iter.first] = std::move(iter.second);
|
||||
|
||||
@ -40,6 +40,7 @@
|
||||
#include "rgw_aio_throttle.h"
|
||||
#include "rgw_bucket.h"
|
||||
#include "rgw_bucket_logging.h"
|
||||
#include "rgw_crypt.h"
|
||||
#include "rgw_bl_rados.h"
|
||||
#include "rgw_lc.h"
|
||||
#include "rgw_lc_tier.h"
|
||||
@ -4279,6 +4280,15 @@ int RadosMultipartUpload::complete(const DoutPrefixProvider *dpp,
|
||||
auto etags_iter = part_etags.begin();
|
||||
rgw::sal::Attrs& attrs = target_obj->get_attrs();
|
||||
|
||||
// Check if this is AEAD encryption to track plaintext size
|
||||
bool is_aead = false;
|
||||
uint64_t plaintext_ofs = 0;
|
||||
auto mode_iter = attrs.find(RGW_ATTR_CRYPT_MODE);
|
||||
if (mode_iter != attrs.end()) {
|
||||
std::string crypt_mode = mode_iter->second.to_str();
|
||||
is_aead = is_aead_mode(crypt_mode);
|
||||
}
|
||||
|
||||
do {
|
||||
ret = list_parts(dpp, cct, max_parts, marker, &marker, &truncated, y);
|
||||
if (ret == -ENOENT) {
|
||||
@ -4394,6 +4404,16 @@ int RadosMultipartUpload::complete(const DoutPrefixProvider *dpp,
|
||||
|
||||
ofs += obj_part.size;
|
||||
accounted_size += obj_part.accounted_size;
|
||||
|
||||
// Track plaintext size for AEAD encryption
|
||||
if (is_aead) {
|
||||
if (part_compressed) {
|
||||
// For compressed parts, use the uncompressed size directly
|
||||
plaintext_ofs += obj_part.accounted_size;
|
||||
} else {
|
||||
plaintext_ofs += aead_encrypted_to_plaintext_size(obj_part.size);
|
||||
}
|
||||
}
|
||||
}
|
||||
} while (truncated);
|
||||
hash.Final((unsigned char *)final_etag);
|
||||
@ -4432,6 +4452,26 @@ int RadosMultipartUpload::complete(const DoutPrefixProvider *dpp,
|
||||
attrs[RGW_ATTR_COMPRESSION] = tmp;
|
||||
}
|
||||
|
||||
// For AEAD encryption: store total plaintext size (calculated during loop)
|
||||
if (is_aead) {
|
||||
bufferlist bl;
|
||||
bl.append(std::to_string(plaintext_ofs));
|
||||
attrs[RGW_ATTR_CRYPT_ORIGINAL_SIZE] = std::move(bl);
|
||||
|
||||
// Store actual S3 part numbers for correct IV/key derivation during decrypt
|
||||
std::vector<uint32_t> part_nums;
|
||||
part_nums.reserve(part_etags.size());
|
||||
for (const auto& part : part_etags) {
|
||||
part_nums.push_back(static_cast<uint32_t>(part.first));
|
||||
}
|
||||
bufferlist part_nums_bl;
|
||||
using ceph::encode;
|
||||
encode(part_nums, part_nums_bl);
|
||||
attrs[RGW_ATTR_CRYPT_PART_NUMS] = std::move(part_nums_bl);
|
||||
ldpp_dout(dpp, 20) << "Stored CRYPT_PART_NUMS with " << part_nums.size()
|
||||
<< " parts" << dendl;
|
||||
}
|
||||
|
||||
target_obj->set_atomic(true);
|
||||
|
||||
const RGWBucketInfo& bucket_info = target_obj->get_bucket()->get_info();
|
||||
|
||||
@ -191,6 +191,10 @@ using ceph::crypto::MD5;
|
||||
#define RGW_ATTR_CRYPT_CONTEXT RGW_ATTR_CRYPT_PREFIX "context"
|
||||
#define RGW_ATTR_CRYPT_DATAKEY RGW_ATTR_CRYPT_PREFIX "datakey"
|
||||
#define RGW_ATTR_CRYPT_PARTS RGW_ATTR_CRYPT_PREFIX "part-lengths"
|
||||
#define RGW_ATTR_CRYPT_PART_NUMS RGW_ATTR_CRYPT_PREFIX "part-numbers"
|
||||
#define RGW_ATTR_CRYPT_SALT RGW_ATTR_CRYPT_PREFIX "salt"
|
||||
#define RGW_ATTR_CRYPT_ORIGINAL_SIZE RGW_ATTR_CRYPT_PREFIX "original-size"
|
||||
#define RGW_ATTR_CRYPT_PREFETCH_ALIGN RGW_ATTR_CRYPT_PREFIX "prefetch-align"
|
||||
|
||||
/* SSE-S3 Encryption Attributes */
|
||||
#define RGW_ATTR_BUCKET_ENCRYPTION_PREFIX RGW_ATTR_PREFIX "sse-s3."
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
// vim: ts=8 sw=2 sts=2 expandtab ft=cpp
|
||||
|
||||
#include "rgw_compression.h"
|
||||
#include "rgw_range_projection.h"
|
||||
|
||||
#define dout_subsys ceph_subsys_rgw
|
||||
|
||||
@ -182,38 +183,18 @@ int RGWGetObj_Decompress::handle_data(bufferlist& bl, off_t bl_ofs, off_t bl_len
|
||||
|
||||
int RGWGetObj_Decompress::fixup_range(off_t& ofs, off_t& end)
|
||||
{
|
||||
if (partial_content) {
|
||||
// if user set range, we need to calculate it in decompressed data
|
||||
first_block = cs_info->blocks.begin(); last_block = cs_info->blocks.begin();
|
||||
if (cs_info->blocks.size() > 1) {
|
||||
vector<compression_block>::iterator fb, lb;
|
||||
// not bad to use auto for lambda, I think
|
||||
auto cmp_u = [] (off_t ofs, const compression_block& e) { return (uint64_t)ofs < e.old_ofs; };
|
||||
auto cmp_l = [] (const compression_block& e, off_t ofs) { return e.old_ofs <= (uint64_t)ofs; };
|
||||
fb = upper_bound(cs_info->blocks.begin()+1,
|
||||
cs_info->blocks.end(),
|
||||
ofs,
|
||||
cmp_u);
|
||||
first_block = fb - 1;
|
||||
lb = lower_bound(fb,
|
||||
cs_info->blocks.end(),
|
||||
end,
|
||||
cmp_l);
|
||||
last_block = lb - 1;
|
||||
}
|
||||
} else {
|
||||
first_block = cs_info->blocks.begin(); last_block = cs_info->blocks.end() - 1;
|
||||
}
|
||||
auto result = project_compress_range(ofs, end, *cs_info, partial_content);
|
||||
|
||||
q_ofs = ofs - first_block->old_ofs;
|
||||
q_len = end + 1 - ofs;
|
||||
|
||||
ofs = first_block->new_ofs;
|
||||
end = last_block->new_ofs + last_block->len - 1;
|
||||
|
||||
cur_ofs = ofs;
|
||||
first_block = cs_info->blocks.begin() + result.first_block_idx;
|
||||
last_block = cs_info->blocks.begin() + result.last_block_idx;
|
||||
q_ofs = result.q_ofs;
|
||||
q_len = result.q_len;
|
||||
cur_ofs = result.ofs;
|
||||
waiting.clear();
|
||||
|
||||
ofs = result.ofs;
|
||||
end = result.end;
|
||||
|
||||
return next->fixup_range(ofs, end);
|
||||
}
|
||||
|
||||
|
||||
1677
src/rgw/rgw_crypt.cc
1677
src/rgw/rgw_crypt.cc
File diff suppressed because it is too large
Load Diff
@ -15,6 +15,16 @@
|
||||
#include "rgw_putobj.h"
|
||||
#include "common/async/yield_context.h"
|
||||
|
||||
struct rgw_crypt_src_identity {
|
||||
std::string bucket_id;
|
||||
std::string bucket;
|
||||
std::string object;
|
||||
|
||||
bool valid() const {
|
||||
return !bucket_id.empty() && !bucket.empty() && !object.empty();
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* \brief Interface for block encryption methods
|
||||
*
|
||||
@ -38,6 +48,15 @@ public:
|
||||
*/
|
||||
virtual size_t get_block_size() = 0;
|
||||
|
||||
/**
|
||||
* Returns size of encrypted block (ciphertext + metadata like auth tags).
|
||||
* For most ciphers this equals get_block_size(), but for AEAD modes like GCM
|
||||
* it includes the authentication tag.
|
||||
*/
|
||||
virtual size_t get_encrypted_block_size() {
|
||||
return get_block_size();
|
||||
}
|
||||
|
||||
/**
|
||||
* Encrypts data.
|
||||
* Argument \ref stream_offset shows where in generalized stream chunk is located.
|
||||
@ -79,9 +98,127 @@ public:
|
||||
bufferlist& output,
|
||||
off_t stream_offset,
|
||||
optional_yield y) = 0;
|
||||
|
||||
/**
|
||||
* Set the part number for multipart object decryption.
|
||||
* AEAD modes use this for per-part IV derivation.
|
||||
* Default is no-op; CBC derives IVs from block offsets instead.
|
||||
*/
|
||||
virtual void set_part_number(uint32_t part_number) {}
|
||||
};
|
||||
|
||||
static const size_t AES_256_KEYSIZE = 256 / 8;
|
||||
static const size_t AES_256_GCM_IV_SIZE = 96 / 8; // 12 bytes, GCM standard
|
||||
static const size_t AES_256_GCM_SALT_SIZE = 32; // 256-bit random salt for HMAC-based key derivation
|
||||
|
||||
/**
|
||||
* AEAD chunk size constants used for size calculations across RGW.
|
||||
* All supported AEAD ciphers use 128-bit (16-byte) authentication tags.
|
||||
*/
|
||||
static constexpr size_t AEAD_CHUNK_SIZE = 4096;
|
||||
static constexpr size_t AEAD_TAG_SIZE = 16;
|
||||
static constexpr size_t AEAD_ENCRYPTED_CHUNK_SIZE = AEAD_CHUNK_SIZE + AEAD_TAG_SIZE; // 4112
|
||||
|
||||
/**
|
||||
* Check if encryption mode is AEAD.
|
||||
* AEAD modes have ciphertext expansion from auth tags and need special
|
||||
* handling for size calculations and multipart part numbers.
|
||||
*
|
||||
* All AEAD modes currently end in "-GCM". When adding non-GCM AEAD modes
|
||||
* (e.g., ChaCha20-Poly1305), update this function to match them.
|
||||
*/
|
||||
inline bool is_aead_mode(const std::string& mode) {
|
||||
return mode.size() >= 4 && mode.compare(mode.size() - 4, 4, "-GCM") == 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if encryption mode string indicates CBC mode.
|
||||
* CBC modes: SSE-C-AES256, SSE-KMS, RGW-AUTO, AES256
|
||||
*/
|
||||
inline bool is_cbc_mode(const std::string& mode) {
|
||||
return mode == "SSE-C-AES256" || mode == "SSE-KMS" ||
|
||||
mode == "RGW-AUTO" || mode == "AES256";
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert encrypted size to plaintext size for AEAD modes.
|
||||
* Each AEAD_CHUNK_SIZE-byte plaintext chunk becomes AEAD_ENCRYPTED_CHUNK_SIZE bytes
|
||||
* (plaintext + AEAD_TAG_SIZE-byte auth tag).
|
||||
* Pass dpp to enable warning logging for malformed data.
|
||||
*/
|
||||
inline uint64_t aead_encrypted_to_plaintext_size(uint64_t encrypted_size,
|
||||
const DoutPrefixProvider* dpp = nullptr) {
|
||||
uint64_t full_chunks = encrypted_size / AEAD_ENCRYPTED_CHUNK_SIZE;
|
||||
uint64_t remainder = encrypted_size % AEAD_ENCRYPTED_CHUNK_SIZE;
|
||||
|
||||
if (remainder > 0 && remainder <= AEAD_TAG_SIZE) {
|
||||
// Malformed: partial chunk has no ciphertext, only tag bytes
|
||||
if (dpp) {
|
||||
ldpp_dout(dpp, 1) << "WARNING: aead_encrypted_to_plaintext_size: "
|
||||
<< "partial chunk size " << remainder
|
||||
<< " is <= tag size " << AEAD_TAG_SIZE
|
||||
<< " - data may be corrupted" << dendl;
|
||||
}
|
||||
return full_chunks * AEAD_CHUNK_SIZE;
|
||||
}
|
||||
|
||||
uint64_t partial = (remainder > AEAD_TAG_SIZE) ? (remainder - AEAD_TAG_SIZE) : 0;
|
||||
return full_chunks * AEAD_CHUNK_SIZE + partial;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert plaintext size to encrypted size for AEAD modes.
|
||||
* Each AEAD_CHUNK_SIZE-byte plaintext chunk becomes AEAD_ENCRYPTED_CHUNK_SIZE bytes.
|
||||
*/
|
||||
inline uint64_t aead_plaintext_to_encrypted_size(uint64_t plaintext_size) {
|
||||
if (plaintext_size == 0) return 0;
|
||||
uint64_t num_chunks = (plaintext_size + AEAD_CHUNK_SIZE - 1) / AEAD_CHUNK_SIZE;
|
||||
return plaintext_size + (num_chunks * AEAD_TAG_SIZE);
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert plaintext offset to encrypted offset for AEAD modes.
|
||||
* Accounts for AEAD_TAG_SIZE-byte auth tag per chunk.
|
||||
*/
|
||||
inline uint64_t aead_plaintext_to_encrypted_offset(uint64_t plaintext_ofs) {
|
||||
uint64_t chunk_idx = plaintext_ofs / AEAD_CHUNK_SIZE;
|
||||
uint64_t offset_in_chunk = plaintext_ofs % AEAD_CHUNK_SIZE;
|
||||
return chunk_idx * AEAD_ENCRYPTED_CHUNK_SIZE + offset_in_chunk;
|
||||
}
|
||||
|
||||
/*
|
||||
* Generic cipher geometry helpers (parameterized by block sizes).
|
||||
* CBC: block_size == enc_block_size (identity).
|
||||
* AEAD: enc_block_size > block_size (expansion).
|
||||
*/
|
||||
|
||||
inline off_t crypt_logical_to_enc_offset(off_t logical_ofs,
|
||||
size_t block_size,
|
||||
size_t enc_block_size) {
|
||||
if (block_size == enc_block_size) return logical_ofs;
|
||||
uint64_t chunk_idx = logical_ofs / block_size;
|
||||
uint64_t offset_in_chunk = logical_ofs % block_size;
|
||||
return chunk_idx * enc_block_size + offset_in_chunk;
|
||||
}
|
||||
|
||||
inline off_t crypt_align_enc_block_end(off_t enc_ofs,
|
||||
size_t block_size,
|
||||
size_t enc_block_size) {
|
||||
if (block_size == enc_block_size) return enc_ofs;
|
||||
return (enc_ofs / enc_block_size) * enc_block_size + (enc_block_size - 1);
|
||||
}
|
||||
|
||||
inline size_t crypt_enc_to_plaintext_size(size_t encrypted_size,
|
||||
size_t block_size,
|
||||
size_t enc_block_size) {
|
||||
if (block_size == enc_block_size) return encrypted_size;
|
||||
uint64_t full_chunks = encrypted_size / enc_block_size;
|
||||
uint64_t remainder = encrypted_size % enc_block_size;
|
||||
uint64_t tag_size = enc_block_size - block_size;
|
||||
uint64_t partial = (remainder > tag_size) ? (remainder - tag_size) : 0;
|
||||
return full_chunks * block_size + partial;
|
||||
}
|
||||
|
||||
bool AES_256_ECB_encrypt(const DoutPrefixProvider* dpp,
|
||||
CephContext* cct,
|
||||
const uint8_t* key,
|
||||
@ -90,28 +227,81 @@ bool AES_256_ECB_encrypt(const DoutPrefixProvider* dpp,
|
||||
uint8_t* data_out,
|
||||
size_t data_size);
|
||||
|
||||
/**
|
||||
* Create an AES-256-GCM BlockCrypt instance.
|
||||
*
|
||||
* For encryption: Pass salt=nullptr to generate a random 32-byte salt.
|
||||
* After creation, call AES_256_GCM_get_salt() to retrieve it for storage.
|
||||
*
|
||||
* For decryption: Pass the stored salt from RGW_ATTR_CRYPT_SALT.
|
||||
*/
|
||||
std::unique_ptr<BlockCrypt> AES_256_GCM_create(const DoutPrefixProvider* dpp,
|
||||
CephContext* cct,
|
||||
const uint8_t* key,
|
||||
size_t key_len,
|
||||
const uint8_t* salt = nullptr,
|
||||
size_t salt_len = 0,
|
||||
uint32_t part_number = 0);
|
||||
|
||||
/**
|
||||
* Retrieve the salt from a BlockCrypt instance for storage in RGW_ATTR_CRYPT_SALT.
|
||||
* Returns empty string if the BlockCrypt is not an AES_256_GCM instance.
|
||||
*/
|
||||
std::string AES_256_GCM_get_salt(BlockCrypt* block_crypt);
|
||||
|
||||
class RGWGetObj_BlockDecrypt : public RGWGetObj_Filter {
|
||||
friend class TestableBlockDecrypt; // For unit testing private members
|
||||
const DoutPrefixProvider *dpp;
|
||||
CephContext* cct;
|
||||
std::unique_ptr<BlockCrypt> crypt; /**< already configured stateless BlockCrypt
|
||||
for operations when enough data is accumulated */
|
||||
off_t enc_begin_skip; /**< amount of data to skip from beginning of received data */
|
||||
off_t ofs; /**< stream offset of data we expect to show up next through \ref handle_data */
|
||||
off_t ofs; /**< plaintext stream offset of data we expect to show up next through \ref handle_data */
|
||||
off_t enc_ofs; /**< encrypted stream offset, for comparing against parts_len which contains encrypted sizes */
|
||||
off_t end; /**< stream offset of last byte that is requested */
|
||||
off_t encrypted_total_size; /**< total encrypted object size (for clamping ranges in fixup_range) */
|
||||
bool has_compression{false}; /**< true if a decompression filter is present downstream */
|
||||
bufferlist cache; /**< stores extra data that could not (yet) be processed by BlockCrypt */
|
||||
size_t block_size; /**< snapshot of \ref BlockCrypt.get_block_size() */
|
||||
size_t block_size; /**< snapshot of \ref BlockCrypt.get_block_size() (plaintext block size) */
|
||||
size_t encrypted_block_size; /**< snapshot of \ref BlockCrypt.get_encrypted_block_size() (includes auth tag for GCM) */
|
||||
optional_yield y;
|
||||
std::vector<size_t> parts_len; /**< size of parts of multipart object, parsed from manifest */
|
||||
std::vector<uint32_t> part_nums; /**< actual S3 part numbers for multipart (e.g., [1,3,5]) */
|
||||
uint32_t current_part_num = 0; /**< current part number (1-based, 0 means single-part object) */
|
||||
|
||||
int process(bufferlist& cipher, size_t part_ofs, size_t size);
|
||||
|
||||
/**
|
||||
* Process cached data across part boundaries.
|
||||
* Updates current_part_num and calls crypt->set_part_number() as needed.
|
||||
* Returns 0 on success, negative error code on failure.
|
||||
* On success, plain_part_ofs_out contains the plaintext offset for remaining data.
|
||||
*/
|
||||
int process_part_boundaries(size_t& plain_part_ofs_out);
|
||||
|
||||
size_t encrypted_to_plaintext_size(size_t encrypted_size) const {
|
||||
return crypt_enc_to_plaintext_size(encrypted_size, block_size, encrypted_block_size);
|
||||
}
|
||||
|
||||
public:
|
||||
RGWGetObj_BlockDecrypt(const DoutPrefixProvider *dpp,
|
||||
CephContext* cct,
|
||||
RGWGetObj_Filter* next,
|
||||
std::unique_ptr<BlockCrypt> crypt,
|
||||
std::vector<size_t> parts_len,
|
||||
std::vector<uint32_t> part_nums,
|
||||
off_t encrypted_total_size,
|
||||
bool has_compression,
|
||||
optional_yield y);
|
||||
// Backward-compatible constructor for CBC mode (no size expansion)
|
||||
RGWGetObj_BlockDecrypt(const DoutPrefixProvider *dpp,
|
||||
CephContext* cct,
|
||||
RGWGetObj_Filter* next,
|
||||
std::unique_ptr<BlockCrypt> crypt,
|
||||
std::vector<size_t> parts_len,
|
||||
optional_yield y)
|
||||
: RGWGetObj_BlockDecrypt(dpp, cct, next, std::move(crypt),
|
||||
std::move(parts_len), {}, 0, false, y) {}
|
||||
virtual ~RGWGetObj_BlockDecrypt();
|
||||
|
||||
virtual int fixup_range(off_t& bl_ofs,
|
||||
@ -124,6 +314,23 @@ public:
|
||||
static int read_manifest_parts(const DoutPrefixProvider *dpp,
|
||||
const bufferlist& manifest_bl,
|
||||
std::vector<size_t>& parts_len);
|
||||
|
||||
/**
|
||||
* Returns true if this cipher expands the data size (e.g., AEAD adds auth tags).
|
||||
* Used to determine if obj_size needs adjustment for Content-Length.
|
||||
*/
|
||||
bool has_size_expansion() const {
|
||||
return block_size != encrypted_block_size;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate the plaintext size from encrypted size.
|
||||
* For AEAD: removes the 16-byte auth tag overhead per chunk.
|
||||
* Public wrapper for use by callers that need to adjust Content-Length.
|
||||
*/
|
||||
uint64_t get_plaintext_size(uint64_t encrypted_size) const {
|
||||
return encrypted_to_plaintext_size(encrypted_size);
|
||||
}
|
||||
}; /* RGWGetObj_BlockDecrypt */
|
||||
|
||||
|
||||
@ -134,7 +341,8 @@ class RGWPutObj_BlockEncrypt : public rgw::putobj::Pipe
|
||||
std::unique_ptr<BlockCrypt> crypt; /**< already configured stateless BlockCrypt
|
||||
for operations when enough data is accumulated */
|
||||
bufferlist cache; /**< stores extra data that could not (yet) be processed by BlockCrypt */
|
||||
const size_t block_size; /**< snapshot of \ref BlockCrypt.get_block_size() */
|
||||
const size_t block_size; /**< snapshot of \ref BlockCrypt.get_block_size() (plaintext block size) */
|
||||
uint64_t encrypted_offset = 0; /**< tracks write position in encrypted stream (differs from plaintext for GCM) */
|
||||
optional_yield y;
|
||||
public:
|
||||
RGWPutObj_BlockEncrypt(const DoutPrefixProvider *dpp,
|
||||
@ -151,13 +359,37 @@ int rgw_s3_prepare_encrypt(req_state* s, optional_yield y,
|
||||
std::map<std::string, ceph::bufferlist>& attrs,
|
||||
std::unique_ptr<BlockCrypt>* block_crypt,
|
||||
std::map<std::string,
|
||||
std::string>& crypt_http_responses);
|
||||
std::string>& crypt_http_responses,
|
||||
uint32_t part_number = 0);
|
||||
|
||||
int rgw_s3_prepare_decrypt(req_state* s, optional_yield y,
|
||||
std::map<std::string, ceph::bufferlist>& attrs,
|
||||
std::unique_ptr<BlockCrypt>* block_crypt,
|
||||
std::map<std::string, std::string>* crypt_http_responses,
|
||||
bool copy_source);
|
||||
bool copy_source,
|
||||
uint32_t part_number = 0,
|
||||
const rgw_crypt_src_identity* src_identity = nullptr);
|
||||
|
||||
/**
|
||||
* Get the original (uncompressed, unencrypted) size from ORIGINAL_SIZE attr.
|
||||
* Use for: Content-Length, bucket index, quota.
|
||||
* Returns false if attr missing, not AEAD mode, or parse failure.
|
||||
*/
|
||||
bool rgw_get_aead_original_size(const DoutPrefixProvider* dpp,
|
||||
const std::map<std::string, ceph::bufferlist>& attrs,
|
||||
uint64_t* original_size);
|
||||
|
||||
/**
|
||||
* Get the decrypted (but possibly still compressed) size.
|
||||
* Uses CRYPT_PARTS if available, otherwise calculates from encrypted size.
|
||||
* Use for: Copy path where data stays in compressed domain.
|
||||
* WARNING: Never use when caller expects original (uncompressed) size AND
|
||||
* compression may be present. This yields compressed-domain sizes.
|
||||
*/
|
||||
bool rgw_get_aead_decrypted_size(const DoutPrefixProvider* dpp,
|
||||
const std::map<std::string, ceph::bufferlist>& attrs,
|
||||
uint64_t encrypted_size,
|
||||
uint64_t* decrypted_size);
|
||||
|
||||
static inline void set_attr(std::map<std::string, bufferlist>& attrs,
|
||||
const char* key,
|
||||
|
||||
@ -2085,12 +2085,21 @@ int RGWGetObj::read_user_manifest_part(rgw::sal::Bucket* bucket,
|
||||
}
|
||||
else
|
||||
{
|
||||
if (part->get_size() != ent.meta.size) {
|
||||
/*
|
||||
* AEAD on-disk size includes auth tags; use the stored plaintext
|
||||
* size for comparison against the bucket index entry.
|
||||
*/
|
||||
uint64_t obj_size = part->get_size();
|
||||
uint64_t original_size = 0;
|
||||
if (rgw_get_aead_original_size(this, part->get_attrs(), &original_size)) {
|
||||
obj_size = original_size;
|
||||
}
|
||||
if (obj_size != ent.meta.size) {
|
||||
// hmm.. something wrong, object not as expected, abort!
|
||||
ldpp_dout(this, 0) << "ERROR: expected obj_size=" << part->get_size()
|
||||
ldpp_dout(this, 0) << "ERROR: expected obj_size=" << obj_size
|
||||
<< ", actual read size=" << ent.meta.size << dendl;
|
||||
return -EIO;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
op_ret = rgw_policy_from_attrset(s, s->cct, part->get_attrs(), &obj_policy);
|
||||
@ -2613,6 +2622,29 @@ static inline void rgw_cond_decode_objtags(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate the correct object size for AEAD modes.
|
||||
*
|
||||
* Used for range request and Content-Length handling. When compression is
|
||||
* present, stays in the compressed domain and avoids using ORIGINAL_SIZE
|
||||
* since the compressed->encrypted pipeline differs from plaintext->encrypted.
|
||||
*/
|
||||
static bool rgw_calc_aead_obj_size(const DoutPrefixProvider* dpp,
|
||||
const std::map<std::string, bufferlist>& attrs,
|
||||
uint64_t encrypted_size,
|
||||
bool has_compression,
|
||||
uint64_t* out_size)
|
||||
{
|
||||
if (!out_size) {
|
||||
return false;
|
||||
}
|
||||
if (!has_compression &&
|
||||
rgw_get_aead_original_size(dpp, attrs, out_size)) {
|
||||
return true;
|
||||
}
|
||||
return rgw_get_aead_decrypted_size(dpp, attrs, encrypted_size, out_size);
|
||||
}
|
||||
|
||||
void RGWGetObj::execute(optional_yield y)
|
||||
{
|
||||
bufferlist bl;
|
||||
@ -2667,6 +2699,9 @@ void RGWGetObj::execute(optional_yield y)
|
||||
op_ret = read_op->prepare(s->yield, this);
|
||||
version_id = s->object->get_instance();
|
||||
s->obj_size = s->object->get_size();
|
||||
// Preserve encrypted size before compression/decompression modifies s->obj_size
|
||||
// (needed for AEAD decrypt filter range clamping)
|
||||
encrypted_obj_size = s->obj_size;
|
||||
attrs = s->object->get_attrs();
|
||||
multipart_parts_count = read_op->params.parts_count;
|
||||
if (op_ret < 0)
|
||||
@ -2683,11 +2718,15 @@ void RGWGetObj::execute(optional_yield y)
|
||||
/* start gettorrent */
|
||||
if (get_torrent) {
|
||||
attr_iter = attrs.find(RGW_ATTR_CRYPT_MODE);
|
||||
if (attr_iter != attrs.end() && attr_iter->second.to_str() == "SSE-C-AES256") {
|
||||
ldpp_dout(this, 0) << "ERROR: torrents are not supported for objects "
|
||||
"encrypted with SSE-C" << dendl;
|
||||
op_ret = -EINVAL;
|
||||
goto done_err;
|
||||
if (attr_iter != attrs.end()) {
|
||||
std::string crypt_mode = attr_iter->second.to_str();
|
||||
// Block torrents for any SSE-C mode (SSE-C-AES256, SSE-C-AES256-GCM, etc.)
|
||||
if (crypt_mode.compare(0, 5, "SSE-C") == 0) {
|
||||
ldpp_dout(this, 0) << "ERROR: torrents are not supported for objects "
|
||||
"encrypted with SSE-C" << dendl;
|
||||
op_ret = -EINVAL;
|
||||
goto done_err;
|
||||
}
|
||||
}
|
||||
// read torrent info from attr
|
||||
bufferlist torrentbl;
|
||||
@ -2811,6 +2850,24 @@ void RGWGetObj::execute(optional_yield y)
|
||||
return;
|
||||
}
|
||||
|
||||
/**
|
||||
* For AEAD encryption: use original size for Content-Length/ranges.
|
||||
* Key rule: compression active => never use AEAD decrypted fallback.
|
||||
* Use encrypted_obj_size (saved earlier) as the raw encrypted input.
|
||||
*/
|
||||
if (encrypted && !skip_decrypt) {
|
||||
if (need_decompress) {
|
||||
// compression active: cs_info.orig_size already set obj_size
|
||||
} else {
|
||||
// no compression: try ORIGINAL_SIZE, then decrypted fallback
|
||||
uint64_t size = 0;
|
||||
if (rgw_calc_aead_obj_size(this, attrs, encrypted_obj_size, false, &size)) {
|
||||
s->obj_size = size;
|
||||
s->object->set_obj_size(s->obj_size);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// for range requests with obj size 0
|
||||
if (range_str && !(s->obj_size)) {
|
||||
total_len = 0;
|
||||
@ -4516,6 +4573,7 @@ int RGWPutObj::get_data(const off_t fst, const off_t lst, bufferlist& bl)
|
||||
return ret;
|
||||
|
||||
obj_size = obj->get_size();
|
||||
uint64_t encrypted_size = obj_size; // capture before any mutation
|
||||
|
||||
bool need_decompress;
|
||||
op_ret = rgw_compression_info_from_attrset(obj->get_attrs(), need_decompress, cs_info);
|
||||
@ -4544,6 +4602,23 @@ int RGWPutObj::get_data(const off_t fst, const off_t lst, bufferlist& bl)
|
||||
return op_ret;
|
||||
}
|
||||
|
||||
/**
|
||||
* For AEAD encryption: adjust obj_size for range validation.
|
||||
* Key rule: compression active => never use AEAD decrypted fallback.
|
||||
*/
|
||||
if (decrypt != nullptr) {
|
||||
if (need_decompress) {
|
||||
// compression active: cs_info.orig_size already set obj_size
|
||||
} else {
|
||||
// no compression: try ORIGINAL_SIZE, then decrypted fallback (safe)
|
||||
uint64_t size = 0;
|
||||
if (rgw_calc_aead_obj_size(this, obj->get_attrs(), encrypted_size,
|
||||
obj->get_attrs().count(RGW_ATTR_COMPRESSION), &size)) {
|
||||
obj_size = size;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ret = obj->range_to_ofs(obj_size, new_ofs, new_end);
|
||||
if (ret < 0)
|
||||
return ret;
|
||||
@ -4938,6 +5013,20 @@ void RGWPutObj::execute(optional_yield y)
|
||||
s->obj_size = ofs;
|
||||
s->object->set_obj_size(ofs);
|
||||
|
||||
/* For AEAD modes, ensure ORIGINAL_SIZE is set now that final size is known.
|
||||
* This handles cases where size was unknown at encryption setup:
|
||||
* - Chunked uploads without x-amz-decoded-content-length
|
||||
* - Copy-source-range operations
|
||||
* Without this, bucket index would get size=0 for chunked AEAD uploads. */
|
||||
{
|
||||
const auto mode = get_str_attribute(attrs, RGW_ATTR_CRYPT_MODE);
|
||||
if (is_aead_mode(mode)) {
|
||||
if (attrs.find(RGW_ATTR_CRYPT_ORIGINAL_SIZE) == attrs.end()) {
|
||||
set_attr(attrs, RGW_ATTR_CRYPT_ORIGINAL_SIZE, std::to_string(s->obj_size));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
rgw::op_counters::inc(counters, l_rgw_op_put_obj_b, s->obj_size);
|
||||
|
||||
op_ret = do_aws4_auth_completion();
|
||||
@ -5322,6 +5411,16 @@ void RGWPostObj::execute(optional_yield y)
|
||||
s->object->set_obj_size(ofs);
|
||||
obj->set_obj_size(ofs);
|
||||
|
||||
/* For AEAD modes, always overwrite ORIGINAL_SIZE with the actual file
|
||||
* payload size. set_gcm_plaintext_size() stored s->content_length which,
|
||||
* for POST form uploads, is the entire HTTP body (form fields + boundaries
|
||||
* + file data) — not just the file payload. */
|
||||
{
|
||||
const auto mode = get_str_attribute(attrs, RGW_ATTR_CRYPT_MODE);
|
||||
if (is_aead_mode(mode)) {
|
||||
set_attr(attrs, RGW_ATTR_CRYPT_ORIGINAL_SIZE, std::to_string(s->obj_size));
|
||||
}
|
||||
}
|
||||
|
||||
op_ret = s->bucket->check_quota(this, quota, s->obj_size, y);
|
||||
if (op_ret < 0) {
|
||||
@ -6002,22 +6101,26 @@ public:
|
||||
}
|
||||
|
||||
bool src_encrypted = s->src_object->get_attrs().count(RGW_ATTR_CRYPT_MODE);
|
||||
// Preserve encrypted size for decrypt range clamping before any plaintext conversion.
|
||||
off_t encrypted_total_size = obj_size;
|
||||
|
||||
// Create decompress filter if source is compressed.
|
||||
// Must be created BEFORE decrypt so the chain is: decrypt → decompress → cb
|
||||
if (need_decompress) {
|
||||
obj_size = decompress_info.orig_size;
|
||||
s->src_object->set_obj_size(obj_size);
|
||||
static constexpr bool partial_content = false;
|
||||
decompress.emplace(s->cct, &decompress_info, partial_content, filter);
|
||||
filter = &*decompress;
|
||||
end_x = obj_size;
|
||||
}
|
||||
|
||||
// decrypt
|
||||
if (src_encrypted) {
|
||||
auto attr_iter = s->src_object->get_attrs().find(RGW_ATTR_MANIFEST);
|
||||
static constexpr bool copy_source = true;
|
||||
|
||||
// part_num=0 for copy source (full object read)
|
||||
ret = get_decrypt_filter(&decrypt, filter, s, s->src_object->get_attrs(),
|
||||
attr_iter != s->src_object->get_attrs().end() ? &attr_iter->second : nullptr,
|
||||
nullptr, copy_source);
|
||||
nullptr, copy_source, 0, encrypted_total_size);
|
||||
if (ret < 0) {
|
||||
return ret;
|
||||
}
|
||||
@ -6026,6 +6129,22 @@ public:
|
||||
}
|
||||
}
|
||||
|
||||
// Set obj_size to the final output size (plaintext) for range handling.
|
||||
if (need_decompress) {
|
||||
obj_size = decompress_info.orig_size;
|
||||
s->src_object->set_obj_size(obj_size);
|
||||
end_x = obj_size;
|
||||
} else if (src_encrypted) {
|
||||
uint64_t decrypted_size = 0;
|
||||
const auto& src_attrs = s->src_object->get_attrs();
|
||||
if (rgw_calc_aead_obj_size(dpp, src_attrs, encrypted_total_size,
|
||||
src_attrs.count(RGW_ATTR_COMPRESSION), &decrypted_size)) {
|
||||
obj_size = decrypted_size;
|
||||
s->src_object->set_obj_size(obj_size);
|
||||
end_x = obj_size;
|
||||
}
|
||||
}
|
||||
|
||||
filter->fixup_range(ofs_x, end_x);
|
||||
|
||||
/* rgw::sal::DataProcessor */
|
||||
@ -6111,6 +6230,19 @@ public:
|
||||
<< ", compressor_message=" << cs_info.compressor_message
|
||||
<< ", blocks=" << cs_info.blocks.size() << dendl;
|
||||
}
|
||||
|
||||
// Copy path: ensure AEAD ORIGINAL_SIZE matches copied payload size.
|
||||
// If it doesn't, stale CRYPT_PARTS and CRYPT_PART_NUMS must be dropped.
|
||||
const auto mode = get_str_attribute(attrs, RGW_ATTR_CRYPT_MODE);
|
||||
if (is_aead_mode(mode)) {
|
||||
uint64_t existing = 0;
|
||||
if (!rgw_get_aead_original_size(s, attrs, &existing) ||
|
||||
existing != obj_size) {
|
||||
set_attr(attrs, RGW_ATTR_CRYPT_ORIGINAL_SIZE, std::to_string(obj_size));
|
||||
attrs.erase(RGW_ATTR_CRYPT_PARTS);
|
||||
attrs.erase(RGW_ATTR_CRYPT_PART_NUMS);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
@ -10193,11 +10325,15 @@ int get_decrypt_filter(
|
||||
std::map<std::string, bufferlist>& attrs,
|
||||
bufferlist* manifest_bl,
|
||||
std::map<std::string, std::string>* crypt_http_responses,
|
||||
bool copy_source)
|
||||
bool copy_source,
|
||||
uint32_t part_num,
|
||||
off_t encrypted_total_size,
|
||||
const rgw_crypt_src_identity* src_identity)
|
||||
{
|
||||
std::unique_ptr<BlockCrypt> block_crypt;
|
||||
int res = rgw_s3_prepare_decrypt(s, s->yield, attrs, &block_crypt,
|
||||
crypt_http_responses, copy_source);
|
||||
crypt_http_responses, copy_source, part_num,
|
||||
src_identity);
|
||||
if (res < 0) {
|
||||
return res;
|
||||
}
|
||||
@ -10209,6 +10345,29 @@ int get_decrypt_filter(
|
||||
// correctly decrypt across part boundaries
|
||||
std::vector<size_t> parts_len;
|
||||
|
||||
// Read actual S3 part numbers from attribute (set by CompleteMultipartUpload)
|
||||
std::vector<uint32_t> part_nums;
|
||||
if (auto it = attrs.find(RGW_ATTR_CRYPT_PART_NUMS); it != attrs.end()) {
|
||||
try {
|
||||
auto p = it->second.cbegin();
|
||||
using ceph::decode;
|
||||
decode(part_nums, p);
|
||||
} catch (const buffer::error&) {
|
||||
ldpp_dout(s, 1) << "failed to decode RGW_ATTR_CRYPT_PART_NUMS" << dendl;
|
||||
// Continue with empty part_nums - will fail for multipart, ok for single-part
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback for GET ?partNumber=N (single part read).
|
||||
* When reading an individual part, the CRYPT_PART_NUMS attribute is skipped
|
||||
* (see rgw_rados.cc skip list), so we use the requested part_num to ensure
|
||||
* correct key derivation and IV generation.
|
||||
*/
|
||||
if (part_nums.empty() && part_num > 0) {
|
||||
part_nums.push_back(part_num);
|
||||
}
|
||||
|
||||
// for replicated objects, the original part lengths are preserved in an xattr
|
||||
if (auto i = attrs.find(RGW_ATTR_CRYPT_PARTS); i != attrs.end()) {
|
||||
try {
|
||||
@ -10228,8 +10387,37 @@ int get_decrypt_filter(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* For AEAD ciphers (GCM), we need encrypted_total_size to properly clamp
|
||||
* range requests to the actual on-disk object size. AEAD ciphers expand
|
||||
* data (block_size != encrypted_block_size due to auth tags).
|
||||
*
|
||||
* Derivation priority:
|
||||
* 1. parts_len sum - most accurate, already in encrypted domain
|
||||
* 2. CRYPT_ORIGINAL_SIZE - convert plaintext to encrypted size
|
||||
*
|
||||
* Skip derivation for compressed objects: compression changes the input
|
||||
* to encryption, so plaintext_to_encrypted(ORIGINAL_SIZE) would be wrong.
|
||||
* Compressed objects rely on the decompression filter for size handling.
|
||||
*/
|
||||
if (encrypted_total_size == 0 &&
|
||||
block_crypt->get_block_size() != block_crypt->get_encrypted_block_size()) {
|
||||
if (!parts_len.empty()) {
|
||||
for (size_t part_len : parts_len) {
|
||||
encrypted_total_size += part_len;
|
||||
}
|
||||
} else if (!attrs.count(RGW_ATTR_COMPRESSION)) {
|
||||
uint64_t orig_size = 0;
|
||||
if (rgw_get_aead_original_size(s, attrs, &orig_size)) {
|
||||
encrypted_total_size = aead_plaintext_to_encrypted_size(orig_size);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const bool has_compression = attrs.count(RGW_ATTR_COMPRESSION);
|
||||
*filter = std::make_unique<RGWGetObj_BlockDecrypt>(
|
||||
s, s->cct, cb, std::move(block_crypt),
|
||||
std::move(parts_len), s->yield);
|
||||
std::move(parts_len), std::move(part_nums), encrypted_total_size,
|
||||
has_compression, s->yield);
|
||||
return 0;
|
||||
}
|
||||
|
||||
@ -38,6 +38,8 @@
|
||||
#include "rgw_cksum.h"
|
||||
#include "rgw_common.h"
|
||||
#include "rgw_dmclock.h"
|
||||
|
||||
struct rgw_crypt_src_identity;
|
||||
#include "rgw_sal.h"
|
||||
#include "driver/rados/rgw_user.h"
|
||||
#include "rgw_bucket.h"
|
||||
@ -393,8 +395,9 @@ public:
|
||||
// DataProcessor requires ownership of the entire bufferlist.
|
||||
// RGWGetObj_Filter, however, may reuse the original bufferlist after this call.
|
||||
// To avoid unintended side effects, we create a copy of the relevant portion.
|
||||
// Note: bl_ofs is the offset into this bufferlist, not an object offset.
|
||||
bufferlist copy_bl;
|
||||
bl.begin().copy(bl_len, copy_bl);
|
||||
bl.begin(bl_ofs).copy(bl_len, copy_bl);
|
||||
|
||||
int ret = processor->process(std::move(copy_bl), ofs);
|
||||
if (ret < 0) return ret;
|
||||
@ -458,6 +461,10 @@ protected:
|
||||
// PartsCount response when partNumber is specified
|
||||
std::optional<int> multipart_parts_count;
|
||||
|
||||
// For AEAD encryption ciphers: preserve original encrypted size before plaintext conversion.
|
||||
// Used by decrypt filter for range clamping. For non-AEAD modes, equals s->obj_size.
|
||||
off_t encrypted_obj_size{0};
|
||||
|
||||
int init_common();
|
||||
public:
|
||||
RGWGetObj() {
|
||||
@ -2907,4 +2914,7 @@ int get_decrypt_filter(
|
||||
std::map<std::string, bufferlist>& attrs,
|
||||
bufferlist* manifest_bl,
|
||||
std::map<std::string, std::string>* crypt_http_responses,
|
||||
bool copy_source);
|
||||
bool copy_source,
|
||||
uint32_t part_num = 0,
|
||||
off_t encrypted_total_size = 0,
|
||||
const rgw_crypt_src_identity* src_identity = nullptr);
|
||||
|
||||
207
src/rgw/rgw_range_projection.h
Normal file
207
src/rgw/rgw_range_projection.h
Normal file
@ -0,0 +1,207 @@
|
||||
// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*-
|
||||
// vim: ts=8 sw=2 sts=2 expandtab ft=cpp
|
||||
|
||||
/*
|
||||
* Ceph - scalable distributed file system
|
||||
*
|
||||
* Copyright (C) 2026 Red Hat, Inc.
|
||||
*
|
||||
* This is free software; you can redistribute it and/or
|
||||
* modify it under the terms of the GNU Lesser General Public
|
||||
* License version 2.1, as published by the Free Software
|
||||
* Foundation. See file COPYING.
|
||||
*/
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <vector>
|
||||
#include <sys/types.h>
|
||||
|
||||
#include "rgw_compression_types.h"
|
||||
#include "rgw_crypt.h"
|
||||
|
||||
// Stateless range projection: plaintext byte range -> on-disk byte range
|
||||
struct DiskRange {
|
||||
off_t ofs = 0; // disk start offset
|
||||
off_t end = -1; // disk end offset (inclusive); end < ofs means empty
|
||||
off_t skip = 0; // bytes to skip in first decompressed/decrypted block
|
||||
uint64_t length = 0; // original plaintext request length
|
||||
size_t enc_begin_skip = 0; // plaintext offset within the starting encryption block
|
||||
|
||||
bool empty() const { return end < ofs; }
|
||||
uint64_t disk_bytes() const { return empty() ? 0 : end - ofs + 1; }
|
||||
};
|
||||
|
||||
// DiskRange plus block indices for decompression filter init
|
||||
struct DecompressRange : DiskRange {
|
||||
size_t first_block_idx = 0;
|
||||
size_t last_block_idx = 0;
|
||||
off_t q_ofs = 0; // offset within first decompressed block
|
||||
uint64_t q_len = 0; // plaintext bytes requested
|
||||
};
|
||||
|
||||
// Map plaintext range to compressed on-disk blocks.
|
||||
inline DecompressRange project_compress_range(
|
||||
off_t ofs, off_t end,
|
||||
const RGWCompressionInfo& cs_info,
|
||||
bool partial_content)
|
||||
{
|
||||
DecompressRange r;
|
||||
r.length = end + 1 - ofs;
|
||||
|
||||
size_t first_idx = 0;
|
||||
size_t last_idx = 0;
|
||||
|
||||
if (partial_content) {
|
||||
// user set a range — find the blocks that bracket it
|
||||
first_idx = 0;
|
||||
last_idx = 0;
|
||||
if (cs_info.blocks.size() > 1) {
|
||||
auto cmp_u = [](off_t o, const compression_block& e) {
|
||||
return (uint64_t)o < e.old_ofs;
|
||||
};
|
||||
auto cmp_l = [](const compression_block& e, off_t o) {
|
||||
return e.old_ofs <= (uint64_t)o;
|
||||
};
|
||||
auto fb = std::upper_bound(cs_info.blocks.begin() + 1,
|
||||
cs_info.blocks.end(),
|
||||
ofs, cmp_u);
|
||||
first_idx = (fb - 1) - cs_info.blocks.begin();
|
||||
|
||||
auto lb = std::lower_bound(fb,
|
||||
cs_info.blocks.end(),
|
||||
end, cmp_l);
|
||||
last_idx = (lb - 1) - cs_info.blocks.begin();
|
||||
}
|
||||
} else {
|
||||
// full object — use first and last blocks
|
||||
first_idx = 0;
|
||||
last_idx = cs_info.blocks.size() - 1;
|
||||
}
|
||||
|
||||
const auto& first_block = cs_info.blocks[first_idx];
|
||||
const auto& last_block = cs_info.blocks[last_idx];
|
||||
|
||||
r.first_block_idx = first_idx;
|
||||
r.last_block_idx = last_idx;
|
||||
r.q_ofs = ofs - first_block.old_ofs;
|
||||
r.q_len = end + 1 - ofs;
|
||||
r.skip = r.q_ofs;
|
||||
|
||||
r.ofs = first_block.new_ofs;
|
||||
r.end = last_block.new_ofs + last_block.len - 1;
|
||||
|
||||
return r;
|
||||
}
|
||||
|
||||
// Find which multipart part contains a given plaintext offset.
|
||||
inline void find_part_for_offset(
|
||||
off_t plaintext_ofs,
|
||||
const std::vector<size_t>& parts_len,
|
||||
size_t block_size,
|
||||
size_t enc_block_size,
|
||||
bool clamp_to_last,
|
||||
size_t& part_idx,
|
||||
off_t& offset_in_part,
|
||||
off_t& cumulative_encrypted)
|
||||
{
|
||||
part_idx = 0;
|
||||
offset_in_part = plaintext_ofs;
|
||||
cumulative_encrypted = 0;
|
||||
|
||||
size_t limit = (clamp_to_last && parts_len.size() > 0)
|
||||
? (parts_len.size() - 1)
|
||||
: parts_len.size();
|
||||
|
||||
while (part_idx < limit) {
|
||||
size_t part_plain = crypt_enc_to_plaintext_size(
|
||||
parts_len[part_idx], block_size, enc_block_size);
|
||||
if (offset_in_part < (off_t)part_plain) {
|
||||
break;
|
||||
}
|
||||
offset_in_part -= part_plain;
|
||||
cumulative_encrypted += parts_len[part_idx];
|
||||
part_idx++;
|
||||
}
|
||||
}
|
||||
|
||||
// Map plaintext range to encrypted on-disk bytes.
|
||||
inline DiskRange project_encrypt_range(
|
||||
off_t ofs, off_t end,
|
||||
size_t block_size,
|
||||
size_t enc_block_size,
|
||||
uint64_t encrypted_total_size,
|
||||
const std::vector<size_t>& parts_len)
|
||||
{
|
||||
DiskRange r;
|
||||
r.length = end + 1 - ofs;
|
||||
|
||||
if (parts_len.size() > 0) {
|
||||
// multipart object
|
||||
size_t start_part_idx;
|
||||
off_t start_offset_in_part;
|
||||
off_t start_cumulative;
|
||||
find_part_for_offset(ofs, parts_len, block_size, enc_block_size,
|
||||
false, start_part_idx, start_offset_in_part,
|
||||
start_cumulative);
|
||||
|
||||
size_t end_part_idx;
|
||||
off_t end_offset_in_part;
|
||||
off_t end_cumulative;
|
||||
find_part_for_offset(end, parts_len, block_size, enc_block_size,
|
||||
true, end_part_idx, end_offset_in_part,
|
||||
end_cumulative);
|
||||
|
||||
// block-align end within its part (in plaintext space)
|
||||
size_t part_plaintext_end = crypt_enc_to_plaintext_size(
|
||||
parts_len[end_part_idx], block_size, enc_block_size);
|
||||
off_t rounded_end = std::min(
|
||||
(off_t)((end_offset_in_part & ~(block_size - 1)) + (block_size - 1)),
|
||||
(off_t)(part_plaintext_end - 1));
|
||||
|
||||
// enc_begin_skip is offset within the starting block
|
||||
r.enc_begin_skip = start_offset_in_part & (block_size - 1);
|
||||
r.skip = r.enc_begin_skip;
|
||||
|
||||
// convert end offset: plaintext -> encrypted, align to encrypted block
|
||||
off_t enc_end = crypt_align_enc_block_end(
|
||||
crypt_logical_to_enc_offset(rounded_end, block_size, enc_block_size),
|
||||
block_size, enc_block_size);
|
||||
enc_end = std::min(enc_end, (off_t)(parts_len[end_part_idx] - 1));
|
||||
r.end = end_cumulative + enc_end;
|
||||
|
||||
// convert start offset: align in plaintext, then to encrypted
|
||||
off_t aligned_start = std::max((off_t)0,
|
||||
start_offset_in_part - (off_t)r.enc_begin_skip);
|
||||
r.ofs = start_cumulative +
|
||||
crypt_logical_to_enc_offset(aligned_start, block_size, enc_block_size);
|
||||
|
||||
// clamp start to end (handles invalid ranges)
|
||||
r.ofs = std::min(r.ofs, r.end);
|
||||
} else {
|
||||
// simple (non-multipart) object
|
||||
r.enc_begin_skip = ofs & (block_size - 1);
|
||||
r.skip = r.enc_begin_skip;
|
||||
|
||||
// block-align in plaintext space
|
||||
off_t aligned_start = ofs & ~(block_size - 1);
|
||||
off_t aligned_end = (end & ~(block_size - 1)) + (block_size - 1);
|
||||
|
||||
// convert to encrypted offsets
|
||||
r.ofs = crypt_logical_to_enc_offset(aligned_start, block_size, enc_block_size);
|
||||
r.end = crypt_align_enc_block_end(
|
||||
crypt_logical_to_enc_offset(aligned_end, block_size, enc_block_size),
|
||||
block_size, enc_block_size);
|
||||
|
||||
// clamp to actual encrypted object size
|
||||
if (encrypted_total_size > 0 && r.end >= (off_t)encrypted_total_size) {
|
||||
r.end = encrypted_total_size - 1;
|
||||
}
|
||||
}
|
||||
|
||||
return r;
|
||||
}
|
||||
|
||||
@ -797,7 +797,12 @@ int RGWGetObj_ObjStore_S3::get_decrypt_filter(std::unique_ptr<RGWGetObj_Filter>
|
||||
}
|
||||
|
||||
static constexpr bool copy_source = false;
|
||||
return ::get_decrypt_filter(filter, cb, s, attrs, manifest_bl, &crypt_http_responses, copy_source);
|
||||
// Only use part_num for actual multipart objects (parts_count is set)
|
||||
uint32_t part_num = (multipart_part_num && multipart_parts_count)
|
||||
? static_cast<uint32_t>(*multipart_part_num)
|
||||
: 0;
|
||||
return ::get_decrypt_filter(filter, cb, s, attrs, manifest_bl, &crypt_http_responses, copy_source,
|
||||
part_num, encrypted_obj_size);
|
||||
}
|
||||
|
||||
int RGWGetObj_ObjStore_S3::verify_requester(const rgw::auth::StrategyRegistry& auth_registry, optional_yield y)
|
||||
@ -3085,7 +3090,10 @@ int RGWPutObj_ObjStore_S3::get_decrypt_filter(
|
||||
bufferlist* manifest_bl)
|
||||
{
|
||||
static constexpr bool copy_source = true;
|
||||
return ::get_decrypt_filter(filter, cb, s, attrs, manifest_bl, nullptr, copy_source);
|
||||
rgw_crypt_src_identity src_identity{copy_source_bucket_info.bucket.bucket_id, copy_source_bucket_name, copy_source_object_name};
|
||||
// part_num=0 for copy source (full object read)
|
||||
return ::get_decrypt_filter(filter, cb, s, attrs, manifest_bl, nullptr, copy_source,
|
||||
0, 0, &src_identity);
|
||||
}
|
||||
|
||||
int RGWPutObj_ObjStore_S3::get_encrypt_filter(
|
||||
@ -3105,8 +3113,10 @@ int RGWPutObj_ObjStore_S3::get_encrypt_filter(
|
||||
/* We are adding to existing object.
|
||||
* We use crypto mode that configured as if we were decrypting. */
|
||||
static constexpr bool copy_source = false;
|
||||
// Pass part_number for AEAD IV derivation - ensures unique IVs across parts
|
||||
res = rgw_s3_prepare_decrypt(s, s->yield, obj->get_attrs(),
|
||||
&block_crypt, &crypt_http_responses, copy_source);
|
||||
&block_crypt, &crypt_http_responses, copy_source,
|
||||
multipart_part_num);
|
||||
if (res == 0 && block_crypt != nullptr)
|
||||
filter->reset(new RGWPutObj_BlockEncrypt(s, s->cct, cb, std::move(block_crypt), s->yield));
|
||||
}
|
||||
@ -3115,8 +3125,9 @@ int RGWPutObj_ObjStore_S3::get_encrypt_filter(
|
||||
else
|
||||
{
|
||||
std::unique_ptr<BlockCrypt> block_crypt;
|
||||
// Pass part_number for AEAD IV derivation - ensures unique IVs across parts
|
||||
res = rgw_s3_prepare_encrypt(s, s->yield, attrs, &block_crypt,
|
||||
crypt_http_responses);
|
||||
crypt_http_responses, multipart_part_num);
|
||||
if (res == 0 && block_crypt != nullptr) {
|
||||
filter->reset(new RGWPutObj_BlockEncrypt(s, s->cct, cb, std::move(block_crypt), s->yield));
|
||||
}
|
||||
|
||||
@ -141,6 +141,19 @@ target_link_libraries(bench_rgw_ratelimit ${rgw_libs})
|
||||
add_executable(bench_rgw_ratelimit_gc bench_rgw_ratelimit_gc.cc )
|
||||
target_link_libraries(bench_rgw_ratelimit_gc ${rgw_libs})
|
||||
|
||||
# Crypto benchmark through the BlockCrypt interface (full RGW encrypt path).
|
||||
# Uses the plugin system to load the configured accelerator (ISA-L or OpenSSL).
|
||||
# Run with --accel crypto_isal or --accel crypto_openssl to compare backends.
|
||||
add_executable(bench_rgw_crypto_accel bench_rgw_crypto_accel.cc)
|
||||
target_link_libraries(bench_rgw_crypto_accel
|
||||
${rgw_libs}
|
||||
global
|
||||
${CURL_LIBRARIES}
|
||||
${EXPAT_LIBRARIES}
|
||||
${CMAKE_DL_LIBS}
|
||||
${CRYPTO_LIBS})
|
||||
add_dependencies(bench_rgw_crypto_accel crypto_plugins)
|
||||
|
||||
add_executable(unittest_rgw_ratelimit test_rgw_ratelimit.cc $<TARGET_OBJECTS:unit-main>)
|
||||
target_link_libraries(unittest_rgw_ratelimit ${rgw_libs})
|
||||
add_ceph_unittest(unittest_rgw_ratelimit)
|
||||
@ -217,6 +230,13 @@ target_link_libraries(unittest_rgw_crypto
|
||||
${CRYPTO_LIBS}
|
||||
)
|
||||
|
||||
add_executable(unittest_rgw_range_projection test_rgw_range_projection.cc)
|
||||
add_ceph_unittest(unittest_rgw_range_projection)
|
||||
target_link_libraries(unittest_rgw_range_projection
|
||||
${rgw_libs}
|
||||
${UNITTEST_LIBS}
|
||||
)
|
||||
|
||||
if(WITH_RADOSGW_RADOS)
|
||||
set(test_rgw_reshard_srcs test_rgw_reshard.cc)
|
||||
add_executable(unittest_rgw_reshard
|
||||
|
||||
476
src/test/rgw/bench_rgw_crypto_accel.cc
Normal file
476
src/test/rgw/bench_rgw_crypto_accel.cc
Normal file
@ -0,0 +1,476 @@
|
||||
// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*-
|
||||
// vim: ts=8 sw=2 sts=2 expandtab
|
||||
|
||||
/*
|
||||
* Ceph - scalable distributed file system
|
||||
*
|
||||
* Copyright (C) 2026 Ceph contributors
|
||||
*
|
||||
* This is free software; you can redistribute it and/or
|
||||
* modify it under the terms of the GNU Lesser General Public
|
||||
* License version 2.1, as published by the Free Software
|
||||
* Foundation. See file COPYING.
|
||||
*
|
||||
*/
|
||||
|
||||
#include "global/global_init.h"
|
||||
#include "common/ceph_argparse.h"
|
||||
#include "common/PluginRegistry.h"
|
||||
#include "rgw_common.h"
|
||||
#include "rgw_crypt.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <chrono>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
#include <functional>
|
||||
#include <iomanip>
|
||||
#include <iostream>
|
||||
#include <random>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
#include <vector>
|
||||
|
||||
#ifdef __x86_64__
|
||||
#include <immintrin.h>
|
||||
#endif
|
||||
|
||||
std::unique_ptr<BlockCrypt> AES_256_CBC_create(
|
||||
const DoutPrefixProvider* dpp, CephContext* cct,
|
||||
const uint8_t* key, size_t len);
|
||||
|
||||
#define dout_subsys ceph_subsys_rgw
|
||||
|
||||
using std::cout;
|
||||
using std::cerr;
|
||||
using std::string;
|
||||
using std::vector;
|
||||
|
||||
static bool is_supported_accel(const string& accel) {
|
||||
return accel == "crypto_isal" || accel == "crypto_openssl" ||
|
||||
accel == "crypto_qat";
|
||||
}
|
||||
|
||||
static string human_bytes(double bytes) {
|
||||
const char* units[] = {"B", "KB", "MB", "GB", "TB"};
|
||||
int u = 0;
|
||||
while (bytes >= 1024.0 && u < 4) { bytes /= 1024.0; ++u; }
|
||||
char buf[64];
|
||||
snprintf(buf, sizeof(buf), "%.2f %s", bytes, units[u]);
|
||||
return buf;
|
||||
}
|
||||
|
||||
static void fill_random(uint8_t* buf, size_t len) {
|
||||
static std::mt19937_64 rng(42);
|
||||
size_t i = 0;
|
||||
for (; i + sizeof(uint64_t) <= len; i += sizeof(uint64_t)) {
|
||||
uint64_t v = rng();
|
||||
memcpy(buf + i, &v, sizeof(v));
|
||||
}
|
||||
for (; i < len; ++i) buf[i] = rng() & 0xff;
|
||||
}
|
||||
|
||||
/* Verify that a crypto instance can perform a basic encrypt operation. */
|
||||
static bool probe_encrypt(std::unique_ptr<BlockCrypt>& crypt,
|
||||
const string& err_msg) {
|
||||
if (!crypt) { cerr << err_msg << "\n"; return false; }
|
||||
bufferlist in, out;
|
||||
buffer::ptr buf(4096);
|
||||
fill_random(reinterpret_cast<uint8_t*>(buf.c_str()), 4096);
|
||||
in.append(buf);
|
||||
if (!crypt->encrypt(in, 0, 4096, out, 0, null_yield)) {
|
||||
cerr << err_msg << "\n";
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Throughput display unit, set once from --unit CLI option. */
|
||||
static const char* tp_unit = "MB/s";
|
||||
static double tp_divisor = 1024.0 * 1024.0;
|
||||
|
||||
struct BenchResult {
|
||||
string label;
|
||||
size_t data_size;
|
||||
int iterations;
|
||||
double elapsed_sec;
|
||||
|
||||
double throughput() const {
|
||||
if (elapsed_sec <= 0) return 0;
|
||||
return static_cast<double>(data_size) * iterations / elapsed_sec / tp_divisor;
|
||||
}
|
||||
double ops_per_sec() const {
|
||||
if (elapsed_sec <= 0) return 0;
|
||||
return iterations / elapsed_sec;
|
||||
}
|
||||
};
|
||||
|
||||
using CryptFactory = std::function<std::unique_ptr<BlockCrypt>()>;
|
||||
using BenchFn = std::function<bool(BlockCrypt&)>;
|
||||
|
||||
static int probe_iters(double target_time, double min_sec,
|
||||
int warmup_iters, BlockCrypt& crypt, BenchFn& fn) {
|
||||
for (int i = 0; i < warmup_iters; ++i) {
|
||||
if (!fn(crypt)) return 0;
|
||||
}
|
||||
int probe_n = 10;
|
||||
double probe_sec = 0;
|
||||
do {
|
||||
auto t0 = std::chrono::high_resolution_clock::now();
|
||||
for (int i = 0; i < probe_n; ++i) {
|
||||
if (!fn(crypt)) return 0;
|
||||
}
|
||||
auto t1 = std::chrono::high_resolution_clock::now();
|
||||
probe_sec = std::chrono::duration<double>(t1 - t0).count();
|
||||
if (probe_sec < min_sec) probe_n *= 2;
|
||||
} while (probe_sec < min_sec);
|
||||
return std::max(static_cast<int>(target_time / (probe_sec / probe_n)), 100);
|
||||
}
|
||||
|
||||
static void spin_barrier(std::atomic<int>& barrier, int target) {
|
||||
barrier.fetch_add(1, std::memory_order_release);
|
||||
while (barrier.load(std::memory_order_acquire) < target) {
|
||||
#ifdef __x86_64__
|
||||
_mm_pause();
|
||||
#else
|
||||
std::this_thread::yield();
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
static BenchResult run_bench(const string& label, size_t data_size,
|
||||
int warmup_iters, double target_time,
|
||||
int fixed_iters, int nthreads,
|
||||
CryptFactory factory, BenchFn bench_fn) {
|
||||
if (nthreads <= 1) {
|
||||
auto crypt = factory();
|
||||
if (!crypt) {
|
||||
cerr << "FAILED: " << label << "\n";
|
||||
return {label, data_size, 0, 0.0};
|
||||
}
|
||||
int iters = fixed_iters;
|
||||
if (iters <= 0)
|
||||
iters = probe_iters(target_time, 0.1, warmup_iters, *crypt, bench_fn);
|
||||
if (iters == 0) return {label, data_size, 0, 0.0};
|
||||
auto start = std::chrono::high_resolution_clock::now();
|
||||
for (int i = 0; i < iters; ++i) {
|
||||
if (!bench_fn(*crypt)) {
|
||||
cerr << "FAILED at iteration " << i << ": " << label << "\n";
|
||||
return {label, data_size, i, 0.0};
|
||||
}
|
||||
}
|
||||
auto end = std::chrono::high_resolution_clock::now();
|
||||
return {label, data_size, iters,
|
||||
std::chrono::duration<double>(end - start).count()};
|
||||
}
|
||||
|
||||
int iters = fixed_iters;
|
||||
if (iters <= 0) {
|
||||
const double probe_dur = 0.25;
|
||||
std::vector<int> per_thread_count(nthreads);
|
||||
std::atomic<int> barrier{0};
|
||||
std::vector<std::thread> threads;
|
||||
std::atomic<bool> probe_failed{false};
|
||||
for (int t = 0; t < nthreads; ++t) {
|
||||
threads.emplace_back([&, t]() {
|
||||
auto crypt = factory();
|
||||
if (!crypt) { probe_failed.store(true); }
|
||||
if (!probe_failed.load()) {
|
||||
for (int i = 0; i < warmup_iters; ++i) bench_fn(*crypt);
|
||||
}
|
||||
spin_barrier(barrier, nthreads);
|
||||
if (probe_failed.load()) return;
|
||||
int count = 0;
|
||||
auto t0 = std::chrono::high_resolution_clock::now();
|
||||
double elapsed = 0;
|
||||
do {
|
||||
bench_fn(*crypt);
|
||||
++count;
|
||||
elapsed = std::chrono::duration<double>(
|
||||
std::chrono::high_resolution_clock::now() - t0).count();
|
||||
} while (elapsed < probe_dur);
|
||||
per_thread_count[t] = count;
|
||||
});
|
||||
}
|
||||
for (auto& t : threads) t.join();
|
||||
int slowest = *std::min_element(per_thread_count.begin(),
|
||||
per_thread_count.end());
|
||||
iters = std::max(static_cast<int>(slowest * (target_time / probe_dur)),
|
||||
100);
|
||||
}
|
||||
|
||||
std::vector<double> thread_elapsed(nthreads);
|
||||
std::atomic<int> barrier{0};
|
||||
std::atomic<bool> failed{false};
|
||||
|
||||
auto thread_body = [&](int tid) {
|
||||
auto crypt = factory();
|
||||
if (!crypt) { failed.store(true); }
|
||||
if (!failed.load()) {
|
||||
for (int i = 0; i < warmup_iters; ++i) {
|
||||
if (!bench_fn(*crypt)) { failed.store(true); break; }
|
||||
}
|
||||
}
|
||||
// always reach the barrier so other threads don't spin forever
|
||||
spin_barrier(barrier, nthreads);
|
||||
if (failed.load()) return;
|
||||
auto start = std::chrono::high_resolution_clock::now();
|
||||
for (int i = 0; i < iters; ++i) {
|
||||
if (!bench_fn(*crypt)) { failed.store(true); return; }
|
||||
}
|
||||
auto end = std::chrono::high_resolution_clock::now();
|
||||
thread_elapsed[tid] = std::chrono::duration<double>(end - start).count();
|
||||
};
|
||||
|
||||
std::vector<std::thread> threads;
|
||||
for (int t = 0; t < nthreads; ++t)
|
||||
threads.emplace_back(thread_body, t);
|
||||
for (auto& t : threads) t.join();
|
||||
|
||||
if (failed.load()) {
|
||||
cerr << "FAILED: " << label << "\n";
|
||||
return {label, data_size, 0, 0.0};
|
||||
}
|
||||
|
||||
return {label, data_size, iters * nthreads,
|
||||
*std::max_element(thread_elapsed.begin(), thread_elapsed.end())};
|
||||
}
|
||||
|
||||
static void print_header() {
|
||||
cout << std::left
|
||||
<< std::setw(36) << "Benchmark"
|
||||
<< std::setw(12) << "DataSize"
|
||||
<< std::setw(10) << "Iters"
|
||||
<< std::setw(12) << "Time(s)"
|
||||
<< std::setw(14) << "Throughput"
|
||||
<< std::setw(12) << "Ops/s"
|
||||
<< "\n" << string(96, '-') << "\n";
|
||||
}
|
||||
|
||||
static void print_result(const BenchResult& r) {
|
||||
cout << std::left
|
||||
<< std::setw(36) << r.label
|
||||
<< std::setw(12) << human_bytes(r.data_size)
|
||||
<< std::setw(10) << r.iterations
|
||||
<< std::setw(12) << std::fixed << std::setprecision(4) << r.elapsed_sec
|
||||
<< std::setw(14) << (std::to_string(static_cast<int>(r.throughput())) + " " + tp_unit)
|
||||
<< std::setw(12) << std::fixed << std::setprecision(1) << r.ops_per_sec()
|
||||
<< "\n";
|
||||
}
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
auto args = argv_to_vec(argc, argv);
|
||||
auto cct = global_init(nullptr, args, CEPH_ENTITY_TYPE_CLIENT,
|
||||
CODE_ENVIRONMENT_UTILITY,
|
||||
CINIT_FLAG_NO_DEFAULT_CONFIG_FILE);
|
||||
|
||||
int warmup = 10;
|
||||
int iterations = 0;
|
||||
size_t fixed_size = 0;
|
||||
double target_time = 2.0;
|
||||
int num_threads = 1;
|
||||
string accel;
|
||||
// parse args (not argv) — global_init() already stripped Ceph options
|
||||
for (size_t i = 0; i < args.size(); ++i) {
|
||||
string arg(args[i]);
|
||||
if ((arg == "--size" || arg == "-s") && i + 1 < args.size()) {
|
||||
fixed_size = std::stoull(args[++i]);
|
||||
} else if (arg == "--iterations" && i + 1 < args.size()) {
|
||||
iterations = std::stoi(args[++i]);
|
||||
} else if ((arg == "--warmup" || arg == "-w") && i + 1 < args.size()) {
|
||||
warmup = std::stoi(args[++i]);
|
||||
} else if ((arg == "--time" || arg == "-t") && i + 1 < args.size()) {
|
||||
target_time = std::stod(args[++i]);
|
||||
} else if (arg == "--accel" && i + 1 < args.size()) {
|
||||
accel = args[++i];
|
||||
} else if ((arg == "--threads" || arg == "-j") && i + 1 < args.size()) {
|
||||
num_threads = std::stoi(args[++i]);
|
||||
} else if (arg == "--unit" && i + 1 < args.size()) {
|
||||
string u = args[++i];
|
||||
if (u == "KB") { tp_unit = "KB/s"; tp_divisor = 1024.0; }
|
||||
else if (u == "MB") { tp_unit = "MB/s"; tp_divisor = 1024.0 * 1024.0; }
|
||||
else if (u == "GB") { tp_unit = "GB/s"; tp_divisor = 1024.0 * 1024.0 * 1024.0; }
|
||||
else if (u == "TB") { tp_unit = "TB/s"; tp_divisor = 1024.0 * 1024.0 * 1024.0 * 1024.0; }
|
||||
else {
|
||||
cerr << "Invalid --unit '" << u << "'. Supported: KB, MB, GB, TB\n";
|
||||
return 2;
|
||||
}
|
||||
} else if (arg == "--help" || arg == "-h") {
|
||||
cout << "Usage: " << argv[0] << " [options]\n"
|
||||
<< " --accel <name> Crypto plugin (crypto_isal, crypto_openssl, crypto_qat)\n"
|
||||
<< " --size, -s <bytes> Fixed data size (default: run multiple sizes)\n"
|
||||
<< " --iterations <N> Fixed iteration count (default: auto-scale)\n"
|
||||
<< " --warmup, -w <N> Warmup iterations (default: 10)\n"
|
||||
<< " --time, -t <sec> Target time for auto-scaling (default: 2.0)\n"
|
||||
<< " --threads, -j <N> Number of threads (default: 1)\n"
|
||||
<< " --unit <UNIT> Throughput unit: KB, MB, GB, TB (default: MB)\n"
|
||||
<< " --help, -h Show this help\n";
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
if (!accel.empty()) {
|
||||
if (!is_supported_accel(accel)) {
|
||||
cerr << "Invalid --accel '" << accel << "'\n";
|
||||
return 2;
|
||||
}
|
||||
int r = g_ceph_context->_conf.set_val(
|
||||
"plugin_crypto_accelerator", accel);
|
||||
if (r < 0) {
|
||||
cerr << "Failed to set plugin_crypto_accelerator\n";
|
||||
return 2;
|
||||
}
|
||||
}
|
||||
|
||||
g_ceph_context->_conf.apply_changes(nullptr);
|
||||
common_init_finish(g_ceph_context);
|
||||
|
||||
string active_accel = g_ceph_context->_conf->plugin_crypto_accelerator;
|
||||
if (!is_supported_accel(active_accel)) {
|
||||
cerr << "Unsupported plugin_crypto_accelerator='" << active_accel << "'\n";
|
||||
return 2;
|
||||
}
|
||||
|
||||
{
|
||||
auto* reg = g_ceph_context->get_plugin_registry();
|
||||
if (!reg->get_with_load("crypto", active_accel)) {
|
||||
cerr << "Failed to load crypto plugin '" << active_accel << "'\n";
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
static constexpr size_t AES_256_KEYSIZE = 32;
|
||||
uint8_t key[AES_256_KEYSIZE];
|
||||
fill_random(key, sizeof(key));
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
|
||||
{
|
||||
auto cbc = AES_256_CBC_create(&no_dpp, g_ceph_context, key, AES_256_KEYSIZE);
|
||||
if (!probe_encrypt(cbc, "CBC probe failed for '" + active_accel +
|
||||
"'. Plugin loaded but hardware may not be available."))
|
||||
return 1;
|
||||
}
|
||||
|
||||
const bool have_gcm = (active_accel != "crypto_qat");
|
||||
if (have_gcm) {
|
||||
auto gcm = AES_256_GCM_create(&no_dpp, g_ceph_context, key, AES_256_KEYSIZE);
|
||||
if (!probe_encrypt(gcm, "GCM probe failed for '" + active_accel + "'."))
|
||||
return 1;
|
||||
}
|
||||
|
||||
vector<size_t> sizes;
|
||||
if (fixed_size > 0) {
|
||||
sizes.push_back(fixed_size);
|
||||
} else {
|
||||
sizes = {4096, 65536, 262144, 1048576, 4194304, 16777216};
|
||||
}
|
||||
|
||||
cout << "\n=== RGW BlockCrypt Benchmark ===\n\n"
|
||||
<< "Accelerator: " << active_accel << "\n"
|
||||
<< "Threads: " << num_threads << "\n"
|
||||
<< "Modes: AES-256-CBC";
|
||||
if (have_gcm) cout << ", AES-256-GCM";
|
||||
else cout << " (GCM skipped)";
|
||||
cout << "\nWarmup: " << warmup << " iterations\n";
|
||||
if (iterations > 0)
|
||||
cout << "Iters: " << iterations << " (fixed, per thread)\n";
|
||||
else
|
||||
cout << "Target: ~" << std::fixed << std::setprecision(1)
|
||||
<< target_time << "s per benchmark\n";
|
||||
cout << "\n";
|
||||
|
||||
vector<BenchResult> results;
|
||||
string suffix = (num_threads > 1)
|
||||
? " [" + active_accel + " x" + std::to_string(num_threads) + "]"
|
||||
: " [" + active_accel + "]";
|
||||
auto emit = [&](BenchResult r) {
|
||||
print_result(r);
|
||||
cout << std::flush;
|
||||
results.push_back(std::move(r));
|
||||
};
|
||||
auto dec_fn = [](string data, size_t len) -> BenchFn {
|
||||
return [data, len](BlockCrypt& c) -> bool {
|
||||
bufferlist in, out;
|
||||
in.append(data);
|
||||
return c.decrypt(in, 0, len, out, 0, null_yield);
|
||||
};
|
||||
};
|
||||
print_header();
|
||||
|
||||
for (size_t sz : sizes) {
|
||||
buffer::ptr plain_buf(sz);
|
||||
fill_random(reinterpret_cast<uint8_t*>(plain_buf.c_str()), sz);
|
||||
bufferlist input;
|
||||
input.append(plain_buf);
|
||||
string input_str = input.to_str();
|
||||
|
||||
auto cbc_factory = [&]() {
|
||||
return AES_256_CBC_create(&no_dpp, g_ceph_context, key, AES_256_KEYSIZE);
|
||||
};
|
||||
auto enc_fn = [input_str, sz](BlockCrypt& c) -> bool {
|
||||
bufferlist in, out;
|
||||
in.append(input_str);
|
||||
return c.encrypt(in, 0, sz, out, 0, null_yield);
|
||||
};
|
||||
|
||||
auto bench = [&](const string& lbl, size_t sz, CryptFactory f, BenchFn fn) {
|
||||
emit(run_bench(lbl, sz, warmup, target_time, iterations, num_threads,
|
||||
std::move(f), std::move(fn)));
|
||||
};
|
||||
|
||||
bench("CBC-Encrypt" + suffix, sz, cbc_factory, enc_fn);
|
||||
|
||||
{
|
||||
auto pre = cbc_factory();
|
||||
bufferlist ct;
|
||||
pre->encrypt(input, 0, sz, ct, 0, null_yield);
|
||||
bench("CBC-Decrypt" + suffix, sz, cbc_factory, dec_fn(ct.to_str(), sz));
|
||||
}
|
||||
|
||||
if (have_gcm) {
|
||||
auto gcm_factory = [&]() {
|
||||
return AES_256_GCM_create(&no_dpp, g_ceph_context, key, AES_256_KEYSIZE);
|
||||
};
|
||||
|
||||
bench("GCM-Encrypt" + suffix, sz, gcm_factory, enc_fn);
|
||||
|
||||
{
|
||||
auto pre = gcm_factory();
|
||||
bufferlist ct;
|
||||
pre->encrypt(input, 0, sz, ct, 0, null_yield);
|
||||
string salt = AES_256_GCM_get_salt(pre.get());
|
||||
size_t enc_sz = ct.length();
|
||||
auto dec_factory = [&, salt]() {
|
||||
return AES_256_GCM_create(&no_dpp, g_ceph_context, key, AES_256_KEYSIZE,
|
||||
reinterpret_cast<const uint8_t*>(salt.data()), salt.size());
|
||||
};
|
||||
bench("GCM-Decrypt" + suffix, sz, dec_factory, dec_fn(ct.to_str(), enc_sz));
|
||||
}
|
||||
}
|
||||
|
||||
cout << "\n";
|
||||
}
|
||||
|
||||
cout << "\n=== Summary (Throughput in " << tp_unit << ") ===\n\n"
|
||||
<< std::left << std::setw(14) << "DataSize"
|
||||
<< std::setw(16) << "CBC-Encrypt"
|
||||
<< std::setw(16) << "CBC-Decrypt";
|
||||
if (have_gcm)
|
||||
cout << std::setw(16) << "GCM-Encrypt"
|
||||
<< std::setw(16) << "GCM-Decrypt";
|
||||
cout << "\n" << string(46 + (have_gcm ? 32 : 0), '-') << "\n";
|
||||
|
||||
for (size_t sz : sizes) {
|
||||
cout << std::setw(14) << human_bytes(sz);
|
||||
for (const auto& r : results) {
|
||||
if (r.data_size == sz)
|
||||
cout << std::setw(16) << std::fixed << std::setprecision(0)
|
||||
<< r.throughput();
|
||||
}
|
||||
cout << "\n";
|
||||
}
|
||||
|
||||
cout << "\n";
|
||||
return 0;
|
||||
}
|
||||
@ -26,6 +26,13 @@ using namespace std;
|
||||
|
||||
|
||||
std::unique_ptr<BlockCrypt> AES_256_CBC_create(const DoutPrefixProvider *dpp, CephContext* cct, const uint8_t* key, size_t len);
|
||||
bool AES_256_GCM_derive_object_key(BlockCrypt* block_crypt,
|
||||
const uint8_t* user_key,
|
||||
size_t key_len,
|
||||
const std::string& bucket_id,
|
||||
const std::string& object,
|
||||
uint32_t part_number,
|
||||
const std::string& domain = "SSE-C-GCM");
|
||||
|
||||
class ut_get_sink : public RGWGetObj_Filter {
|
||||
std::stringstream sink;
|
||||
@ -94,6 +101,16 @@ public:
|
||||
}
|
||||
};
|
||||
|
||||
// Mock that simulates AEAD size expansion (encrypted_block_size > block_size)
|
||||
class BlockCryptNoneAEAD: public BlockCryptNone {
|
||||
public:
|
||||
BlockCryptNoneAEAD() : BlockCryptNone(AEAD_CHUNK_SIZE) {}
|
||||
size_t get_encrypted_block_size() override
|
||||
{
|
||||
return AEAD_ENCRYPTED_CHUNK_SIZE; // 4096 + 16 = 4112
|
||||
}
|
||||
};
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_CBC_identity)
|
||||
{
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
@ -813,6 +830,658 @@ TEST(TestRGWCrypto, verify_Encrypt_Decrypt)
|
||||
}
|
||||
|
||||
|
||||
// AEAD Tests
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_identity)
|
||||
{
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
// Create some input for encryption
|
||||
const off_t test_range = 1024*1024;
|
||||
buffer::ptr buf(test_range);
|
||||
char* p = buf.c_str();
|
||||
for(size_t i = 0; i < buf.length(); i++)
|
||||
p[i] = i + i*i + (i >> 2);
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
for (unsigned int step : {1, 2, 3, 5, 7, 11, 13, 17})
|
||||
{
|
||||
// Make some random key
|
||||
uint8_t key[32];
|
||||
for(size_t i=0;i<sizeof(key);i++)
|
||||
key[i]=i*step;
|
||||
|
||||
auto aes(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes.get(), nullptr);
|
||||
|
||||
size_t block_size = aes->get_block_size();
|
||||
ASSERT_EQ(block_size, 4096u);
|
||||
|
||||
for (size_t r = 97; r < 123 ; r++)
|
||||
{
|
||||
off_t begin = (r*r*r*r*r % test_range);
|
||||
begin = begin - begin % block_size;
|
||||
off_t end = begin + r*r*r*r*r*r*r % (test_range - begin);
|
||||
if (r % 3)
|
||||
end = end - end % block_size;
|
||||
off_t offset = r*r*r*r*r*r*r*r % (1000*1000*1000);
|
||||
offset = offset - offset % block_size;
|
||||
|
||||
ASSERT_EQ(begin % block_size, 0u);
|
||||
ASSERT_LE(end, test_range);
|
||||
ASSERT_EQ(offset % block_size, 0u);
|
||||
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes->encrypt(input, begin, end - begin, encrypted, offset, null_yield));
|
||||
|
||||
size_t expected_encrypted_size = aead_plaintext_to_encrypted_size(end - begin);
|
||||
ASSERT_EQ(encrypted.length(), expected_encrypted_size);
|
||||
|
||||
bufferlist decrypted;
|
||||
ASSERT_TRUE(aes->decrypt(encrypted, 0, encrypted.length(), decrypted, offset, null_yield));
|
||||
|
||||
ASSERT_EQ(decrypted.length(), end - begin);
|
||||
ASSERT_EQ(std::string_view(input.c_str() + begin, end - begin),
|
||||
std::string_view(decrypted.c_str(), end - begin) );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_chunk_sizes)
|
||||
{
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
uint8_t key[32];
|
||||
for(size_t i=0;i<sizeof(key);i++)
|
||||
key[i]=i*3;
|
||||
|
||||
auto aes(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes.get(), nullptr);
|
||||
|
||||
// Test various sizes to verify chunk + tag handling
|
||||
for (size_t size : {0, 1, 16, 4095, 4096, 4097, 8192, 10000})
|
||||
{
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for(size_t i = 0; i < size; i++)
|
||||
p[i] = i & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes->encrypt(input, 0, size, encrypted, 0, null_yield));
|
||||
|
||||
size_t expected_size = aead_plaintext_to_encrypted_size(size);
|
||||
ASSERT_EQ(encrypted.length(), expected_size);
|
||||
|
||||
bufferlist decrypted;
|
||||
ASSERT_TRUE(aes->decrypt(encrypted, 0, encrypted.length(), decrypted, 0, null_yield));
|
||||
ASSERT_EQ(decrypted.length(), size);
|
||||
|
||||
if (size > 0) {
|
||||
ASSERT_EQ(std::string_view(input.c_str(), size),
|
||||
std::string_view(decrypted.c_str(), size));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
TEST(TestRGWCrypto, verify_AEAD_size_conversion)
|
||||
{
|
||||
// Test is_aead_mode() - AEAD modes have ciphertext expansion
|
||||
ASSERT_TRUE(is_aead_mode("SSE-C-AES256-GCM"));
|
||||
ASSERT_TRUE(is_aead_mode("SSE-KMS-GCM"));
|
||||
ASSERT_TRUE(is_aead_mode("RGW-AUTO-GCM"));
|
||||
ASSERT_TRUE(is_aead_mode("AES256-GCM"));
|
||||
ASSERT_FALSE(is_aead_mode("SSE-C-AES256"));
|
||||
ASSERT_FALSE(is_aead_mode("SSE-KMS"));
|
||||
ASSERT_FALSE(is_aead_mode(""));
|
||||
ASSERT_FALSE(is_aead_mode("GCM"));
|
||||
|
||||
// Test is_cbc_mode() - exact match for known CBC modes
|
||||
ASSERT_TRUE(is_cbc_mode("SSE-C-AES256"));
|
||||
ASSERT_TRUE(is_cbc_mode("SSE-KMS"));
|
||||
ASSERT_TRUE(is_cbc_mode("RGW-AUTO"));
|
||||
ASSERT_TRUE(is_cbc_mode("AES256"));
|
||||
ASSERT_FALSE(is_cbc_mode("SSE-C-AES256-GCM"));
|
||||
ASSERT_FALSE(is_cbc_mode("SSE-KMS-GCM"));
|
||||
ASSERT_FALSE(is_cbc_mode(""));
|
||||
ASSERT_FALSE(is_cbc_mode("invalid"));
|
||||
|
||||
// Test aead_plaintext_to_encrypted_size()
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(0), 0UL); // Zero bytes
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(1), 17UL); // 1 + 16 tag
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(100), 116UL); // 100 + 16 tag
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(4095), 4111UL); // 4095 + 16 tag
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(4096), 4112UL); // Full chunk
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(4097), 4129UL); // 4097 + 32 tags (2 chunks)
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_size(8192), 8224UL); // Two full chunks
|
||||
|
||||
// Test aead_encrypted_to_plaintext_size()
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(0), 0UL); // Zero bytes
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(16), 0UL); // Tag only -> 0 plaintext
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(17), 1UL); // 1 byte + tag
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(100), 84UL); // 100 - 16 tag
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(4112), 4096UL); // Full chunk
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(4212), 4180UL); // 4096 + 84 (one chunk + partial)
|
||||
ASSERT_EQ(aead_encrypted_to_plaintext_size(8224), 8192UL); // Two full chunks
|
||||
|
||||
// Test roundtrip: plaintext -> encrypted -> plaintext
|
||||
for (uint64_t plain : {0UL, 1UL, 16UL, 100UL, 4095UL, 4096UL, 4097UL,
|
||||
8192UL, 10000UL, 1000000UL}) {
|
||||
uint64_t enc = aead_plaintext_to_encrypted_size(plain);
|
||||
uint64_t recovered = aead_encrypted_to_plaintext_size(enc);
|
||||
ASSERT_EQ(plain, recovered)
|
||||
<< "Roundtrip failed for plaintext=" << plain
|
||||
<< " encrypted=" << enc << " recovered=" << recovered;
|
||||
}
|
||||
|
||||
// Test aead_plaintext_to_encrypted_offset()
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_offset(0), 0UL); // Start of file
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_offset(100), 100UL); // Within first chunk
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_offset(4095), 4095UL); // End of first chunk
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_offset(4096), 4112UL); // Start of second chunk
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_offset(4196), 4212UL); // 100 bytes into second chunk
|
||||
ASSERT_EQ(aead_plaintext_to_encrypted_offset(8192), 8224UL); // Start of third chunk
|
||||
|
||||
// Test has_size_expansion() via RGWGetObj_BlockDecrypt
|
||||
// GCM expands data (adds 16-byte auth tag per chunk), CBC does not
|
||||
{
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
ut_get_sink get_sink;
|
||||
|
||||
auto cbc = std::make_unique<BlockCryptNone>(4096);
|
||||
RGWGetObj_BlockDecrypt decrypt_cbc(&no_dpp, g_ceph_context, &get_sink,
|
||||
std::move(cbc), {}, null_yield);
|
||||
ASSERT_FALSE(decrypt_cbc.has_size_expansion());
|
||||
|
||||
auto gcm = std::make_unique<BlockCryptNoneAEAD>();
|
||||
RGWGetObj_BlockDecrypt decrypt_gcm(&no_dpp, g_ceph_context, &get_sink,
|
||||
std::move(gcm), {}, null_yield);
|
||||
ASSERT_TRUE(decrypt_gcm.has_size_expansion());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_tag_verification)
|
||||
{
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
uint8_t key[32];
|
||||
for(size_t i=0;i<sizeof(key);i++)
|
||||
key[i]=i*7;
|
||||
|
||||
auto aes(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes.get(), nullptr);
|
||||
|
||||
const size_t size = 8192; // 2 chunks
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for(size_t i = 0; i < size; i++)
|
||||
p[i] = i & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes->encrypt(input, 0, size, encrypted, 0, null_yield));
|
||||
|
||||
// Corrupt the ciphertext
|
||||
char* enc_p = encrypted.c_str();
|
||||
enc_p[100] ^= 0xFF;
|
||||
|
||||
bufferlist decrypted;
|
||||
// Decryption should fail due to tag mismatch
|
||||
ASSERT_FALSE(aes->decrypt(encrypted, 0, encrypted.length(), decrypted, 0, null_yield));
|
||||
}
|
||||
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_salt_key_isolation)
|
||||
{
|
||||
/**
|
||||
* Verify salt-based key derivation produces unique keys:
|
||||
* 1. Two instances with same raw key but different salts
|
||||
* 2. After derive_object_key(), they have different derived keys
|
||||
* 3. Cross-decryption fails (GCM tag mismatch)
|
||||
* 4. Self-decryption succeeds
|
||||
*/
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
uint8_t key[32];
|
||||
for (size_t i = 0; i < sizeof(key); i++)
|
||||
key[i] = i * 11;
|
||||
|
||||
const size_t size = 4096;
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for (size_t i = 0; i < size; i++)
|
||||
p[i] = i & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
// Instance 1: auto-generated salt + derive
|
||||
auto aes1(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes1.get(), nullptr);
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes1.get(), key, 32,
|
||||
"bucket-id-1", "object.txt", 0));
|
||||
|
||||
bufferlist encrypted1;
|
||||
ASSERT_TRUE(aes1->encrypt(input, 0, size, encrypted1, 0, null_yield));
|
||||
|
||||
// Instance 2: different auto-generated salt + derive with same identity
|
||||
auto aes2(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes2.get(), nullptr);
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes2.get(), key, 32,
|
||||
"bucket-id-1", "object.txt", 0));
|
||||
|
||||
// Cross-decrypt should FAIL (different salts -> different derived keys)
|
||||
bufferlist decrypted_wrong;
|
||||
ASSERT_FALSE(aes2->decrypt(encrypted1, 0, encrypted1.length(),
|
||||
decrypted_wrong, 0, null_yield));
|
||||
|
||||
// Self-decrypt should succeed
|
||||
bufferlist decrypted_correct;
|
||||
ASSERT_TRUE(aes1->decrypt(encrypted1, 0, encrypted1.length(),
|
||||
decrypted_correct, 0, null_yield));
|
||||
ASSERT_EQ(decrypted_correct.length(), size);
|
||||
ASSERT_EQ(std::string_view(input.c_str(), size),
|
||||
std::string_view(decrypted_correct.c_str(), size));
|
||||
}
|
||||
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_salt_restore)
|
||||
{
|
||||
/**
|
||||
* Simulate the full encrypt/decrypt flow with stored salt:
|
||||
* 1. Encrypt with auto-generated salt + derive_object_key
|
||||
* 2. Extract salt (would be stored in RGW_ATTR_CRYPT_SALT)
|
||||
* 3. Create new instance with restored salt + derive_object_key
|
||||
* 4. Decrypt successfully
|
||||
*/
|
||||
const NoDoutPrefix no_dpp(g_ceph_context, dout_subsys);
|
||||
uint8_t key[32];
|
||||
for (size_t i = 0; i < sizeof(key); i++)
|
||||
key[i] = i * 13;
|
||||
|
||||
const size_t size = 8192;
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for (size_t i = 0; i < size; i++)
|
||||
p[i] = (i * 7) & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
// Simulate encryption path
|
||||
auto aes_encrypt(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes_encrypt.get(), nullptr);
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes_encrypt.get(), key, 32,
|
||||
"my-bucket-id", "my-object", 0));
|
||||
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes_encrypt->encrypt(input, 0, size, encrypted, 0, null_yield));
|
||||
|
||||
// Extract salt (simulates storing to RGW_ATTR_CRYPT_SALT)
|
||||
std::string stored_salt = AES_256_GCM_get_salt(aes_encrypt.get());
|
||||
ASSERT_EQ(stored_salt.size(), AES_256_GCM_SALT_SIZE);
|
||||
|
||||
// Release encryption instance (simulates different request)
|
||||
aes_encrypt.reset();
|
||||
|
||||
// Simulate decryption path with restored salt + same identity
|
||||
auto aes_decrypt(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32,
|
||||
reinterpret_cast<const uint8_t*>(stored_salt.c_str()),
|
||||
stored_salt.size()));
|
||||
ASSERT_NE(aes_decrypt.get(), nullptr);
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes_decrypt.get(), key, 32,
|
||||
"my-bucket-id", "my-object", 0));
|
||||
|
||||
bufferlist decrypted;
|
||||
ASSERT_TRUE(aes_decrypt->decrypt(encrypted, 0, encrypted.length(), decrypted, 0, null_yield));
|
||||
|
||||
ASSERT_EQ(decrypted.length(), size);
|
||||
ASSERT_EQ(std::string_view(input.c_str(), size),
|
||||
std::string_view(decrypted.c_str(), size));
|
||||
}
|
||||
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_key_derivation)
|
||||
{
|
||||
NoDoutPrefix no_dpp(g_ceph_context, ceph_subsys_rgw);
|
||||
uint8_t user_key[32];
|
||||
for (size_t i = 0; i < sizeof(user_key); i++) user_key[i] = i;
|
||||
const std::string plaintext = "test data for key derivation";
|
||||
|
||||
// Test 1: Same identity produces same derived key (encrypt/decrypt roundtrip)
|
||||
{
|
||||
auto aes1(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32));
|
||||
ASSERT_NE(aes1.get(), nullptr);
|
||||
std::string salt = AES_256_GCM_get_salt(aes1.get());
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes1.get(), user_key, 32,
|
||||
"mybucket", "myobject", 0));
|
||||
|
||||
bufferlist input;
|
||||
input.append(plaintext);
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes1->encrypt(input, 0, input.length(), encrypted, 0, null_yield));
|
||||
|
||||
auto aes2(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32,
|
||||
reinterpret_cast<const uint8_t*>(salt.c_str()),
|
||||
salt.size()));
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes2.get(), user_key, 32,
|
||||
"mybucket", "myobject", 0));
|
||||
|
||||
bufferlist decrypted;
|
||||
ASSERT_TRUE(aes2->decrypt(encrypted, 0, encrypted.length(), decrypted, 0, null_yield));
|
||||
ASSERT_EQ(std::string_view(input.c_str(), input.length()),
|
||||
std::string_view(decrypted.c_str(), decrypted.length()));
|
||||
}
|
||||
|
||||
// Test 2: Different bucket/object produces different derived key
|
||||
{
|
||||
auto aes1(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32));
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes1.get(), user_key, 32,
|
||||
"bucket1", "object1", 0));
|
||||
std::string salt = AES_256_GCM_get_salt(aes1.get());
|
||||
|
||||
auto aes2(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32,
|
||||
reinterpret_cast<const uint8_t*>(salt.c_str()),
|
||||
salt.size()));
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes2.get(), user_key, 32,
|
||||
"bucket2", "object2", 0));
|
||||
|
||||
bufferlist input;
|
||||
input.append(plaintext);
|
||||
bufferlist enc1, enc2;
|
||||
ASSERT_TRUE(aes1->encrypt(input, 0, input.length(), enc1, 0, null_yield));
|
||||
ASSERT_TRUE(aes2->encrypt(input, 0, input.length(), enc2, 0, null_yield));
|
||||
ASSERT_NE(std::string_view(enc1.c_str(), enc1.length()),
|
||||
std::string_view(enc2.c_str(), enc2.length()));
|
||||
}
|
||||
|
||||
// Test 3: Different part numbers produce different derived keys
|
||||
{
|
||||
auto aes1(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32));
|
||||
std::string salt = AES_256_GCM_get_salt(aes1.get());
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes1.get(), user_key, 32,
|
||||
"bucket", "object", 1));
|
||||
|
||||
auto aes2(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32,
|
||||
reinterpret_cast<const uint8_t*>(salt.c_str()),
|
||||
salt.size()));
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes2.get(), user_key, 32,
|
||||
"bucket", "object", 2));
|
||||
|
||||
bufferlist input;
|
||||
input.append(plaintext);
|
||||
bufferlist enc1, enc2;
|
||||
ASSERT_TRUE(aes1->encrypt(input, 0, input.length(), enc1, 0, null_yield));
|
||||
ASSERT_TRUE(aes2->encrypt(input, 0, input.length(), enc2, 0, null_yield));
|
||||
ASSERT_NE(std::string_view(enc1.c_str(), enc1.length()),
|
||||
std::string_view(enc2.c_str(), enc2.length()));
|
||||
}
|
||||
|
||||
// Test 4: Wrong identity fails decryption (auth tag mismatch)
|
||||
{
|
||||
auto aes_enc(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32));
|
||||
std::string salt = AES_256_GCM_get_salt(aes_enc.get());
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes_enc.get(), user_key, 32,
|
||||
"bucket1", "object1", 0));
|
||||
|
||||
bufferlist input;
|
||||
input.append(plaintext);
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes_enc->encrypt(input, 0, input.length(), encrypted, 0, null_yield));
|
||||
|
||||
auto aes_dec(AES_256_GCM_create(&no_dpp, g_ceph_context, &user_key[0], 32,
|
||||
reinterpret_cast<const uint8_t*>(salt.c_str()),
|
||||
salt.size()));
|
||||
ASSERT_TRUE(AES_256_GCM_derive_object_key(aes_dec.get(), user_key, 32,
|
||||
"bucket2", "object2", 0));
|
||||
|
||||
bufferlist decrypted;
|
||||
ASSERT_FALSE(aes_dec->decrypt(encrypted, 0, encrypted.length(), decrypted, 0, null_yield));
|
||||
}
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_chunk_reorder_detection)
|
||||
{
|
||||
// Verify that swapping chunk positions is detected via AAD
|
||||
NoDoutPrefix no_dpp(g_ceph_context, ceph_subsys_rgw);
|
||||
uint8_t key[32];
|
||||
for (size_t i = 0; i < sizeof(key); i++) key[i] = i * 7;
|
||||
|
||||
auto aes(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes.get(), nullptr);
|
||||
|
||||
// Create 2 chunks of data (8192 bytes = 2 x 4096)
|
||||
const size_t size = 8192;
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for (size_t i = 0; i < size; i++) p[i] = i & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes->encrypt(input, 0, size, encrypted, 0, null_yield));
|
||||
|
||||
// Encrypted layout: [chunk0_cipher(4096)][tag0(16)][chunk1_cipher(4096)][tag1(16)]
|
||||
// Total: 8192 + 32 = 8224 bytes
|
||||
ASSERT_EQ(encrypted.length(), 8224u);
|
||||
|
||||
// Swap the two encrypted chunks (including their tags)
|
||||
buffer::ptr enc_copy(encrypted.length());
|
||||
encrypted.begin().copy(encrypted.length(), enc_copy.c_str());
|
||||
char* enc_data = enc_copy.c_str();
|
||||
const size_t enc_chunk_size = 4096 + 16; // ciphertext + tag
|
||||
char temp[4112];
|
||||
memcpy(temp, enc_data, enc_chunk_size); // save chunk0
|
||||
memcpy(enc_data, enc_data + enc_chunk_size, enc_chunk_size); // chunk1 -> chunk0 position
|
||||
memcpy(enc_data + enc_chunk_size, temp, enc_chunk_size); // chunk0 -> chunk1 position
|
||||
|
||||
bufferlist swapped;
|
||||
swapped.append(enc_copy);
|
||||
|
||||
// Decryption should fail because AAD (chunk_index) won't match
|
||||
bufferlist decrypted;
|
||||
ASSERT_FALSE(aes->decrypt(swapped, 0, swapped.length(), decrypted, 0, null_yield));
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_aad_roundtrip)
|
||||
{
|
||||
// Verify basic encrypt/decrypt still works correctly with AAD
|
||||
NoDoutPrefix no_dpp(g_ceph_context, ceph_subsys_rgw);
|
||||
uint8_t key[32];
|
||||
for (size_t i = 0; i < sizeof(key); i++) key[i] = i * 11;
|
||||
|
||||
// Test multiple chunk scenarios
|
||||
for (size_t size : {4096u, 8192u, 12288u, 10000u}) {
|
||||
auto aes(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes.get(), nullptr);
|
||||
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for (size_t i = 0; i < size; i++) p[i] = (i * 17) & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes->encrypt(input, 0, size, encrypted, 0, null_yield));
|
||||
|
||||
bufferlist decrypted;
|
||||
ASSERT_TRUE(aes->decrypt(encrypted, 0, encrypted.length(), decrypted, 0, null_yield));
|
||||
|
||||
ASSERT_EQ(decrypted.length(), size);
|
||||
ASSERT_EQ(std::string_view(input.c_str(), size),
|
||||
std::string_view(decrypted.c_str(), size));
|
||||
}
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_AES_256_GCM_aad_offset_mismatch)
|
||||
{
|
||||
// Verify AAD detects wrong stream offset during decryption
|
||||
NoDoutPrefix no_dpp(g_ceph_context, ceph_subsys_rgw);
|
||||
uint8_t key[32];
|
||||
for (size_t i = 0; i < sizeof(key); i++) key[i] = i * 13;
|
||||
|
||||
auto aes(AES_256_GCM_create(&no_dpp, g_ceph_context, &key[0], 32));
|
||||
ASSERT_NE(aes.get(), nullptr);
|
||||
|
||||
const size_t size = 4096;
|
||||
buffer::ptr buf(size);
|
||||
char* p = buf.c_str();
|
||||
for (size_t i = 0; i < size; i++) p[i] = i & 0xFF;
|
||||
|
||||
bufferlist input;
|
||||
input.append(buf);
|
||||
|
||||
// Encrypt at offset 8192 (chunk index 2)
|
||||
off_t stream_offset = 8192;
|
||||
bufferlist encrypted;
|
||||
ASSERT_TRUE(aes->encrypt(input, 0, size, encrypted, stream_offset, null_yield));
|
||||
|
||||
// Decrypt with same offset - should succeed
|
||||
bufferlist decrypted;
|
||||
ASSERT_TRUE(aes->decrypt(encrypted, 0, encrypted.length(), decrypted, stream_offset, null_yield));
|
||||
ASSERT_EQ(std::string_view(input.c_str(), size),
|
||||
std::string_view(decrypted.c_str(), size));
|
||||
|
||||
// Decrypt with wrong offset (0 instead of 8192) - should FAIL (AAD mismatch)
|
||||
bufferlist decrypted_wrong;
|
||||
ASSERT_FALSE(aes->decrypt(encrypted, 0, encrypted.length(), decrypted_wrong, 0, null_yield));
|
||||
}
|
||||
|
||||
|
||||
// Test helper class to expose private methods for unit testing
|
||||
// (declared as friend in RGWGetObj_BlockDecrypt)
|
||||
// PartLocation tests use the free function find_part_for_offset()
|
||||
// from rgw_range_projection.h instead of the removed class method.
|
||||
#include "rgw_range_projection.h"
|
||||
|
||||
TEST(TestRGWCrypto, verify_PartLocation_single_part)
|
||||
{
|
||||
std::vector<size_t> parts = {10 * 1024 * 1024};
|
||||
size_t idx; off_t ofs_in_part, cumulative;
|
||||
|
||||
find_part_for_offset(0, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u);
|
||||
ASSERT_EQ(ofs_in_part, 0);
|
||||
ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(1000, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u);
|
||||
ASSERT_EQ(ofs_in_part, 1000);
|
||||
ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(10 * 1024 * 1024 - 1, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u);
|
||||
ASSERT_EQ(ofs_in_part, 10 * 1024 * 1024 - 1);
|
||||
ASSERT_EQ(cumulative, 0);
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_PartLocation_multipart_cbc)
|
||||
{
|
||||
const size_t ps = 5 * 1024 * 1024;
|
||||
std::vector<size_t> parts = {ps, ps, ps};
|
||||
size_t idx; off_t ofs_in_part, cumulative;
|
||||
|
||||
find_part_for_offset(0, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(ps - 1, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, (off_t)(ps - 1)); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(ps, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 1u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, (off_t)ps);
|
||||
|
||||
find_part_for_offset(ps + 1000, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 1u); ASSERT_EQ(ofs_in_part, 1000); ASSERT_EQ(cumulative, (off_t)ps);
|
||||
|
||||
find_part_for_offset(2 * ps, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, (off_t)(2 * ps));
|
||||
|
||||
find_part_for_offset(3 * ps - 100, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, (off_t)(ps - 100)); ASSERT_EQ(cumulative, (off_t)(2 * ps));
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_PartLocation_multipart_aead)
|
||||
{
|
||||
const size_t pp = 1024 * AEAD_CHUNK_SIZE; // 4MB plaintext per part
|
||||
const size_t ep = aead_plaintext_to_encrypted_size(pp);
|
||||
std::vector<size_t> parts = {ep, ep, ep};
|
||||
size_t idx; off_t ofs_in_part, cumulative;
|
||||
|
||||
find_part_for_offset(0, parts, 4096, 4112, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(pp - 1, parts, 4096, 4112, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, (off_t)(pp - 1)); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(pp, parts, 4096, 4112, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 1u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, (off_t)ep);
|
||||
|
||||
find_part_for_offset(pp + 8192, parts, 4096, 4112, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 1u); ASSERT_EQ(ofs_in_part, 8192); ASSERT_EQ(cumulative, (off_t)ep);
|
||||
|
||||
find_part_for_offset(2 * pp, parts, 4096, 4112, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, (off_t)(2 * ep));
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_PartLocation_clamp_to_last)
|
||||
{
|
||||
const size_t ps = 5 * 1024 * 1024;
|
||||
std::vector<size_t> parts = {ps, ps, ps};
|
||||
size_t idx; off_t ofs_in_part, cumulative;
|
||||
|
||||
find_part_for_offset(2 * ps + 1000, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, 1000);
|
||||
|
||||
find_part_for_offset(2 * ps + 1000, parts, 4096, 4096, true, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, 1000);
|
||||
|
||||
find_part_for_offset(3 * ps + 5000, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 3u); ASSERT_EQ(ofs_in_part, 5000);
|
||||
|
||||
find_part_for_offset(3 * ps + 5000, parts, 4096, 4096, true, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, (off_t)(ps + 5000));
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_PartLocation_unequal_parts)
|
||||
{
|
||||
// CBC parts: 1MB, 3MB, 2MB
|
||||
std::vector<size_t> parts = {1024 * 1024, 3 * 1024 * 1024, 2 * 1024 * 1024};
|
||||
size_t idx; off_t ofs_in_part, cumulative;
|
||||
|
||||
find_part_for_offset(500 * 1024, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, 500 * 1024); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(1536 * 1024, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 1u); ASSERT_EQ(ofs_in_part, 512 * 1024); ASSERT_EQ(cumulative, 1024 * 1024);
|
||||
|
||||
find_part_for_offset(4608 * 1024, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 2u); ASSERT_EQ(ofs_in_part, 512 * 1024); ASSERT_EQ(cumulative, 4 * 1024 * 1024);
|
||||
}
|
||||
|
||||
TEST(TestRGWCrypto, verify_PartLocation_no_manifest)
|
||||
{
|
||||
// Empty parts = single-part object
|
||||
std::vector<size_t> parts = {};
|
||||
size_t idx; off_t ofs_in_part, cumulative;
|
||||
|
||||
find_part_for_offset(0, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, 0); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(1000, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, 1000); ASSERT_EQ(cumulative, 0);
|
||||
|
||||
find_part_for_offset(10 * 1024 * 1024, parts, 4096, 4096, false, idx, ofs_in_part, cumulative);
|
||||
ASSERT_EQ(idx, 0u); ASSERT_EQ(ofs_in_part, 10 * 1024 * 1024); ASSERT_EQ(cumulative, 0);
|
||||
}
|
||||
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
auto args = argv_to_vec(argc, argv);
|
||||
auto cct = global_init(NULL, args, CEPH_ENTITY_TYPE_CLIENT,
|
||||
|
||||
338
src/test/rgw/test_rgw_range_projection.cc
Normal file
338
src/test/rgw/test_rgw_range_projection.cc
Normal file
@ -0,0 +1,338 @@
|
||||
// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*-
|
||||
// vim: ts=8 sw=2 sts=2 expandtab ft=cpp
|
||||
|
||||
/*
|
||||
* Ceph - scalable distributed file system
|
||||
*
|
||||
* Copyright (C) 2026 Red Hat, Inc.
|
||||
*
|
||||
* This is free software; you can redistribute it and/or
|
||||
* modify it under the terms of the GNU Lesser General Public
|
||||
* License version 2.1, as published by the Free Software
|
||||
* Foundation. See file COPYING.
|
||||
*/
|
||||
|
||||
#include <gtest/gtest.h>
|
||||
#include "rgw_range_projection.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
// helpers to build synthetic compression block maps
|
||||
|
||||
static RGWCompressionInfo make_cs_info(
|
||||
const vector<pair<uint64_t, uint64_t>>& blocks,
|
||||
uint64_t orig_size,
|
||||
const string& type = "zlib")
|
||||
{
|
||||
/*
|
||||
* Each pair is (uncompressed_size, compressed_size).
|
||||
* We compute old_ofs / new_ofs cumulatively.
|
||||
*/
|
||||
RGWCompressionInfo cs;
|
||||
cs.compression_type = type;
|
||||
cs.orig_size = orig_size;
|
||||
|
||||
uint64_t old_ofs = 0;
|
||||
uint64_t new_ofs = 0;
|
||||
for (auto& [plain_len, comp_len] : blocks) {
|
||||
compression_block b;
|
||||
b.old_ofs = old_ofs;
|
||||
b.new_ofs = new_ofs;
|
||||
b.len = comp_len;
|
||||
cs.blocks.push_back(b);
|
||||
old_ofs += plain_len;
|
||||
new_ofs += comp_len;
|
||||
}
|
||||
return cs;
|
||||
}
|
||||
|
||||
TEST(DiskRange, EmptyWhenEndLessThanOfs)
|
||||
{
|
||||
DiskRange r;
|
||||
r.ofs = 10;
|
||||
r.end = 5;
|
||||
EXPECT_TRUE(r.empty());
|
||||
EXPECT_EQ(0u, r.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(DiskRange, NotEmptyWhenEndEqualsOfs)
|
||||
{
|
||||
DiskRange r;
|
||||
r.ofs = 10;
|
||||
r.end = 10;
|
||||
EXPECT_FALSE(r.empty());
|
||||
EXPECT_EQ(1u, r.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(DiskRange, DiskBytesCorrect)
|
||||
{
|
||||
DiskRange r;
|
||||
r.ofs = 100;
|
||||
r.end = 199;
|
||||
EXPECT_EQ(100u, r.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(DiskRange, DefaultIsEmpty)
|
||||
{
|
||||
DiskRange r;
|
||||
// default: ofs=0, end=-1
|
||||
EXPECT_TRUE(r.empty());
|
||||
EXPECT_EQ(0u, r.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecompress, FullObjectReturnsEntireCompressedRange)
|
||||
{
|
||||
// 3 blocks: 4096 plain -> 2048 compressed each
|
||||
auto cs = make_cs_info(
|
||||
{{4096, 2048}, {4096, 2048}, {4096, 2048}},
|
||||
12288);
|
||||
|
||||
auto dr = project_compress_range(0, 12287, cs, false);
|
||||
|
||||
// should span all blocks: disk [0, 6143]
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(6143, dr.end);
|
||||
EXPECT_EQ(0u, dr.first_block_idx);
|
||||
EXPECT_EQ(2u, dr.last_block_idx);
|
||||
EXPECT_EQ(0, dr.q_ofs);
|
||||
EXPECT_EQ(12288u, dr.q_len);
|
||||
EXPECT_EQ(6144u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecompress, PartialRangeMapsToCorrectBlocks)
|
||||
{
|
||||
// 4 blocks, each 4096 plain -> 2000 compressed
|
||||
auto cs = make_cs_info(
|
||||
{{4096, 2000}, {4096, 2000}, {4096, 2000}, {4096, 2000}},
|
||||
16384);
|
||||
|
||||
// request plaintext [4200, 8191] — entirely within block 1
|
||||
auto dr = project_compress_range(4200, 8191, cs, true);
|
||||
|
||||
EXPECT_EQ(1u, dr.first_block_idx);
|
||||
EXPECT_EQ(1u, dr.last_block_idx);
|
||||
EXPECT_EQ(104, dr.q_ofs); // 4200 - 4096
|
||||
EXPECT_EQ(3992u, dr.q_len); // 8191 - 4200 + 1
|
||||
EXPECT_EQ(2000, dr.ofs);
|
||||
EXPECT_EQ(3999, dr.end);
|
||||
}
|
||||
|
||||
TEST(ProjectDecompress, PartialRangeSpansMultipleBlocks)
|
||||
{
|
||||
// 4 blocks, each 4096 plain -> 2000 compressed
|
||||
auto cs = make_cs_info(
|
||||
{{4096, 2000}, {4096, 2000}, {4096, 2000}, {4096, 2000}},
|
||||
16384);
|
||||
|
||||
// request plaintext [100, 8999] — spans blocks 0, 1, and 2
|
||||
auto dr = project_compress_range(100, 8999, cs, true);
|
||||
|
||||
EXPECT_EQ(0u, dr.first_block_idx);
|
||||
EXPECT_EQ(2u, dr.last_block_idx);
|
||||
EXPECT_EQ(100, dr.q_ofs);
|
||||
EXPECT_EQ(8900u, dr.q_len);
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(5999, dr.end);
|
||||
}
|
||||
|
||||
TEST(ProjectDecompress, SingleBlock)
|
||||
{
|
||||
auto cs = make_cs_info({{8192, 4000}}, 8192);
|
||||
|
||||
auto dr = project_compress_range(100, 999, cs, true);
|
||||
|
||||
EXPECT_EQ(0u, dr.first_block_idx);
|
||||
EXPECT_EQ(0u, dr.last_block_idx);
|
||||
EXPECT_EQ(100, dr.q_ofs);
|
||||
EXPECT_EQ(900u, dr.q_len);
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(3999, dr.end);
|
||||
EXPECT_EQ(4000u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptCBC, AlignedToBlockBoundary)
|
||||
{
|
||||
// [100, 999] with block_size=4096 -> aligned to [0, 4095]
|
||||
auto dr = project_encrypt_range(100, 999, 4096, 4096, 8192, {});
|
||||
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(4095, dr.end);
|
||||
EXPECT_EQ(100u, dr.enc_begin_skip);
|
||||
EXPECT_EQ(100u, dr.skip);
|
||||
EXPECT_EQ(4096u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptCBC, FullObject)
|
||||
{
|
||||
// full object [0, 8191]
|
||||
auto dr = project_encrypt_range(0, 8191, 4096, 4096, 8192, {});
|
||||
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(8191, dr.end);
|
||||
EXPECT_EQ(0u, dr.enc_begin_skip);
|
||||
EXPECT_EQ(0u, dr.skip);
|
||||
EXPECT_EQ(8192u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptCBC, NoExpansion)
|
||||
{
|
||||
// CBC: enc_block_size == block_size, so no expansion
|
||||
// [4096, 8191] -> [4096, 8191]
|
||||
auto dr = project_encrypt_range(4096, 8191, 4096, 4096, 16384, {});
|
||||
|
||||
EXPECT_EQ(4096, dr.ofs);
|
||||
EXPECT_EQ(8191, dr.end);
|
||||
EXPECT_EQ(0u, dr.enc_begin_skip);
|
||||
EXPECT_EQ(4096u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptCBC, MultipartCrossesPartBoundary)
|
||||
{
|
||||
// 2 parts, each 8192 bytes encrypted (=plaintext for CBC)
|
||||
// request [4000, 12000] crosses parts
|
||||
vector<size_t> parts = {8192, 8192};
|
||||
auto dr = project_encrypt_range(4000, 12000, 4096, 4096, 16384, parts);
|
||||
|
||||
// 4000 < 4096, so start is within first block of part 0
|
||||
EXPECT_EQ(4000u, dr.enc_begin_skip);
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
// end lands in part 1 at offset 3808, block-aligned to 4095, plus part 0 size
|
||||
EXPECT_EQ(12287, dr.end);
|
||||
}
|
||||
|
||||
static const size_t GCM_BLOCK = 4096;
|
||||
static const size_t GCM_ENC_BLOCK = 4112; // 4096 + 16 byte tag
|
||||
|
||||
TEST(ProjectDecryptGCM, SizeExpansion)
|
||||
{
|
||||
// plaintext [0, 4095] -> encrypted [0, 4111]
|
||||
// 1 chunk of 4096 plain -> 4112 encrypted
|
||||
auto dr = project_encrypt_range(0, 4095, GCM_BLOCK, GCM_ENC_BLOCK, 4112, {});
|
||||
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(4111, dr.end);
|
||||
EXPECT_EQ(0u, dr.enc_begin_skip);
|
||||
EXPECT_EQ(4112u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptGCM, TailClampedToEncryptedTotalSize)
|
||||
{
|
||||
// object with 2 chunks: total encrypted = 2 * 4112 = 8224
|
||||
// request plaintext [0, 8191] -> should clamp to [0, 8223]
|
||||
auto dr = project_encrypt_range(0, 8191, GCM_BLOCK, GCM_ENC_BLOCK, 8224, {});
|
||||
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(8223, dr.end);
|
||||
EXPECT_EQ(8224u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptGCM, MidRangeAlignment)
|
||||
{
|
||||
// plaintext [100, 4095] with 2 chunks (total 8224)
|
||||
auto dr = project_encrypt_range(100, 4095, GCM_BLOCK, GCM_ENC_BLOCK, 8224, {});
|
||||
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(4111, dr.end);
|
||||
EXPECT_EQ(100u, dr.enc_begin_skip);
|
||||
EXPECT_EQ(4112u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptGCM, MultipartCumulativeOffsets)
|
||||
{
|
||||
// 2 parts: part0 = 4112 bytes (1 chunk), part1 = 8224 bytes (2 chunks)
|
||||
// total encrypted = 12336
|
||||
// request plaintext [0, 4095]: all in part0
|
||||
vector<size_t> parts = {4112, 8224};
|
||||
auto dr = project_encrypt_range(0, 4095, GCM_BLOCK, GCM_ENC_BLOCK, 12336, parts);
|
||||
|
||||
// entirely within part 0
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(4111, dr.end);
|
||||
EXPECT_EQ(0u, dr.enc_begin_skip);
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptGCM, MultipartSecondPart)
|
||||
{
|
||||
// 2 parts: part0 = 4112 (1 plain chunk), part1 = 8224 (2 plain chunks)
|
||||
// plaintext sizes: part0=4096, part1=8192
|
||||
// request plaintext [4096, 8191]: starts at part1 offset 0
|
||||
vector<size_t> parts = {4112, 8224};
|
||||
auto dr = project_encrypt_range(4096, 8191, GCM_BLOCK, GCM_ENC_BLOCK, 12336, parts);
|
||||
|
||||
// starts at beginning of part 1 (cumulative enc offset 4112)
|
||||
EXPECT_EQ(4112, dr.ofs);
|
||||
EXPECT_EQ(8223, dr.end); // 4112 + 4111
|
||||
EXPECT_EQ(0u, dr.enc_begin_skip);
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptGCM, EndBeyondLastPartClamped)
|
||||
{
|
||||
// 1 part of 4112 bytes (1 chunk, 4096 plain)
|
||||
// request beyond: [0, 99999]
|
||||
vector<size_t> parts = {4112};
|
||||
auto dr = project_encrypt_range(0, 99999, GCM_BLOCK, GCM_ENC_BLOCK, 4112, parts);
|
||||
|
||||
// end is clamped to the single part's encrypted size
|
||||
EXPECT_EQ(0, dr.ofs);
|
||||
EXPECT_EQ(4111, dr.end);
|
||||
}
|
||||
|
||||
TEST(ProjectDecryptGCM, InvalidRangeStartClampedToEnd)
|
||||
{
|
||||
// start > total plaintext but within part index bounds
|
||||
// For a simple (non-multipart) object: [50000, 50100] with total=4112
|
||||
auto dr = project_encrypt_range(50000, 50100, GCM_BLOCK, GCM_ENC_BLOCK, 4112, {});
|
||||
|
||||
// ofs expands well past encrypted_total_size, so ofs > end -> empty range
|
||||
EXPECT_TRUE(dr.empty());
|
||||
EXPECT_EQ(0u, dr.disk_bytes());
|
||||
}
|
||||
|
||||
TEST(CryptHelpers, LogicalToEncOffsetCBC)
|
||||
{
|
||||
// CBC: identity
|
||||
EXPECT_EQ(1000, crypt_logical_to_enc_offset(1000, 4096, 4096));
|
||||
EXPECT_EQ(0, crypt_logical_to_enc_offset(0, 4096, 4096));
|
||||
EXPECT_EQ(8192, crypt_logical_to_enc_offset(8192, 4096, 4096));
|
||||
}
|
||||
|
||||
TEST(CryptHelpers, LogicalToEncOffsetGCM)
|
||||
{
|
||||
// GCM: chunk 0 stays in [0..4095]
|
||||
EXPECT_EQ(0, crypt_logical_to_enc_offset(0, 4096, 4112));
|
||||
EXPECT_EQ(4095, crypt_logical_to_enc_offset(4095, 4096, 4112));
|
||||
// chunk 1 starts at 4112
|
||||
EXPECT_EQ(4112, crypt_logical_to_enc_offset(4096, 4096, 4112));
|
||||
// chunk 2 starts at 8224
|
||||
EXPECT_EQ(8224, crypt_logical_to_enc_offset(8192, 4096, 4112));
|
||||
}
|
||||
|
||||
TEST(CryptHelpers, AlignEncBlockEndCBC)
|
||||
{
|
||||
// CBC: identity
|
||||
EXPECT_EQ(4095, crypt_align_enc_block_end(4095, 4096, 4096));
|
||||
EXPECT_EQ(100, crypt_align_enc_block_end(100, 4096, 4096));
|
||||
}
|
||||
|
||||
TEST(CryptHelpers, AlignEncBlockEndGCM)
|
||||
{
|
||||
// GCM: rounds up to end of encrypted block
|
||||
EXPECT_EQ(4111, crypt_align_enc_block_end(0, 4096, 4112));
|
||||
EXPECT_EQ(4111, crypt_align_enc_block_end(4095, 4096, 4112));
|
||||
EXPECT_EQ(8223, crypt_align_enc_block_end(4112, 4096, 4112));
|
||||
}
|
||||
|
||||
TEST(CryptHelpers, EncToPlaintextSize)
|
||||
{
|
||||
// CBC: identity
|
||||
EXPECT_EQ(8192u, crypt_enc_to_plaintext_size(8192, 4096, 4096));
|
||||
// GCM: 1 chunk
|
||||
EXPECT_EQ(4096u, crypt_enc_to_plaintext_size(4112, 4096, 4112));
|
||||
// GCM: 2 chunks
|
||||
EXPECT_EQ(8192u, crypt_enc_to_plaintext_size(8224, 4096, 4112));
|
||||
// GCM: partial chunk (4100 - 16 tag = 4084 plaintext)
|
||||
EXPECT_EQ(4084u, crypt_enc_to_plaintext_size(4100, 4096, 4112));
|
||||
// GCM: less than tag size -> 0
|
||||
EXPECT_EQ(0u, crypt_enc_to_plaintext_size(10, 4096, 4112));
|
||||
}
|
||||
|
||||
Loading…
Reference in New Issue
Block a user