rgw/cls: fix usage log issues caused by incorrect form of name-by-user keys
Reviewed-by: Yixin Jin <yjin@akamai.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
now that the workers run as coroutines instead of threads, we can afford
to run several more workers per bucket
Signed-off-by: Casey Bodley <cbodley@redhat.com>
1. The old name-by-user keys start with user/payer IDs. If those IDs
start with '0', the resulting name-by-user keys could fall within
the range of name-by-time keys, which has the prefix of '0' followed
by 11 digits of the epoch. The intermingle of name-by-user and
name-by-time keys causes the usage function to mistakenly treat
name-by-user usage logs as name-by-time logs. This leads issues like
failure to trim old logs because some name-by-user logs with later
epoch listed BEFORE name-by-time logs with earlier epoch, causing
the trim logic to stop prematurally.
2. To fix the above issue, adding prefix '~' in name-by-user keys when
the user/payer IDs starting with '0'.
3. Introduce a rgw config option to indicate the need to deal with
the co-existence of old and new name-by-user keys. Since handling
both keys incurs performance penalty, this option helps avoid it
once the cluster no longer has the old keys.
4. usage_iterate_range() is splitted to usage_iterate_range_by_user()
and usage_iterate_range_by_time() as only the former needs to deal
with the potential co-existence of old and new name-by-user keys.
This split intends to simplify the logic.
5. "usage_basic" test in test_cls_rgw.cc is adapted slightly to reproduce
the issue mentioned above.
6. A new test case "usage_key_transition" is added to test the handling
of the co-existence of old and new keys.
Signed-off-by: Matthew N Heler <mheler@akamai.com>
* sharding is based on hash of bucket_name:object_key
* compatible with v1 topics
* include tests to cover backward compatibility and mixed cluster
Signed-off-by: Adarsh <dev.9401adarsh@gmail.com>
This commit adds docs for the new config option introduced as
well as updates the feature docs on how to use this config
option.
Fixes: https://tracker.ceph.com/issues/72619
Signed-off-by: Shraddha Agrawal <shraddhaag@ibm.com>
Removed `openssl_engine_opts` configuration option. OpenSSL engine configurations in string format are no longer supported.
Added `openssl_conf` configuration option for loading specified providers as default providers.
Signed-off-by: cailianchun <arm7star@qq.com>
* refs/pull/63214/head:
release note: add a note that "subvolume info" cmd output can also...
doc/cephfs: update docs since "subvolume info" cmd output can also...
qa/cephfs: add test to check clone source info's present in...
mgr/vol: show clone source info in "subvolume info" cmd output
mgr/vol: keep clone source info even after cloning is finished
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
mgr/vol: include group name in subvolume's pool namespace name
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
librbd/api/Mirror: return EINVAL from image_get_mode() when the image is disabled for mirroring
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
... when mirroring is not enabled for the image.
Mirror::image_resync() returns ENOENT when mirroring is disabled for the
image. Instead, make it return EINVAL indicating that the call is
invalid when mirroring is not enabled for the image. This also causes
the public facing C, C++, and Python APIs that resync an image to
return EINVAL or raise an equivalent exception when mirroring is not
enabled for the image.
Signed-off-by: Ramana Raja <rraja@redhat.com>
... instead of ENOENT when mirroring is not enabled for the image.
The PromoteRequest async state machine returns ENOENT when mirroring is
not enabled for the image. Instead, make it return EINVAL similar to
DemoteRequest's behavior, which is more appropriate. This also causes
the public facing C, C++, and Python APIs that promote an image
to return EINVAL or raise an equivalent exception when mirroring is
not enabled for the image.
Signed-off-by: Ramana Raja <rraja@redhat.com>
... when the image is disabled for mirroring.
When an image is disabled for mirroring, fetching the image's
mirroring mode is invalid. So, modify the Mirror::image_get_mode()
internal API to return EINVAL instead of success when mirroring is
disabled.
The Mirror::image_get_mode() method is called by the public C++, C, and
Python APIs that fetch the mirroring mode of an image. The behavior of
these public APIs will change. They will return an error code or raise
an exception indicating that it's an invalid operation to fetch the
image's mirroring mode when mirroring is disabled.
Fixes: https://tracker.ceph.com/issues/71226
Signed-off-by: Ramana Raja <rraja@redhat.com>
The idea here is to leverage the `SO_REUSEPORT` param supported by kernel 3.9 and above, which allows the multiple services to run on the same port.
Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
Removing an image that is a member of a group has always been
disallowed. However, moving an image that is a member of a group to
trash is currently allowed and this is deceptive -- the only reason for
a user to move an image to trash should be the intent to remove it.
More importantly, group APIs operate in terms of image names -- there
are no corresponding variants that would operate in terms of image IDs.
For example, even though internally GroupImageSpec struct stores an
image ID, the public rbd_group_image_info_t struct insists on an image
name. When rbd_group_image_list() encounters a trashed member image
(i.e. one that doesn't have a name), it just fails with ENOENT and no
listing gets produced at all until the offending image is restored from
trash. Something like this can be very hard to debug for an average
user, so let's make rbd_trash_move() fail with EMLINK the same way as
rbd_remove() does in this scenario.
The one case where moving a member image to trash makes sense is live
migration where the source image gets trashed to be almost immediately
replaced by the destination image as part of preparing migration.
Fixes: https://tracker.ceph.com/issues/71026
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
