Merge pull request #64818 from avanthakkar/smb-rate-limit

mgr/smb: add rate limiting support

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
This commit is contained in:
John Mulligan 2026-02-24 10:12:01 -05:00 committed by GitHub
commit 530d9dc8a1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
13 changed files with 915 additions and 49 deletions

View File

@ -261,6 +261,67 @@ Create a read-only share at a custom path in the CephFS volume:
ceph smb share create test1 plans cephfs \
--path=/qbranch/top/secret/plans --readonly
Update Share QoS
++++++++++++++++
.. prompt:: bash #
ceph smb share update cephfs qos <cluster_id> <share_id> [--read-iops-limit=<int>] [--write-iops-limit=<int>] [--read-bw-limit=<int>] [--write-bw-limit=<int>] [--read-delay-max=<int>] [--write-delay-max=<int>]
Update Quality of Service (QoS) settings for a CephFS-backed share. This allows administrators to apply per-share rate limits on SMB input/output (I/O) operations, specifically limits on IOPS (Input/Output Operations per Second) and bandwidth (in bytes per second) for both read and write operations.
Options:
read_iops_limit
Optional integer. Maximum number of read operations per second (0 = disabled).
Valid range: ``0`` to ``1,000,000``. Values above this will be capped.
write_iops_limit
Optional integer. Maximum number of write operations per second (0 = disabled).
Valid range: ``0`` to ``1,000,000``. Values above this will be capped.
read_bw_limit
Optional integer. Maximum allowed bandwidth for read operations in bytes per second (0 = disabled).
Valid range: ``0`` to ``1 << 40`` (≈1 TB/s). Values above this will be capped.
write_bw_limit
Optional integer. Maximum allowed bandwidth for write operations in bytes per second (0 = disabled).
Valid range: ``0`` to ``1 << 40`` (≈1 TB/s). Values above this will be capped.
read_delay_max
Optional integer. Maximum allowed delay value in seconds for read operations (default: 30, max: 300).
This acts as an upper bound on how long a read request will be delayed if the defined IOPS or bandwidth limits are exceeded.
write_delay_max
Optional integer. Maximum allowed delay value in seconds for write operations (default: 30, max: 300).
Similar to `read_delay_max`, but applies to write requests.
Behavior:
- All limits are optional
- Setting a limit to ``0`` disables that specific QoS limit
- Setting all four limits to ``0`` completely removes QoS configuration
Examples:
Set QoS limits for a share:
.. prompt:: bash #
ceph smb share update cephfs qos foo bar \
--read-iops-limit=100 \
--write-iops-limit=200 \
--read-bw-limit=1048576 \
--write-bw-limit=2097152 \
--read-delay-max=5 \
--write-delay-max=5
Disable QoS for a share:
.. prompt:: bash #
ceph smb share update cephfs qos foo bar \
--read-iops-limit=0 \
--write-iops-limit=0 \
--read-bw-limit=0 \
--write-bw-limit=0 \
--read-delay-max=0 \
--write-delay-max=0
Remove Share
@ -804,6 +865,27 @@ cephfs
based implementation, currently ``samba-vfs/proxied``. This option is
suitable for the majority of use cases and can be left unspecified for most
shares.
qos
Optional object. Quality of Service settings for the share. Fields:
read_iops_limit
Optional integer. Maximum number of read operations per second (0 = disabled).
Valid range: ``0`` to ``1,000,000``. Values above this will be capped.
write_iops_limit
Optional integer. Maximum number of write operations per second (0 = disabled).
Valid range: ``0`` to ``1,000,000``. Values above this will be capped.
read_bw_limit
Optional integer. Maximum allowed bandwidth for read operations in bytes per second (0 = disabled).
Valid range: ``0`` to ``1 << 40`` (≈1 TB/s). Values above this will be capped.
write_bw_limit
Optional integer. Maximum allowed bandwidth for write operations in bytes per second (0 = disabled).
Valid range: ``0`` to ``1 << 40`` (≈1 TB/s). Values above this will be capped.
read_delay_max
Optional integer. Maximum allowed delay value in seconds for read operations (default: 30, max: 300).
This acts as an upper bound on how long a read request will be delayed if the defined IOPS or bandwidth limits are exceeded.
write_delay_max
Optional integer. Maximum allowed delay value in seconds for write operations (default: 30, max: 300).
Similar to `read_delay_max`, but applies to write requests.
restrict_access
Optional boolean, defaulting to false. If true the share will only permit
access by users explicitly listed in ``login_control``.
@ -851,7 +933,7 @@ custom_smb_share_options
things in ways that the Ceph team can not help with. This special key will
automatically be removed from the list of options passed to Samba.
The following is an example of a share:
The following is an example of a share with QoS settings:
.. code-block:: yaml
@ -864,9 +946,34 @@ The following is an example of a share:
path: /pics
subvolumegroup: smbshares
subvolume: staff
qos:
read_iops_limit: 100 # Max 100 read operations per second
write_iops_limit: 50 # Max 50 write operations per second
read_bw_limit: 1048576 # 1 MiB/s read bandwidth limit
write_bw_limit: 524288 # 512 KiB/s write bandwidth limit
read_delay_max: 5 # Max 5 seconds delay for read operations
write_delay_max: 5 # Max 5 seconds delay for write operations
Another example, this time of a share with an intent to be removed:
Another example, this time of a share with QoS disabled:
.. code-block:: yaml
resource_type: ceph.smb.share
cluster_id: tango
share_id: sp2
cephfs:
volume: cephfs
path: /data
qos:
read_iops_limit: 0
write_iops_limit: 0
read_bw_limit: 0
write_bw_limit: 0
read_delay_max: 0
write_delay_max: 0
And finally, a share with an intent to be removed:
.. code-block:: yaml

View File

@ -93,7 +93,7 @@ tasks:
timeout: 1h
clients:
client.0:
- [default, hosts_access]
- [default, hosts_access, rate_limiting]
- cephadm.shell:
host.a:

View File

@ -91,7 +91,7 @@ tasks:
timeout: 1h
clients:
client.0:
- [default, hosts_access]
- [default, hosts_access, rate_limiting]
- cephadm.shell:
host.a:

View File

@ -4,3 +4,4 @@ addopts = -m 'default'
markers =
default: Default tests
hosts_access: Host access tests
rate_limiting: Rate limit tests

View File

@ -1,6 +1,9 @@
import base64
import contextlib
import pathlib
import time
import cephutil
import smbclient
from smbprotocol.header import NtStatus
@ -154,6 +157,57 @@ class PathWrapper:
with self.open(mode='w') as fh:
fh.write(txt)
def write_bytes(self, data):
"""Open the file in binary mode, write bytes to it, and close the file."""
with self.open(mode='wb') as fh:
fh.write(data)
def unlink(self):
"""Unlink (remove) a file."""
smbclient.remove(str(self.share_path))
def get_shares(smb_cfg):
"""Get all SMB shares."""
jres = cephutil.cephadm_shell_cmd(
smb_cfg,
["ceph", "smb", "show", "ceph.smb.share"],
load_json=True,
)
assert jres.obj
resources = jres.obj['resources']
assert len(resources) > 0
assert all(r['resource_type'] == 'ceph.smb.share' for r in resources)
return resources
def get_share_by_id(smb_cfg, cluster_id, share_id):
"""Get a specific share by cluster_id and share_id."""
shares = get_shares(smb_cfg)
for share in shares:
if share['cluster_id'] == cluster_id and share['share_id'] == share_id:
return share
return None
def apply_share_config(smb_cfg, share):
"""Apply share configuration via the apply command."""
jres = cephutil.cephadm_shell_cmd(
smb_cfg,
['ceph', 'smb', 'apply', '-i-'],
input_json={'resources': [share]},
load_json=True,
)
assert jres.returncode == 0
assert jres.obj and jres.obj.get('success')
assert 'results' in jres.obj
_results = jres.obj['results']
assert len(_results) == 1, "more than one result found"
_result = _results[0]
assert 'resource' in _result
resources_ret = _result['resource']
assert resources_ret['resource_type'] == 'ceph.smb.share'
# sleep to ensure the settings got applied in smbd
# TODO: make this more dynamic somehow
time.sleep(60)
return resources_ret

View File

@ -1,48 +1,11 @@
import copy
import time
import pytest
import smbprotocol
import cephutil
import smbutil
def _get_shares(smb_cfg):
jres = cephutil.cephadm_shell_cmd(
smb_cfg,
["ceph", "smb", "show", "ceph.smb.share"],
load_json=True,
)
assert jres.obj
resources = jres.obj['resources']
assert len(resources) > 0
assert all(r['resource_type'] == 'ceph.smb.share' for r in resources)
return resources
def _apply(smb_cfg, share):
jres = cephutil.cephadm_shell_cmd(
smb_cfg,
['ceph', 'smb', 'apply', '-i-'],
input_json={'resources': [share]},
load_json=True,
)
assert jres.returncode == 0
assert jres.obj and jres.obj.get('success')
assert 'results' in jres.obj
_results = jres.obj['results']
assert len(_results) == 1, "more then one result found"
_result = _results[0]
assert 'resource' in _result
resources_ret = _result['resource']
assert resources_ret['resource_type'] == 'ceph.smb.share'
# sleep to ensure the settings got applied in smbd
# TODO: make this more dynamic somehow
time.sleep(60)
return resources_ret
# BOGUS is an IP that should never be assigned to a test node running in
# teuthology (or in general)
BOGUS = '192.0.2.222'
@ -56,7 +19,7 @@ class TestHostsAccessToggle1:
@pytest.fixture(scope='class')
def config(self, smb_cfg):
filename = 'TestHostAcess1.txt'
orig = _get_shares(smb_cfg)[0]
orig = smbutil.get_shares(smb_cfg)[0]
share_name = orig['name']
print('Testing original share configuration...')
@ -67,7 +30,7 @@ class TestHostsAccessToggle1:
yield (filename, orig)
print('Restoring original share configuration...')
_apply(smb_cfg, orig)
smbutil.apply_share_config(smb_cfg, orig)
# With the IP restriction removed, access should succeed and we can
# clean up our test file
with smbutil.connection(smb_cfg, share_name) as sharep:
@ -91,7 +54,7 @@ class TestHostsAccessToggle1:
mod_share['hosts_access'] = [
{'access': 'allow', 'address': BOGUS},
]
applied = _apply(smb_cfg, mod_share)
applied = smbutil.apply_share_config(smb_cfg, mod_share)
assert applied['share_id'] == mod_share['share_id']
assert applied['hosts_access'] == mod_share['hosts_access']
@ -106,7 +69,7 @@ class TestHostsAccessToggle1:
mod_share['hosts_access'] = [
{'access': 'deny', 'address': BOGUS},
]
applied = _apply(smb_cfg, mod_share)
applied = smbutil.apply_share_config(smb_cfg, mod_share)
assert applied['share_id'] == mod_share['share_id']
assert applied['hosts_access'] == mod_share['hosts_access']
@ -121,7 +84,7 @@ class TestHostsAccessToggle1:
{'access': 'allow', 'address': BOGUS},
{'access': 'allow', 'address': smb_cfg.default_client.ip_address},
]
applied = _apply(smb_cfg, mod_share)
applied = smbutil.apply_share_config(smb_cfg, mod_share)
assert applied['share_id'] == mod_share['share_id']
assert applied['hosts_access'] == mod_share['hosts_access']
@ -135,7 +98,7 @@ class TestHostsAccessToggle1:
mod_share['hosts_access'] = [
{'access': 'deny', 'address': smb_cfg.default_client.ip_address},
]
applied = _apply(smb_cfg, mod_share)
applied = smbutil.apply_share_config(smb_cfg, mod_share)
assert applied['share_id'] == mod_share['share_id']
assert applied['hosts_access'] == mod_share['hosts_access']
@ -150,7 +113,7 @@ class TestHostsAccessToggle1:
mod_share['hosts_access'] = [
{'access': 'allow', 'network': BOGUS_NET},
]
applied = _apply(smb_cfg, mod_share)
applied = smbutil.apply_share_config(smb_cfg, mod_share)
assert applied['share_id'] == mod_share['share_id']
assert applied['hosts_access'] == mod_share['hosts_access']

View File

@ -0,0 +1,291 @@
import time
import pytest
import cephutil
import smbutil
def _update_qos(smb_cfg, cluster_id, share_id, **qos_params):
"""Update QoS settings for a share using the CLI command."""
cmd = [
"ceph",
"smb",
"share",
"update",
"cephfs",
"qos",
"--cluster-id",
cluster_id,
"--share-id",
share_id,
]
for param, value in qos_params.items():
if value is not None:
cmd.extend([f"--{param.replace('_', '-')}", str(value)])
jres = cephutil.cephadm_shell_cmd(
smb_cfg,
cmd,
load_json=True,
)
assert (
jres.returncode == 0
), f"Command failed with return code {jres.returncode}"
assert jres.obj, "No JSON response from command"
assert jres.obj.get("success"), f"Command not successful: {jres.obj}"
assert "resource" in jres.obj, f"Response missing 'resource' key: {jres.obj}"
time.sleep(60)
return jres.obj["resource"]
@pytest.mark.rate_limiting
class TestSMBRateLimiting:
@pytest.fixture(scope="class")
def config(self, smb_cfg):
"""Setup initial configuration with a test file."""
orig = smbutil.get_shares(smb_cfg)[0]
share_name = orig["name"]
cluster_id = orig["cluster_id"]
share_id = orig["share_id"]
original_qos = None
if "cephfs" in orig and "qos" in orig["cephfs"]:
original_qos = orig["cephfs"]["qos"]
test_filename = "test_ratelimit_data.bin"
with smbutil.connection(smb_cfg, share_name) as sharep:
test_data = b"X" * (1 * 1024 * 1024)
file_path = sharep / test_filename
file_path.write_bytes(test_data)
yield {
"share_name": share_name,
"cluster_id": cluster_id,
"share_id": share_id,
"test_filename": test_filename,
"original_share": orig,
"original_qos": original_qos,
}
smbutil.apply_share_config(smb_cfg, orig)
with smbutil.connection(smb_cfg, share_name) as sharep:
(sharep / test_filename).unlink()
def test_qos_read_iops_limit(self, smb_cfg, config):
"""Test read IOPS rate limiting."""
updated_share = _update_qos(
smb_cfg, config["cluster_id"], config["share_id"], read_iops_limit=100
)
assert updated_share is not None
assert updated_share["cluster_id"] == config["cluster_id"]
assert updated_share["share_id"] == config["share_id"]
assert "cephfs" in updated_share
assert "qos" in updated_share["cephfs"]
assert updated_share["cephfs"]["qos"]["read_iops_limit"] == 100
show_share = smbutil.get_share_by_id(
smb_cfg, config["cluster_id"], config["share_id"]
)
assert show_share["cephfs"]["qos"]["read_iops_limit"] == 100
def test_qos_write_iops_limit(self, smb_cfg, config):
"""Test write IOPS rate limiting."""
updated_share = _update_qos(
smb_cfg, config["cluster_id"], config["share_id"], write_iops_limit=50
)
assert updated_share is not None
assert updated_share["cephfs"]["qos"]["write_iops_limit"] == 50
show_share = smbutil.get_share_by_id(
smb_cfg, config["cluster_id"], config["share_id"]
)
assert show_share["cephfs"]["qos"]["write_iops_limit"] == 50
def test_qos_read_bandwidth_limit(self, smb_cfg, config):
"""Test read bandwidth rate limiting."""
read_bw_limit = 1048576
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_bw_limit=read_bw_limit,
)
assert updated_share["cephfs"]["qos"]["read_bw_limit"] == read_bw_limit
show_share = smbutil.get_share_by_id(
smb_cfg, config["cluster_id"], config["share_id"]
)
assert show_share["cephfs"]["qos"]["read_bw_limit"] == read_bw_limit
def test_qos_write_bandwidth_limit(self, smb_cfg, config):
"""Test write bandwidth rate limiting."""
write_bw_limit = 2097152
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
write_bw_limit=write_bw_limit,
)
assert updated_share["cephfs"]["qos"]["write_bw_limit"] == write_bw_limit
show_share = smbutil.get_share_by_id(
smb_cfg, config["cluster_id"], config["share_id"]
)
assert show_share["cephfs"]["qos"]["write_bw_limit"] == write_bw_limit
def test_qos_delay_max(self, smb_cfg, config):
"""Test delay_max settings."""
read_delay_max = 100
write_delay_max = 150
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_delay_max=read_delay_max,
write_delay_max=write_delay_max,
)
qos = updated_share["cephfs"]["qos"]
assert qos["read_delay_max"] == read_delay_max
assert qos["write_delay_max"] == write_delay_max
show_share = smbutil.get_share_by_id(
smb_cfg, config["cluster_id"], config["share_id"]
)
assert show_share["cephfs"]["qos"]["read_delay_max"] == read_delay_max
assert show_share["cephfs"]["qos"]["write_delay_max"] == write_delay_max
def test_qos_multiple_limits(self, smb_cfg, config):
"""Test applying multiple QoS limits simultaneously."""
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_iops_limit=100,
write_iops_limit=50,
read_bw_limit=4194304,
write_bw_limit=2097152,
read_delay_max=100,
write_delay_max=150,
)
qos = updated_share["cephfs"]["qos"]
assert qos["read_iops_limit"] == 100
assert qos["write_iops_limit"] == 50
assert qos["read_bw_limit"] == 4194304
assert qos["write_bw_limit"] == 2097152
assert qos["read_delay_max"] == 100
assert qos["write_delay_max"] == 150
def test_qos_update_existing(self, smb_cfg, config):
"""Test updating existing QoS settings."""
_update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_iops_limit=100,
read_bw_limit=1048576,
)
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_iops_limit=200,
write_iops_limit=100,
read_bw_limit=2097152,
)
qos = updated_share["cephfs"]["qos"]
assert qos["read_iops_limit"] == 200
assert qos["write_iops_limit"] == 100
assert qos["read_bw_limit"] == 2097152
def test_qos_limits_clamping(self, smb_cfg, config):
"""Test that QoS limits are properly clamped to maximum values."""
excessive_iops = 2_000_000 # Above IOPS_LIMIT_MAX = 1,000,000
excessive_bw = 2 << 40 # Above BYTES_LIMIT_MAX = 1 << 40 (1 TB)
excessive_delay = 500 # Above DELAY_MAX_LIMIT = 300
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_iops_limit=excessive_iops,
write_iops_limit=excessive_iops,
read_bw_limit=excessive_bw,
write_bw_limit=excessive_bw,
read_delay_max=excessive_delay,
write_delay_max=excessive_delay,
)
qos = updated_share["cephfs"]["qos"]
assert qos["read_iops_limit"] == 1_000_000 # IOPS_LIMIT_MAX
assert qos["write_iops_limit"] == 1_000_000 # IOPS_LIMIT_MAX
assert qos["read_bw_limit"] == 1 << 40 # BYTES_LIMIT_MAX (1 TB)
assert qos["write_bw_limit"] == 1 << 40 # BYTES_LIMIT_MAX (1 TB)
assert qos["read_delay_max"] == 300 # DELAY_MAX_LIMIT
assert qos["write_delay_max"] == 300 # DELAY_MAX_LIMIT
def test_qos_zero_values(self, smb_cfg, config):
"""Test that zero values are handled correctly (should be treated as no limit)."""
updated_share = _update_qos(
smb_cfg,
config["cluster_id"],
config["share_id"],
read_iops_limit=0,
write_iops_limit=0,
read_bw_limit=0,
write_bw_limit=0,
read_delay_max=0,
write_delay_max=0,
)
assert "qos" not in updated_share["cephfs"]
def test_qos_apply_via_resources(self, smb_cfg, config):
"""Test applying QoS settings via the apply command with resources JSON."""
current_share = smbutil.get_share_by_id(
smb_cfg, config["cluster_id"], config["share_id"]
)
assert (
current_share is not None
), f"Share {config['cluster_id']}/{config['share_id']} not found"
share_resource = current_share.copy()
assert "cephfs" in share_resource, "Share resource missing 'cephfs' key"
share_resource["cephfs"]["qos"] = {
"read_iops_limit": 300,
"write_iops_limit": 150,
"read_bw_limit": 3145728, # 3 MB/s
"write_bw_limit": 1572864, # 1.5 MB/s
"read_delay_max": 50,
"write_delay_max": 75,
}
updated_share = smbutil.apply_share_config(smb_cfg, share_resource)
assert updated_share is not None
assert "cephfs" in updated_share
assert "qos" in updated_share["cephfs"]
qos = updated_share["cephfs"]["qos"]
assert qos["read_iops_limit"] == 300
assert qos["write_iops_limit"] == 150
assert qos["read_bw_limit"] == 3145728
assert qos["write_bw_limit"] == 1572864
assert qos["read_delay_max"] == 50
assert qos["write_delay_max"] == 75

View File

@ -814,11 +814,19 @@ def _generate_share(conf: _ShareConf) -> Dict[str, Dict[str, str]]:
if conf.ceph_cluster
else '/etc/ceph/ceph.conf'
)
modules = ["acl_xattr", "ceph_snapshots"]
if qos := cephfs.qos:
vfs_rl = "aio_ratelimit"
modules.extend([vfs_rl, ceph_vfs])
else:
modules.append(ceph_vfs)
cfg = {
# smb.conf options
'options': {
'path': path,
"vfs objects": f"acl_xattr ceph_snapshots {ceph_vfs}",
"vfs objects": " ".join(modules),
'acl_xattr:security_acl_name': 'user.NTACL',
f'{ceph_vfs}:config_file': ceph_config_file,
f'{ceph_vfs}:filesystem': cephfs.volume,
@ -831,6 +839,19 @@ def _generate_share(conf: _ShareConf) -> Dict[str, Dict[str, str]]:
'posix locking': 'no',
}
}
if qos:
opts = cfg["options"]
for field in (
"read_iops_limit",
"read_bw_limit",
"read_delay_max",
"write_iops_limit",
"write_bw_limit",
"write_delay_max",
):
if value := getattr(qos, field):
opts[f"{vfs_rl}:{field}"] = str(value)
if share.comment is not None:
cfg['options']['comment'] = share.comment

View File

@ -1,6 +1,7 @@
from typing import TYPE_CHECKING, Any, List, Optional, cast
import logging
from dataclasses import replace
import orchestrator
from ceph.deployment.service_spec import PlacementSpec, SMBSpec
@ -388,6 +389,45 @@ class Module(orchestrator.OrchestratorClientMixin, MgrModule):
)
return self._apply_res([share]).one()
@SMBCLICommand('share update cephfs qos', perm='rw')
def share_update_qos(
self,
cluster_id: str,
share_id: str,
read_iops_limit: Optional[int] = None,
write_iops_limit: Optional[int] = None,
read_bw_limit: Optional[int] = None,
write_bw_limit: Optional[int] = None,
read_delay_max: Optional[int] = 30,
write_delay_max: Optional[int] = 30,
) -> results.Result:
"""Update QoS settings for a CephFS share"""
try:
shares = self._handler.matching_resources(
[f'ceph.smb.share.{cluster_id}.{share_id}']
)
if not shares or not isinstance(shares[0], resources.Share):
raise ValueError(f"Share {cluster_id}/{share_id} not found")
share = shares[0]
if not share.cephfs:
raise ValueError("Share has no CephFS configuration")
updated_cephfs = share.cephfs.update_qos(
read_iops_limit=read_iops_limit,
write_iops_limit=write_iops_limit,
read_bw_limit=read_bw_limit,
write_bw_limit=write_bw_limit,
read_delay_max=read_delay_max,
write_delay_max=write_delay_max,
)
updated_share = replace(share, cephfs=updated_cephfs)
return self._apply_res([updated_share]).one()
except resources.InvalidResourceError as err:
return results.InvalidResourceResult(err.resource_data, str(err))
@SMBCLICommand("show", perm="r")
def show(
self,

View File

@ -4,6 +4,7 @@ import base64
import dataclasses
import errno
import json
from dataclasses import replace
import yaml
@ -133,6 +134,18 @@ class _RBase:
return self
@resourcelib.component()
class QoSConfig(_RBase):
"""Quality of Service configuration for CephFS shares."""
read_iops_limit: Optional[int] = None
write_iops_limit: Optional[int] = None
read_bw_limit: Optional[int] = None
write_bw_limit: Optional[int] = None
read_delay_max: Optional[int] = 30
write_delay_max: Optional[int] = 30
@resourcelib.component()
class CephFSStorage(_RBase):
"""Description of where in a CephFS file system a share is located."""
@ -142,6 +155,12 @@ class CephFSStorage(_RBase):
subvolumegroup: str = ''
subvolume: str = ''
provider: CephFSStorageProvider = CephFSStorageProvider.SAMBA_VFS
qos: Optional[QoSConfig] = None
DELAY_MAX_LIMIT = 300
# Maximal value for iops_limit
IOPS_LIMIT_MAX = 1_000_000
# Maximal value for bw_limit (1 << 40 = 1 TB)
BYTES_LIMIT_MAX = 1 << 40
def __post_init__(self) -> None:
# Allow a shortcut form of <subvolgroup>/<subvol> in the subvolume
@ -175,8 +194,53 @@ class CephFSStorage(_RBase):
def _customize_resource(rc: resourcelib.Resource) -> resourcelib.Resource:
rc.subvolumegroup.quiet = True
rc.subvolume.quiet = True
rc.qos.quiet = True
return rc
def update_qos(
self,
*,
read_iops_limit: Optional[int] = None,
write_iops_limit: Optional[int] = None,
read_bw_limit: Optional[int] = None,
write_bw_limit: Optional[int] = None,
read_delay_max: Optional[int] = 30,
write_delay_max: Optional[int] = 30,
) -> Self:
"""Return a new CephFSStorage instance with updated QoS values."""
qos_updates = {}
new_qos: Optional[QoSConfig] = None
if read_iops_limit is not None and read_iops_limit > 0:
qos_updates["read_iops_limit"] = min(
read_iops_limit, self.IOPS_LIMIT_MAX
)
if write_iops_limit is not None and write_iops_limit > 0:
qos_updates["write_iops_limit"] = min(
write_iops_limit, self.IOPS_LIMIT_MAX
)
if read_bw_limit is not None and read_bw_limit > 0:
qos_updates["read_bw_limit"] = min(
read_bw_limit, self.BYTES_LIMIT_MAX
)
if write_bw_limit is not None and write_bw_limit > 0:
qos_updates["write_bw_limit"] = min(
write_bw_limit, self.BYTES_LIMIT_MAX
)
if read_delay_max is not None and read_delay_max > 0:
qos_updates["read_delay_max"] = min(
read_delay_max, self.DELAY_MAX_LIMIT
)
if write_delay_max is not None and write_delay_max > 0:
qos_updates["write_delay_max"] = min(
write_delay_max, self.DELAY_MAX_LIMIT
)
if qos_updates:
new_qos = replace(self.qos or QoSConfig(), **qos_updates)
return replace(self, qos=new_qos)
@resourcelib.component()
class LoginAccessEntry(_RBase):

View File

@ -1772,3 +1772,45 @@ def test_share_name_in_use(thandler, params):
assert not results.success
assert params['error_msg'] in rs['results'][0]['msg']
assert rs['results'][0]['conflicting_share_id'] in params['conflicts']
def test_apply_share_with_qos(thandler):
cluster = _cluster(
cluster_id='qoscluster',
auth_mode=smb.enums.AuthMode.USER,
user_group_settings=[
smb.resources.UserGroupSource(
source_type=smb.resources.UserGroupSourceType.EMPTY,
),
],
)
share = smb.resources.Share(
cluster_id='qoscluster',
share_id='qostest',
name='QoS Test Share',
cephfs=_cephfs(
volume='cephfs',
path='/',
qos=smb.resources.QoSConfig(
read_iops_limit=100,
write_iops_limit=200,
read_bw_limit=1048576,
write_bw_limit=2097152,
read_delay_max=20,
write_delay_max=30,
),
),
)
rg = thandler.apply([cluster, share])
assert rg.success, rg.to_simplified()
# Verify QoS settings were stored
share_dict = thandler.internal_store.data[
('shares', 'qoscluster.qostest')
]
assert share_dict['cephfs']['qos']['read_iops_limit'] == 100
assert share_dict['cephfs']['qos']['write_iops_limit'] == 200
assert share_dict['cephfs']['qos']['read_bw_limit'] == 1048576
assert share_dict['cephfs']['qos']['write_bw_limit'] == 2097152
assert share_dict['cephfs']['qos']['read_delay_max'] == 20
assert share_dict['cephfs']['qos']['write_delay_max'] == 30

View File

@ -955,3 +955,185 @@ comment: "Invalid\\nComment"
data = yaml.safe_load_all(yaml_str)
with pytest.raises(ValueError, match="Comment cannot contain newlines"):
smb.resources.load(data)
def test_share_with_qos():
import yaml
yaml_str = """
resource_type: ceph.smb.share
cluster_id: qoscluster
share_id: qostest
name: QoS Test Share
cephfs:
volume: myvol
path: /qos
qos:
read_iops_limit: 100
write_iops_limit: 200
read_bw_limit: 1048576
write_bw_limit: 2097152
read_delay_max: 20
write_delay_max: 30
"""
data = yaml.safe_load_all(yaml_str)
loaded = smb.resources.load(data)
assert loaded
share = loaded[0]
assert share.cephfs.qos is not None
assert share.cephfs.qos.read_iops_limit == 100
assert share.cephfs.qos.write_iops_limit == 200
assert share.cephfs.qos.read_bw_limit == 1048576
assert share.cephfs.qos.write_bw_limit == 2097152
assert share.cephfs.qos.read_delay_max == 20
assert share.cephfs.qos.write_delay_max == 30
def test_share_update_qos():
share = smb.resources.Share(
cluster_id='qoscluster',
share_id='qostest',
name='QoS Test Share',
cephfs=smb.resources.CephFSStorage(
volume='myvol',
path='/qos',
qos=smb.resources.QoSConfig(
read_iops_limit=100,
write_iops_limit=200,
read_delay_max=5,
write_delay_max=30,
),
),
)
# Update with new QoS values
updated_cephfs = share.cephfs.update_qos(
read_bw_limit=1048576,
write_bw_limit=2097152,
read_iops_limit=300,
read_delay_max=15,
)
assert updated_cephfs.qos is not None
assert updated_cephfs.qos.read_iops_limit == 300 # new value
assert updated_cephfs.qos.write_iops_limit == 200 # preserved original
assert updated_cephfs.qos.read_bw_limit == 1048576 # new value
assert updated_cephfs.qos.write_bw_limit == 2097152 # new value
assert updated_cephfs.qos.read_delay_max == 15 # new value
assert updated_cephfs.qos.write_delay_max == 30 # preserved original
# Verify share with updated QoS works
data = share.to_simplified()
data.pop("resource_type", None)
updated_share = smb.resources.Share(**{**data, 'cephfs': updated_cephfs})
assert updated_share.cephfs.qos.read_bw_limit == 1048576
assert updated_share.cephfs.qos.read_delay_max == 15
def test_share_qos_remove():
share = smb.resources.Share(
cluster_id='qoscluster',
share_id='qostest',
name='QoS Test Share',
cephfs=smb.resources.CephFSStorage(
volume='myvol',
path='/qos',
qos=smb.resources.QoSConfig(
read_iops_limit=100,
write_iops_limit=200,
read_delay_max=5,
write_delay_max=30,
),
),
)
# Disable QoS by setting all limits to 0
updated_cephfs = share.cephfs.update_qos(
read_iops_limit=0,
write_iops_limit=0,
read_bw_limit=0,
write_bw_limit=0,
read_delay_max=0,
write_delay_max=0,
)
# Verify QoS is completely removed
assert updated_cephfs.qos is None
def test_share_qos_default_delay():
"""Test that delay_max defaults to 30 when not specified"""
share = smb.resources.Share(
cluster_id='qoscluster',
share_id='qostest',
name='QoS Test Share',
cephfs=smb.resources.CephFSStorage(
volume='myvol',
path='/qos',
qos=smb.resources.QoSConfig(
read_iops_limit=100,
write_iops_limit=200
# delay_max not specified - should use defaults
),
),
)
assert share.cephfs.qos is not None
assert share.cephfs.qos.read_delay_max == 30 # Default value
assert share.cephfs.qos.write_delay_max == 30 # Default value
def test_share_qos_max_allowed_delay():
"""Test that delay_max values exceeding 300 will be capped to 300"""
share = smb.resources.Share(
cluster_id='qoscluster',
share_id='qostest',
name='QoS Test Share',
cephfs=smb.resources.CephFSStorage(
volume='myvol',
path='/qos',
qos=smb.resources.QoSConfig(
read_iops_limit=100,
write_iops_limit=200,
read_delay_max=30,
write_delay_max=30,
),
),
)
updated_cephfs = share.cephfs.update_qos(read_delay_max=350)
assert updated_cephfs.qos is not None
assert updated_cephfs.qos.read_delay_max == 300 # Capped value
def test_share_qos_max_allowed_iops_and_bandwidth():
"""Test that iops and bandwidth values exceeding limits will be capped to IOPS_LIMIT_MAX and BYTES_LIMIT_MAX"""
IOPS_LIMIT_MAX = 1_000_000
BYTES_LIMIT_MAX = 1 << 40 # 1 TB
share = smb.resources.Share(
cluster_id="qoscluster",
share_id="qostest",
name="QoS Test Share",
cephfs=smb.resources.CephFSStorage(
volume="myvol",
path="/qos",
qos=smb.resources.QoSConfig(
read_iops_limit=100,
write_iops_limit=200,
read_delay_max=30,
write_delay_max=30,
),
),
)
updated_cephfs = share.cephfs.update_qos(
read_iops_limit=1_500_000_000, # way above limit
write_bw_limit=2_000_000_000_000, # ~2 TB, way above limit
)
assert updated_cephfs.qos is not None
assert updated_cephfs.qos.read_iops_limit == IOPS_LIMIT_MAX # capped
assert updated_cephfs.qos.write_bw_limit == BYTES_LIMIT_MAX # capped

View File

@ -929,3 +929,104 @@ def test_tls_credential_yaml_show(tmodule):
assert res == 0
body = body.strip()
assert 'value: |' in body
def test_cmd_share_update_qos(tmodule):
cluster = _cluster(
cluster_id='qoscluster',
auth_mode=smb.enums.AuthMode.USER,
user_group_settings=[
smb.resources.UserGroupSource(
source_type=smb.resources.UserGroupSourceType.EMPTY,
),
],
)
share = smb.resources.Share(
cluster_id='qoscluster',
share_id='qostest',
name='QoS Test Share',
cephfs=smb.resources.CephFSStorage(
volume='cephfs',
path='/',
),
)
rg = tmodule._handler.apply([cluster, share])
assert rg.success, rg.to_simplified()
# Test updating with positive values
res, body, status = tmodule.share_update_qos.command(
cluster_id='qoscluster',
share_id='qostest',
read_iops_limit=100,
write_iops_limit=200,
read_bw_limit=1048576,
write_bw_limit=2097152,
read_delay_max=20,
write_delay_max=30,
)
assert res == 0
bdata = json.loads(body)
assert bdata['success']
assert bdata['state'] == 'updated'
# Verify the QoS settings were updated
updated_shares = tmodule._handler.matching_resources(
['ceph.smb.share.qoscluster.qostest']
)
assert len(updated_shares) == 1
updated_share = updated_shares[0]
assert updated_share.cephfs.qos is not None
assert updated_share.cephfs.qos.read_iops_limit == 100
assert updated_share.cephfs.qos.write_iops_limit == 200
assert updated_share.cephfs.qos.read_bw_limit == 1048576
assert updated_share.cephfs.qos.write_bw_limit == 2097152
assert updated_share.cephfs.qos.read_delay_max == 20
assert updated_share.cephfs.qos.write_delay_max == 30
# Test updating with None values (should remove QoS)
res, body, status = tmodule.share_update_qos.command(
cluster_id='qoscluster',
share_id='qostest',
read_iops_limit=0,
write_iops_limit=0,
read_bw_limit=0,
write_bw_limit=0,
read_delay_max=0,
write_delay_max=0,
)
assert res == 0
bdata = json.loads(body)
assert bdata['success']
assert bdata['state'] == 'updated'
# Verify QoS was removed
updated_shares = tmodule._handler.matching_resources(
['ceph.smb.share.qoscluster.qostest']
)
updated_share = updated_shares[0]
assert updated_share.cephfs.qos is None
# Test updating with some values and keeping others
res, body, status = tmodule.share_update_qos.command(
cluster_id='qoscluster',
share_id='qostest',
read_iops_limit=500,
write_bw_limit=524288,
)
assert res == 0
bdata = json.loads(body)
assert bdata['success']
assert bdata['state'] == 'updated'
# Verify partial update
updated_shares = tmodule._handler.matching_resources(
['ceph.smb.share.qoscluster.qostest']
)
updated_share = updated_shares[0]
assert updated_share.cephfs.qos is not None
assert updated_share.cephfs.qos.read_iops_limit == 500
assert updated_share.cephfs.qos.write_iops_limit is None
assert updated_share.cephfs.qos.read_bw_limit is None
assert updated_share.cephfs.qos.write_bw_limit == 524288
assert updated_share.cephfs.qos.read_delay_max == 30
assert updated_share.cephfs.qos.write_delay_max == 30