Switch all stat panels in the hardware and hardware compression
dashboards from lastNotNull to last reduceFunction. When a host is
unreachable or node-proxy stops, metric silence is ambiguous — it
could mean host down, agent crash, or scrape failure. lastNotNull
freezes the last known value (e.g. 4 NVMe drives), causing
inconsistency with the global Overview which reflects live count
reductions.
Use last so panels show N/A when no current data arrives.
Fixes: https://tracker.ceph.com/issues/78360
Signed-off-by: Afreen Misbah <afreen@ibm.com>
Replication (objects) from Source Zone title is misleading, the panel actually shows How many objects are being replicated per second
Fixes: https://tracker.ceph.com/issues/78230
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
get_dashboards_data() keeps every query in a single map keyed by
"<panel title>-<legendFormat>", but that id is not unique: several
dashboards have a panel titled "IOPS", "OSDs" or "Throughput". The
dashboard read last overwrites the earlier entry, so one query shadows
another and never gets tested. glob() walks the files in filesystem
order, so which query survives, and whether the test passes, depends on
readdir.
run-tox-promql-query-test hit this in "Test IOPS Read"
(ceph-cluster.feature): the scenario feeds ceph_osd_op_r but the id
resolved to a CephFS pool panel.
FAILED:
expr: "sum(rate(ceph_pool_rd{cluster=~"mycluster|", pool_id=~"UNSET VARIABLE"}[1m]))", time: 1m,
exp:"{} 2.5E+01"
got:"nil"
HOOK-ERROR in after_scenario: AssertionError:
pool_id is "UNSET VARIABLE" because the scenario does not set $mdatapool,
and no ceph_pool_rd series were fed, so the query returns nil. It only
fails when readdir returns cephfsdashboard after ceph-cluster-advanced,
so the earlier change that walked nested rows and made the winner
deterministic did not fix it; it fixed which query wins, not that the
wrong one can win.
Key the queries per dashboard (data['queries'][<dashboard>][<id>]) and
have each .feature name its dashboard in a Background step. The lookup is
confined to that dashboard, so a title/legend used elsewhere no longer
shadows it. A duplicate within one dashboard is still an error, unless it
is in a collapsed row.
ceph-cluster.feature covers ceph-cluster-advanced; the rest map to one
dashboard each.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
- export `firmware_version` labels from `node_proxy_storage_capacity_bytes` metrics to be used in device firmware panel
- improve iterations for performance - dropping redundant node_proxy_firmware() RPC; firmware data is already
present in the fullreport response, and removing unnecessary loops.
- replace health if/elif chain with HEALTH_STATUS_MAP dict lookup
- rename metrics to ceph_hardware_*, fix component labels and add tests
- added unit test cases
- add serial name, slot info to capacity metric
- fix health metrics showing ID instead of component name
Signed-off-by: Afreen Misbah <afreen@ibm.com>
- CPU/NVMe temp: change from stat to gauge panels with colored
arc thresholds (green/yellow/red) and proper °C units
- Health panels: wrap queries in max() aggregation to show
single worst-case value instead of overlapping series
- Pie charts: switch to donut style with visible legends
- Fan RPM: use locale unit for comma formatting instead of
short which auto-scaled to "K"
- Fix temperature panels units
- Add Device List and Platform Firmware table panels
Fixes https://tracker.ceph.com/issues/77723
Signed-off-by: Afreen Misbah <afreen@ibm.com>
- Fix fan repeating panels: set multi=true on fan_speeds template
variable so Grafana generates one panel per fan instead of one
- Remove TACH-only regex filter on fan_speeds template and AVG
Cooling query so all system fans are visible regardless of naming
- Replace duplicate Power Control panel with Network health panel
- Fix NVMe drive count to use storage_capacity_bytes{protocol=NVMe}
instead of counting temperature sensors (inaccurate proxy)
- Normalize all hostname filters to regex match (=~) for consistency
- Register hardware.libsonnet in dashboards.libsonnet so the
dashboard JSON is generated during ceph-mixin builds
- Add temperatures category to prometheus health metrics loop
Signed-off-by: Afreen Misbah <afreen@ibm.com>
Assisted-by: Claude
Signed-off-by: Afreen Misbah <afreen@ibm.com>
A run that aborts before the cleanup fixture runs (timeout, OOM,
interrupt) leaves the clone dir behind, so the next build fails with
"destination path already exists". Remove it first.
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
Do not trigger alert NVMeoFMultipleNamespacesOfRBDImage
for same pool/image name used in multiple nvmeof namespaces,
if they are in different rados namespaces (rados_namespace_name)
These are valid repeation of pool/image name:
- mypool/rados_ns1/myimage1
- mypool/rados_ns2/myimage1
- mypool/myimage1 (default rados namespace)
Fixes: https://tracker.ceph.com/issues/77128
Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
- Updated capacity used to show Raw capacity
- Pool table shows Raw capacity
- Total used capacity graph shows raw capacity
fixes: https://tracker.ceph.com/issues/76456
Signed-off-by: Ankush Behl <cloudbehl@gmail.com>
- multiple values were shown in graph and single state
- Remove All selection from subvolume path
fixes: https://tracker.ceph.com/issues/75849
Signed-off-by: Ankush Behl <cloudbehl@gmail.com>
when 1 MDS active and 2 MDS standby (on 3Node-Cluster)
found duplicate series for the match group {fs_id="-1"} on the right hand-side of the operation
many-to-many matching not allowed: matching labels must be unique on one side
Fixes: https://tracker.ceph.com/issues/75754
Signed-off-by: bst2002git <bst2002@ios.dhs.org>
Ceph 18.* adds a `cluster` label to all Prometheus metrics. When
upgrading from earlier releases, historical metrics lack this label
and are excluded by Grafana queries that strictly match on `cluster`.
Update the shared Grafana matcher logic to use a regex matcher that
also matches series without the `cluster` label, restoring visibility
of pre-upgrade metrics while preserving multi-cluster behavior.
Fixes: https://tracker.ceph.com/issues/74342
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
- fixes tox.ini using and undefined env - `grafonnet-check`( instead of `jsonnet-check`)
- adds steps for local development of mixins and building jsonnet
- added help command in Makefile
- added comments and descriptions for Makefile and tox.ini
Signed-off-by: Afreen Misbah <afreen@ibm.com>
We recently added a new CephFS Overview dashboard and removed the
older MDS Performance dashboard. We need to remove the mention
of cephfs.libsonnet from dashboards.libsonnet file as well.
Fixes: https://tracker.ceph.com/issues/73910
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Use this metric to show accurate data for "down"
NVMeoF gateways in ceph NVMeoF Overview dashboard.
Fixes: https://tracker.ceph.com/issues/71384
Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
naming convention
Improve consistency by aligning the naming scheme
across all ceph Grafana dashboards.
Fixes: https://tracker.ceph.com/issues/73296
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
1. Fixes the promql expr used to calculate "In" OSDs in
ceph-cluster-advanced.json.
2. Fixes the color coding for the single state panels used in the OSDs
grafana panel like "In", "Out" etc
Fixes: https://tracker.ceph.com/issues/72810
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
