gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-02 02:00:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,258 Commits 48 Branches 52 Tags 378 MiB
blobstore-develop
Commit Graph

6258 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
zhangjianwei2
d5b0b72e81 fix(blobnode): only parse files with the .iostat suffix for blobnode iostat 
Problem: error: invalid iostat file
RootCause: there are unexpected files from other processes
  in directories /tmp/shm or /dev/shm

. #1001088459

Signed-off-by: zhangjianwei2 <zhangjianwei2@cmss.chinamobile.com>
 2026-07-27 15:12:47 +08:00
slasher
6d424e5ef5 chore(access): to avoid panic when decoding location 
. #1001088459

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-07-24 10:32:39 +08:00
zhumingze
d0493e7541 fix(blobstore): Fixed some conditional logic related to access. 
.#1000824491

Signed-off-by: zhumingze <zhumingze@oppo.com>
 2026-07-21 16:48:19 +08:00
JasonHu520
04ada63766 fix(clustermgr): if code mode enable, create 1 volume at least 
with: #1001162036
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-07-15 19:26:19 +08:00
xiejian
18c6e744dc perf(shardnode): report shard meta stats from leader only 
with: #1001138628

Signed-off-by: xiejian <xiejian3@oppo.com>
 2026-07-15 11:34:03 +08:00
xiejian
655127511b fix(shardnode): fix meta stats tdigest index and insert replay 
with: #1001138628

Signed-off-by: xiejian <xiejian3@oppo.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-15 11:34:03 +08:00
JasonHu520
003b84afc7 refactor(scheduler): rebuild part of test unit for sc 
with: #1001120494
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-07-08 10:59:55 +08:00
slasher
2d4c724644 feat(dashboard): add usage and rate from used volumes 
. #1001125222

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-07-03 11:38:46 +08:00
JasonHu520
43c3b97b9b fix(clustermgr): fixed the failure to verify configuration due to missing precision 
with: #1001119322
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-07-02 11:08:24 +08:00
yujrchyang
ff1b231007 feat(blobstore): add AZ-aware blobnode selection for shard repair 
The scheduler now routes shard repair tasks to blobnode workers
in the same availability zone (IDC) as the damaged shard,
reducing cross-AZ data transfer during local stripe repair.

. #1001113674

Signed-off-by: yujrchyang <yujrchyang@gmail.com>
 2026-06-30 17:35:01 +08:00
zhumingze
898b19eb44 feat(cli): load extend codemode for cli 
. #1001106679

Signed-off-by: zhumingze <zhumingze@oppo.com>
 2026-06-30 11:44:03 +08:00
xiejian
e899894ce1 feat(blobnode): repair crc-corrupt shards in place during inspect 
with: #1001092565

Signed-off-by: xiejian <xiejian3@oppo.com>
 2026-06-30 10:58:24 +08:00
slasher
7dde9a7d3c fix(clustermgr): list service of expired time 
. #1001101726 #1001108374

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-06-30 09:30:20 +08:00
zhumingze
77b2def258 feat(access): Access memory optimization 
. #1001098459
No longer copying host information from the unit.

Signed-off-by: zhumingze <zhumingze@oppo.com>
 2026-06-25 17:43:15 +08:00
slasher
8a6c170068 feat(access): adapt tiny shard if its size less than MinShardSize 
. #1001088459

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-06-23 14:59:28 +08:00
slasher
5a17858611 fix(clustermgr): exclude old disk items which has empty node 
. #1000925801

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-06-23 12:29:41 +08:00
JasonHu520
8a4928b804 fix(clustermgr): list volume unit add logic size field 
with: #1001074153
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-22 21:06:53 +08:00
JasonHu520
1dbdd01580 refactor(scheduler): add log for balance task 
with: #1001074153
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-22 11:43:41 +08:00
JasonHu520
a6d47dd6c5 perf(blobnode): aware disk broken at inspect process in time 
with: #1001074153
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-17 14:58:15 +08:00
JasonHu520
7e3cbd5064 fix(blobnode): fix blobnode panic if not open inspect but request /inspect/chunk interface 
with: #1001059261
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-17 14:58:15 +08:00
JasonHu520
4fa520913a fix(clustermgr): healths of idleVolumes just count healthy allocatable volume 
with: #1001059261
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-17 14:58:13 +08:00
JasonHu520
42c21e5ebe perf(scheduler): unlock volume only if the volume is locked 
with: #1001016703 #1001058696

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-11 17:29:19 +08:00
JasonHu520
715e6ff2e7 refactor(scheduler): update error status for HTTPManualMigrateTaskAdd interface 
with: #1001041628
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-11 17:29:19 +08:00
JasonHu520
8b7912d575 perf(clustermgr): optimize volume creation, allocation, and renewal processes. 
with: #1001016703
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-06-02 16:54:02 +08:00
slasher
2b144df3a1 feat(clustermgr): add metric for dashboard 
. #1001004832

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-27 14:40:34 +08:00
slasher
4a3b8441e6 feat(cli): show dashboard in cli 
. #1001002832

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-26 16:58:03 +08:00
slasher
14fb599732 feat(clustermgr): fresh all volume every n-tick 
. #1000995387

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
45ec73f435 feat(clustermgr): dashboard for simulate 
. #1000949317

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
2608fc141a feat(clustermgr): dashboard for volume safety 
. #1000949281

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
622077aa15 feat(clustermgr): dashboard cache all nodes 
disk is dropped if node was dropped

. #1000925801

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
f81d4fe21e feat(clustermgr): dashboard for volume stat 
. #1000948474

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
1ea0f4a603 feat(clustermgr): dashboard for services status 
. #1000925801

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
7b3aad29ea feat(clustermgr): dashboard for disk usage 
. #1000925801

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
d7d902f9aa feat(clustermgr): add http dashboard and scope interface 
. #1000911000 and #1000911005

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
slasher
cc01412983 chore(ci): run some benchmark in ci 
. #1000910653

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:16:18 +08:00
JasonHu520
f230337013 perf(clustermgr): alloc first choose volume which disks have more free space 
with: #1000910653

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-05-25 11:16:18 +08:00
xiejian
c170c6c606 feat(clustermgr): add HTTP API to list blob nodes by status 
with: #1000985574

Signed-off-by: xiejian <xiejian3@oppo.com>
 2026-05-25 11:16:18 +08:00
JasonHu520
3211a5820c feat(scheduler): repair disk one by one for host 
evenly repairs bad disks with multiple hosts
with: #1000984660
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-05-25 11:16:18 +08:00
taozhijiang
81f255feec feat(blobnode): add blobnode compact metrics 
. #1000968420 #1000968420

Signed-off-by: taozhijiang <taozhijiang@oppo.com>
 2026-05-25 11:16:18 +08:00
JasonHu520
d53a874d80 feat(scheduler): implement compaction functionality for balanced tasks 
with: #1000953041
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-05-25 11:16:18 +08:00
zhumingze
a6d4d8119b feat(access): add testcase for access put&get 
. #1000824491

Signed-off-by: zhumingze <zhumingze@oppo.com>
 2026-05-25 11:16:18 +08:00
zhumingze
66d849ae03 feat(access): Optimize access put performance through pipeline 
. #1000824491

Signed-off-by: zhumingze <zhumingze@oppo.com>
 2026-05-25 11:16:18 +08:00
JasonHu520
74466f305d feat(blobnode): data inspect support batch read 
with: #1000817962
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-05-25 11:16:18 +08:00
zhumingze
e2d2a0ee5c feat(access): Add cluster_choose_alg configuration for access 
. #1000824920

Signed-off-by: zhumingze <zhumingze@oppo.com>
 2026-05-25 11:16:18 +08:00
JasonHu520
b00e03fccb feat(scheduler): optimization balance strategy 
If the disk water level is higher than the configured value, prioritize data migration
with: #1000949704
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-05-25 11:16:18 +08:00
xiejian
b09bd2abcb fix(proxy): handle current/backup swap during retain 
with: #1000910244

Signed-off-by: xiejian <xiejian3@oppo.com>
 2026-05-25 11:16:18 +08:00
yanghonggang
a763740261 perf(blobstore): optimize read buffer size calculation 
Previously, the read buffer was always allocated using the fixed
CrcBlockUnitSize, which wasted memory for small shards.

Now calculate buffer size as min(actual shard size, CrcBlockUnitSize)
to reduce memory usage.

Fixes: github-4033
. #1000824490

Signed-off-by: yanghonggang <yanghonggang@cmss.chinamobile.com>
 2026-05-25 11:16:17 +08:00
JasonHu520
07f397377e perf(clustermgr): before create volume task first check whether cluster has enough space 
with: #1000895756

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2026-05-25 11:16:13 +08:00
slasher
e80c255010 fix(access): retry on blobnode network error 
. #1000902483

Signed-off-by: slasher <shenjie1@oppo.com>
 2026-05-25 11:13:29 +08:00
yanghonggang
0655542269 feat(blobstore): add diskchunk backend for blobstore bench tool 
+ Use DiskStorage as the storage backend to facilitate analysis of
  performance bottlenecks at the storage engine level.
+ Update the help documentation for the benchmarking tool.

Fixes: github-4031

. #1000820154

Signed-off-by: yanghonggang <yanghonggang@cmss.chinamobile.com>
 2026-05-25 11:13:29 +08:00