Commit Graph

227 Commits

Author SHA1 Message Date
zhumingze
b2f54cfa25 fix(data): print repairing error when reading a repairing dp copy encounter error. #1000196321
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:17 +08:00
shuqiang-zheng
465976624b fix(master): fix the problem that bad disk may not be able to be recovered due to a non-existent dp.
close:#1000186419

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
Victor1319
dce30cb1c5 refactor(data): refactor extent has been deleted log. #1000133844
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
shuqiang-zheng
d38270ebc9 fix(master): clear the repairingStatus when removing the raft member.
close:#1000164568

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:15 +08:00
shuqiang-zheng
c8176fc7fc fix(master): add upper and lower limit judgment when setting gogc value
overwrite is been allowed when dp decommission.

close:#1000011090 #1000161090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
94ff433984 fix(master): set the repairingStatus of dp via raft.
close:#1000158166 #1000158134

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
a35f51ba73 fix(master): extent deletion and business writes are forbidden when dp is repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
67fd938f90 fix(master): set all replicas of dp to repairingStatus during dp decommission repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
71dee686ef feat(data): limitIO strategy for separating repair io from business io. #1000095860
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
b519bf1223 fix(master): fix raft member additions not taking effect during dp decommission.
close:#1000136087

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
55c0331c14 fix(master): add some logs to locate raft member additions and removals that didn't take effect.
close:#1000105813

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
Victor1319
631baaf14d feat(master): add switch to control whether ignore tiny extent recover. #1000094081
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
36936cafd8 feat(data): refactor tiny extent delete logic. #1000089221
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
zhumingze
bfe93e050c feat(datamaster): Supplement test cases related to lost and bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
1964c60240 feat(datamaster): Optimize the check disk lost logic. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
1c794069a1 feat(masterdatacli): lost disks detection and reporting. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
clinx
3641507fbb fix(datanode): change the log level from error to warn for doStreamFixTinyDeleteRecord #1000068627
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:03 +08:00
chihe
78c727cda3 fix(datanode): adjust log level for no leader
close:#1000001079

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:52:02 +08:00
shuqiang-zheng
501dc61922 feat(meta): Support for dynamic adjustment of gogc.
close:#1000011090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:42:25 +08:00
Victor1319
4ed781a41a fix(data): fix gc clear data failed bug. #22957485
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-07 16:00:30 +08:00
leonrayang
2e8bf80543 fix(testcase): comment the test case has high ratio of failure out #23059813
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:26 +08:00
leonrayang
88ca620bf8 fix(datanode): Optimize io limiter exit process. #23059813
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:26 +08:00
leonrayang
30b221b2af feat(datanode): Return error if request is hung by io limiter and exceeds maximum wait time
. #23057829

Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:26 +08:00
clinx
56fc4fa9b1 fix(data): outofslice error #22883846
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-07 15:57:57 +08:00
clinx
2b0f857d01 feat(data): compatible json serialization for old datanode. #22883846
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-07 15:57:56 +08:00
clinx
e3ae2ea3f1 feat(data): change extentinfo serialize for getting remote extend info#22883846
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-07 15:57:56 +08:00
morphes1995
ab398d669d fix(master): RaftPartitionCanUsingDifferentPort make effective after and all dn/mn upgraded
Signed-off-by: morphes1995 <morphes1995@gmail.com>
2024-12-31 17:36:32 +08:00
Victor1319
8bb1131500 refactor(all): refactor go code with golint. #22906448
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
dac1a12d0a refactor(master): resolve conflicts when picking to master branch. #22906448
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
leonrayang
3396accd80 feat(projectstructure): reorganize the module directory in the root to a suitable subdirectory
close:#22458839

Signed-off-by: leonrayang <chl696@sina.com>
2024-12-26 09:24:22 +08:00
Victor1319
f9c2f49204 fix(meta): don't block extent delete req when no success exts. #22852635
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
ced8ca0978 feat(masterdata): add config to control whether vol read direct disk. #22818122
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
ab8217379e refactor(datanode): refactor data read and write performance. #22504692
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
9f7ffeb194 feat(masterdatameta): support volume to forbidden write operate codes of lower packet protocol version
close#22531837

Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
a43f96d403 feat(dataclientmaster): support configuring datanode and metanode to forbidden
write operate codes of lower packet protocol version.
       	 close#22531837
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
037df5f5f9 fix(sdkclient): client compatible with older version server modules
#22400600
	@formatter:off
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
852dafafd2 fix(datanode): if dpBackupTimeout is not set, then do not delete back up directories.
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
leonrayang
111f9e9903 feat(dataemeta): cluster switch on snapshot take effect on module handle protocal process
Signed-off-by: leonrayang <chl696@sina.com>
2024-12-26 09:24:22 +08:00
chihe
1888c9b8a8 fix(raft): if the previous member change did not return, reject the submission of the next member change
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
c3131b01f9 fix(datanode): use dp leader real size as to be repaired size.
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
323133a9f9 fix(datanode): return error if receive OpDeleteDataPartition when disks are not loaded
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
61e0ab3aa6 fix(master): fix deadlock for getDataPartitionsView
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
da33bc2068 feat(cli): support raftForce for datanode decommission by cfs-cli
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
46157d2c06 refactor(all): resolve conflict when pick commit from 3.3.2 to 3.4.0.
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
7b979b17e6 fix(data): fix delete gc data failed bug.
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
52c74006fe fix(datanode): stop raft if replica is deleted when StartRaftAfterRepair is finished
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
2fb56e7282 feat(master): auto delete backup directories
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
54e2526793 fix(datanode): delete backup directories by async
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
1f16566a92 fix(data): correctly determine the extent type in function handleBatchMarkDeletePacket
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
a348b6acb3 feat(master): api for removing backup directories on disk
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00