Commit Graph

598 Commits

Author SHA1 Message Date
shuqiang-zheng
5bea69cd67 feat(master): the dp decommission progress will not exceed 100%.
close: #1000471110

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
(cherry picked from commit c171ba1a1c)
2025-12-24 17:01:31 +08:00
shuqiang-zheng
c2120b249a fix(master): reported bad disk not complete decommission timely metric.
close:#1000242504

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:20 +08:00
zhumingze
ea04e20415 fix(data): Determine the disk lost based on master and locally loaded Dp information
. #1000244723

Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:20 +08:00
zhumingze
4f0b89ebb6 fix(data): Fix autoComputeCRC log issue. #1000232936
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:20 +08:00
shuqiang-zheng
44cd11860e fix(master): modify dp decommission failure retry interval to 5 minutes.
close:#1000219034

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:19 +08:00
shuqiang-zheng
e39e185356 fix(master): show dp of missing tiny extent when cli performs data partition checking.
close:#1000059418

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:18 +08:00
zhumingze
0aabac43db fix(all): Downgrade some log levels from error to warn. #1000202936
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:18 +08:00
Victor1319
ddde96c5f6 refactor(data): modify punch hole sync size to 128M for tiny extent. #1000212704
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:18 +08:00
zhumingze
30f81f1575 fix(data): Fix data log problem. #1000202936
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:17 +08:00
zhumingze
b2f54cfa25 fix(data): print repairing error when reading a repairing dp copy encounter error. #1000196321
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:17 +08:00
Victor1319
809d023e9e refactor(vendor): user smux in depends to replace pkg in vendor. #1000193713
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:16 +08:00
Victor1319
5ea53eb575 fix(data): update extent size after load from disk. #1000182163
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:16 +08:00
shuqiang-zheng
999f2989ac fix(master): fix panic caused in build heartbeat response after loading broken disk.
close:#1000191783

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
zhumingze
12895d975f feat(meta): Add readDir iops limiter. #1000187434
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:16 +08:00
shuqiang-zheng
b258e285f2 fix(master): fix the problem of disk not found when using cli to query disk information.
close:#1000188189

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
shuqiang-zheng
465976624b fix(master): fix the problem that bad disk may not be able to be recovered due to a non-existent dp.
close:#1000186419

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
zhumingze
9e9eb0e753 fix(data): bad disk judgment logic fix. #1000184384
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
33c6e2d96d fix(data): limitIO โ€‹โ€‹logic fix. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
b7cc2f2b8a fix(cli): Add logs for loacting problem. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
Victor1319
9bb5598edf fix(data): retry repair directly for tiny empty recover offset check. #1000173499
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
Victor1319
e4b0d9a4ea fix(data): check extent store whether closed before tiny extent recover. #1000182163
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
d624108ddc fix(data): fix panic while create datapartition. #1000173883
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
Victor1319
dce30cb1c5 refactor(data): refactor extent has been deleted log. #1000133844
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
4034350ab6 fix(data): selectDisk skip lost disk. #1000173883
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
shuqiang-zheng
d38270ebc9 fix(master): clear the repairingStatus when removing the raft member.
close:#1000164568

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:15 +08:00
shuqiang-zheng
ed53a80882 fix(master): fix repairingStatus problem.
close:#1000164568

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
b73b063598 fix(data): Limit asynchronous read and write iocc to 100. #1000167799
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:14 +08:00
Victor1319
8385bf3e0c refactor(client): Limit prefetching to use at most one-third of the memory. #1000160740
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:32 +08:00
zhumingze
8f6bc07b66 fix(cli): Add logs for locating problem. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
c8176fc7fc fix(master): add upper and lower limit judgment when setting gogc value
overwrite is been allowed when dp decommission.

close:#1000011090 #1000161090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
94ff433984 fix(master): set the repairingStatus of dp via raft.
close:#1000158166 #1000158134

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
cadef3627a fix(data): Change the log level of dp not exist in repl_protocol from error to warn. #1000157333
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
a35f51ba73 fix(master): extent deletion and business writes are forbidden when dp is repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
67fd938f90 fix(master): set all replicas of dp to repairingStatus during dp decommission repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
36118447cd fix(master): modify alignment logic for new replica decommission progress to improve accuracy.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
71dee686ef feat(data): limitIO strategy for separating repair io from business io. #1000095860
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
1cf0bafea2 fix(data): fix null pointer panic When CreateDataPartition. #1000131492
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
b519bf1223 fix(master): fix raft member additions not taking effect during dp decommission.
close:#1000136087

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
Victor1319
e7edd63446 fix(data): avoid replicas empty when buildDataPartitionRepairTask. #1000134827
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
55c0331c14 fix(master): add some logs to locate raft member additions and removals that didn't take effect.
close:#1000105813

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
f39493a9b4 fix(data): change ActionBatchMarkDelete limitIOError log level to info. #1000124626
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:29 +08:00
shuqiang-zheng
3cbb46bd80 fix(master): fix some issues found during decommission testing.
close:#1000059418

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:28 +08:00
zhumingze
c2611b2459 feat(data): Check dp availability at random times. #1000022788
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:28 +08:00
Wu Huocheng
df6a12c5d6 chore(datanode): change checking interval from hour to minute.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
631baaf14d feat(master): add switch to control whether ignore tiny extent recover. #1000094081
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
36936cafd8 feat(data): refactor tiny extent delete logic. #1000089221
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
a5467a3d58 fix(datanode): replace ExtentInfo with RepairExentInfo.#1000005968
This reverts commit e9d5ce73902aa196f89c5545ede7ca808e494bb3.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
62b28d0626 refactor(fmt): use gofumpt to format code. #1000063488
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
0ebe0b874f refactor(data): remove dp cache code from datanode. #1000062928
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
57d0b95284 feat(datanode): rotate all data partition raft log.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00