Commit Graph

573 Commits

Author SHA1 Message Date
shuqiang-zheng
ed53a80882 fix(master): fix repairingStatus problem.
close:#1000164568

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
b73b063598 fix(data): Limit asynchronous read and write iocc to 100. #1000167799
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:14 +08:00
Victor1319
8385bf3e0c refactor(client): Limit prefetching to use at most one-third of the memory. #1000160740
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:32 +08:00
zhumingze
8f6bc07b66 fix(cli): Add logs for locating problem. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
c8176fc7fc fix(master): add upper and lower limit judgment when setting gogc value
overwrite is been allowed when dp decommission.

close:#1000011090 #1000161090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
94ff433984 fix(master): set the repairingStatus of dp via raft.
close:#1000158166 #1000158134

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
cadef3627a fix(data): Change the log level of dp not exist in repl_protocol from error to warn. #1000157333
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
a35f51ba73 fix(master): extent deletion and business writes are forbidden when dp is repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
67fd938f90 fix(master): set all replicas of dp to repairingStatus during dp decommission repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
36118447cd fix(master): modify alignment logic for new replica decommission progress to improve accuracy.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
71dee686ef feat(data): limitIO strategy for separating repair io from business io. #1000095860
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
1cf0bafea2 fix(data): fix null pointer panic When CreateDataPartition. #1000131492
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
b519bf1223 fix(master): fix raft member additions not taking effect during dp decommission.
close:#1000136087

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
Victor1319
e7edd63446 fix(data): avoid replicas empty when buildDataPartitionRepairTask. #1000134827
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
55c0331c14 fix(master): add some logs to locate raft member additions and removals that didn't take effect.
close:#1000105813

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
f39493a9b4 fix(data): change ActionBatchMarkDelete limitIOError log level to info. #1000124626
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:29 +08:00
shuqiang-zheng
3cbb46bd80 fix(master): fix some issues found during decommission testing.
close:#1000059418

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:28 +08:00
zhumingze
c2611b2459 feat(data): Check dp availability at random times. #1000022788
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:28 +08:00
Wu Huocheng
df6a12c5d6 chore(datanode): change checking interval from hour to minute.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
631baaf14d feat(master): add switch to control whether ignore tiny extent recover. #1000094081
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
36936cafd8 feat(data): refactor tiny extent delete logic. #1000089221
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
a5467a3d58 fix(datanode): replace ExtentInfo with RepairExentInfo.#1000005968
This reverts commit e9d5ce73902aa196f89c5545ede7ca808e494bb3.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
62b28d0626 refactor(fmt): use gofumpt to format code. #1000063488
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
0ebe0b874f refactor(data): remove dp cache code from datanode. #1000062928
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
57d0b95284 feat(datanode): rotate all data partition raft log.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
22ca01df2f perf(datanode): reduce the extent info struct size.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
30b29217a1 feat(datanode): evict the extent cache by ttl.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
167e4c488a perf(datanode): replace make with bytespool.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
d7182d70bd feat(datameta): Record the read-only reason for dp&mp replicas. #1000060034
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00
Victor1319
6e05be7156 refactor(data): refactor the usage of the buffer pool during the prefetch process. #1000048465
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
24cb92350d feat(data): Mark the disks that encounter disk errors during startup as bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
bfe93e050c feat(datamaster): Supplement test cases related to lost and bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
1964c60240 feat(datamaster): Optimize the check disk lost logic. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
f1ad286a21 feat(data): Adjust some parameters and initialize. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
bf6d4f062f feat(masterdatacli): Add alerts for the completion of disk repair and improve cli display.
. #1000004886

Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
e0a1ff8c0f feat(data): Regularly check dp availability. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
1c794069a1 feat(masterdatacli): lost disks detection and reporting. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
clinx
6af47faed0 fix(datanode): change warn message log level #1000091570
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:59:24 +08:00
leonrayang
1ab568cfc1 fix(datanode): Reducing the sync process to enhance the speed of creating extent #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-08 14:59:24 +08:00
leonrayang
efeeea3dcd fix(datanode): Fix the enhance the persistence process of extentId #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-08 14:59:24 +08:00
clinx
a1fe50c7be fix(datanode): change write with return try other addr message to warn #1000082414
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:12 +08:00
clinx
818e4834b0 fix(datanode): opstreamfollowerread change timeout message to warn #1000082031
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:12 +08:00
leonrayang
5bd3da5846 feat(datanode): Enhance the persistence process of extentId #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-08 14:52:12 +08:00
clinx
3641507fbb fix(datanode): change the log level from error to warn for doStreamFixTinyDeleteRecord #1000068627
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:03 +08:00
chihe
78c727cda3 fix(datanode): adjust log level for no leader
close:#1000001079

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:52:02 +08:00
chihe
99a71d0cd7 fix(datanode): do not print log if diskBaseFileID equals to 0
close:#1000012686

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:51:45 +08:00
shuqiang-zheng
57fefaa661 feat(meta): perform security checks when configuring gogc, set upper and lower limits.
close:#1000011090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:42:25 +08:00
shuqiang-zheng
501dc61922 feat(meta): Support for dynamic adjustment of gogc.
close:#1000011090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:42:25 +08:00
chihe
ca90d3bac8 feat(master): add setRead/WriteLimits for flashnode
close:#1000004942

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:39:41 +08:00
chihe
380b8dde52 fix(flashnode): reply to client as soon as possible
close:#23109728

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:39:41 +08:00