Commit Graph

561 Commits

Author SHA1 Message Date
shuqiang-zheng
b519bf1223 fix(master): fix raft member additions not taking effect during dp decommission.
close:#1000136087

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
Victor1319
e7edd63446 fix(data): avoid replicas empty when buildDataPartitionRepairTask. #1000134827
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
55c0331c14 fix(master): add some logs to locate raft member additions and removals that didn't take effect.
close:#1000105813

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
f39493a9b4 fix(data): change ActionBatchMarkDelete limitIOError log level to info. #1000124626
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:29 +08:00
shuqiang-zheng
3cbb46bd80 fix(master): fix some issues found during decommission testing.
close:#1000059418

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:28 +08:00
zhumingze
c2611b2459 feat(data): Check dp availability at random times. #1000022788
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:28 +08:00
Wu Huocheng
df6a12c5d6 chore(datanode): change checking interval from hour to minute.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
631baaf14d feat(master): add switch to control whether ignore tiny extent recover. #1000094081
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
36936cafd8 feat(data): refactor tiny extent delete logic. #1000089221
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
a5467a3d58 fix(datanode): replace ExtentInfo with RepairExentInfo.#1000005968
This reverts commit e9d5ce73902aa196f89c5545ede7ca808e494bb3.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
62b28d0626 refactor(fmt): use gofumpt to format code. #1000063488
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
0ebe0b874f refactor(data): remove dp cache code from datanode. #1000062928
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
57d0b95284 feat(datanode): rotate all data partition raft log.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
22ca01df2f perf(datanode): reduce the extent info struct size.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
30b29217a1 feat(datanode): evict the extent cache by ttl.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
167e4c488a perf(datanode): replace make with bytespool.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
d7182d70bd feat(datameta): Record the read-only reason for dp&mp replicas. #1000060034
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00
Victor1319
6e05be7156 refactor(data): refactor the usage of the buffer pool during the prefetch process. #1000048465
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
24cb92350d feat(data): Mark the disks that encounter disk errors during startup as bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
bfe93e050c feat(datamaster): Supplement test cases related to lost and bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
1964c60240 feat(datamaster): Optimize the check disk lost logic. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
f1ad286a21 feat(data): Adjust some parameters and initialize. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
bf6d4f062f feat(masterdatacli): Add alerts for the completion of disk repair and improve cli display.
. #1000004886

Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
e0a1ff8c0f feat(data): Regularly check dp availability. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
1c794069a1 feat(masterdatacli): lost disks detection and reporting. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
clinx
6af47faed0 fix(datanode): change warn message log level #1000091570
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:59:24 +08:00
leonrayang
1ab568cfc1 fix(datanode): Reducing the sync process to enhance the speed of creating extent #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-08 14:59:24 +08:00
leonrayang
efeeea3dcd fix(datanode): Fix the enhance the persistence process of extentId #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-08 14:59:24 +08:00
clinx
a1fe50c7be fix(datanode): change write with return try other addr message to warn #1000082414
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:12 +08:00
clinx
818e4834b0 fix(datanode): opstreamfollowerread change timeout message to warn #1000082031
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:12 +08:00
leonrayang
5bd3da5846 feat(datanode): Enhance the persistence process of extentId #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-08 14:52:12 +08:00
clinx
3641507fbb fix(datanode): change the log level from error to warn for doStreamFixTinyDeleteRecord #1000068627
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-08 14:52:03 +08:00
chihe
78c727cda3 fix(datanode): adjust log level for no leader
close:#1000001079

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:52:02 +08:00
chihe
99a71d0cd7 fix(datanode): do not print log if diskBaseFileID equals to 0
close:#1000012686

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:51:45 +08:00
shuqiang-zheng
57fefaa661 feat(meta): perform security checks when configuring gogc, set upper and lower limits.
close:#1000011090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:42:25 +08:00
shuqiang-zheng
501dc61922 feat(meta): Support for dynamic adjustment of gogc.
close:#1000011090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:42:25 +08:00
chihe
ca90d3bac8 feat(master): add setRead/WriteLimits for flashnode
close:#1000004942

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:39:41 +08:00
chihe
380b8dde52 fix(flashnode): reply to client as soon as possible
close:#23109728

Signed-off-by: chihe <chihe@oppo.com>
2025-08-08 14:39:41 +08:00
Victor1319
4ed781a41a fix(data): fix gc clear data failed bug. #22957485
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-07 16:00:30 +08:00
chihe
be79605ea0 fix(flashnode): read data source one by one
close:#23095482

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:30 +08:00
leonrayang
8d897d1289 feat(datanode): Enable crc check for BaseExtentID information #23118077
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:29 +08:00
leonrayang
549128cf1a fix(datanode): Stop the tiny extent markdelete process while store stopped #23115007
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:29 +08:00
leonrayang
8d3fe7499c fix(datanode): Temporarily disable the crc read due to the wrong extentid in meta EXTENT_META
close: #23118077

Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:29 +08:00
zhumingze
175e6f4ee7 fix(datamaster): Adjust some error logs. #23101965
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-07 16:00:29 +08:00
zhumingze
3c91dcbf84 fix(data): Add dp information to cache in advance to avoid start raft error. #23113653
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-07 16:00:28 +08:00
leonrayang
72bf18cdff fix(data): Temporily modifying the test case has a high ratio of failure out #23059813
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:26 +08:00
leonrayang
2e8bf80543 fix(testcase): comment the test case has high ratio of failure out #23059813
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:26 +08:00
leonrayang
88ca620bf8 fix(datanode): Optimize io limiter exit process. #23059813
Signed-off-by: leonrayang <chl696@sina.com>
2025-08-07 16:00:26 +08:00
Wu Huocheng
45afc5aee0 fix(raft): use raftfsm.replicas instead of get peers.@formatter:off
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-07 16:00:26 +08:00
Wu Huocheng
ed61daa568 feat(raft): display raft peers.@formatter:off
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-07 16:00:26 +08:00