Commit Graph

592 Commits

Author SHA1 Message Date
zhumingze
0aabac43db fix(all): Downgrade some log levels from error to warn. #1000202936
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:18 +08:00
Victor1319
ddde96c5f6 refactor(data): modify punch hole sync size to 128M for tiny extent. #1000212704
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:18 +08:00
zhumingze
30f81f1575 fix(data): Fix data log problem. #1000202936
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:17 +08:00
zhumingze
b2f54cfa25 fix(data): print repairing error when reading a repairing dp copy encounter error. #1000196321
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:17 +08:00
Victor1319
809d023e9e refactor(vendor): user smux in depends to replace pkg in vendor. #1000193713
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:16 +08:00
Victor1319
5ea53eb575 fix(data): update extent size after load from disk. #1000182163
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:16 +08:00
shuqiang-zheng
999f2989ac fix(master): fix panic caused in build heartbeat response after loading broken disk.
close:#1000191783

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
zhumingze
12895d975f feat(meta): Add readDir iops limiter. #1000187434
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:16 +08:00
shuqiang-zheng
b258e285f2 fix(master): fix the problem of disk not found when using cli to query disk information.
close:#1000188189

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
shuqiang-zheng
465976624b fix(master): fix the problem that bad disk may not be able to be recovered due to a non-existent dp.
close:#1000186419

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:16 +08:00
zhumingze
9e9eb0e753 fix(data): bad disk judgment logic fix. #1000184384
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
33c6e2d96d fix(data): limitIO ​​logic fix. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
b7cc2f2b8a fix(cli): Add logs for loacting problem. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
Victor1319
9bb5598edf fix(data): retry repair directly for tiny empty recover offset check. #1000173499
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
Victor1319
e4b0d9a4ea fix(data): check extent store whether closed before tiny extent recover. #1000182163
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
d624108ddc fix(data): fix panic while create datapartition. #1000173883
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
Victor1319
dce30cb1c5 refactor(data): refactor extent has been deleted log. #1000133844
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
4034350ab6 fix(data): selectDisk skip lost disk. #1000173883
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:15 +08:00
shuqiang-zheng
d38270ebc9 fix(master): clear the repairingStatus when removing the raft member.
close:#1000164568

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:15 +08:00
shuqiang-zheng
ed53a80882 fix(master): fix repairingStatus problem.
close:#1000164568

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-11 17:21:15 +08:00
zhumingze
b73b063598 fix(data): Limit asynchronous read and write iocc to 100. #1000167799
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-11 17:21:14 +08:00
Victor1319
8385bf3e0c refactor(client): Limit prefetching to use at most one-third of the memory. #1000160740
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:32 +08:00
zhumingze
8f6bc07b66 fix(cli): Add logs for locating problem. #1000158099
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
c8176fc7fc fix(master): add upper and lower limit judgment when setting gogc value
overwrite is been allowed when dp decommission.

close:#1000011090 #1000161090

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
94ff433984 fix(master): set the repairingStatus of dp via raft.
close:#1000158166 #1000158134

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
cadef3627a fix(data): Change the log level of dp not exist in repl_protocol from error to warn. #1000157333
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
a35f51ba73 fix(master): extent deletion and business writes are forbidden when dp is repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
67fd938f90 fix(master): set all replicas of dp to repairingStatus during dp decommission repair.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
shuqiang-zheng
36118447cd fix(master): modify alignment logic for new replica decommission progress to improve accuracy.
close:#1000121688

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:31 +08:00
zhumingze
71dee686ef feat(data): limitIO strategy for separating repair io from business io. #1000095860
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
1cf0bafea2 fix(data): fix null pointer panic When CreateDataPartition. #1000131492
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
b519bf1223 fix(master): fix raft member additions not taking effect during dp decommission.
close:#1000136087

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
Victor1319
e7edd63446 fix(data): avoid replicas empty when buildDataPartitionRepairTask. #1000134827
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:30 +08:00
shuqiang-zheng
55c0331c14 fix(master): add some logs to locate raft member additions and removals that didn't take effect.
close:#1000105813

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:30 +08:00
zhumingze
f39493a9b4 fix(data): change ActionBatchMarkDelete limitIOError log level to info. #1000124626
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:29 +08:00
shuqiang-zheng
3cbb46bd80 fix(master): fix some issues found during decommission testing.
close:#1000059418

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-08 14:59:28 +08:00
zhumingze
c2611b2459 feat(data): Check dp availability at random times. #1000022788
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:28 +08:00
Wu Huocheng
df6a12c5d6 chore(datanode): change checking interval from hour to minute.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
631baaf14d feat(master): add switch to control whether ignore tiny extent recover. #1000094081
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
36936cafd8 feat(data): refactor tiny extent delete logic. #1000089221
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
a5467a3d58 fix(datanode): replace ExtentInfo with RepairExentInfo.#1000005968
This reverts commit e9d5ce73902aa196f89c5545ede7ca808e494bb3.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
62b28d0626 refactor(fmt): use gofumpt to format code. #1000063488
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
0ebe0b874f refactor(data): remove dp cache code from datanode. #1000062928
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Wu Huocheng
57d0b95284 feat(datanode): rotate all data partition raft log.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
22ca01df2f perf(datanode): reduce the extent info struct size.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
30b29217a1 feat(datanode): evict the extent cache by ttl.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
Wu Huocheng
167e4c488a perf(datanode): replace make with bytespool.#1000005968
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
d7182d70bd feat(datameta): Record the read-only reason for dp&mp replicas. #1000060034
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00
Victor1319
6e05be7156 refactor(data): refactor the usage of the buffer pool during the prefetch process. #1000048465
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
24cb92350d feat(data): Mark the disks that encounter disk errors during startup as bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00