Commit Graph

1606 Commits

Author SHA1 Message Date
Wu Huocheng
664f2a9586 fix(master): fix the bug that add learner plan's overload is null.#1000578895
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-26 15:44:05 +08:00
Wu Huocheng
7ea467d7cb fix(metanode): batch commit in ApplySnapshot and trace op time.#1000579599
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-26 15:34:44 +08:00
Wu Huocheng
b9bd14a91b feat(metanode): add warn metric for rocksdb disk usage.#1000578895
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-26 14:50:09 +08:00
Wu Huocheng
85cdb71774 feat(master): add default volume store mode into configuration.#1000578895
fix(master): limit the max learner number to 5.#1000587619

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-26 14:48:09 +08:00
Wu Huocheng
93c4a75db3 chore(master): optimization of the master codes.#1000511705
fix(master): selete one address tobe delete.#1000577106

fix(master): count the required mp replica. #1000577106

Restore the count define back to original.

fix(master): check commit in the raft status.#1000580439

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-26 14:42:52 +08:00
zhumingze
4c7e409a73 feat(master): Adaptation of the maximum number of large disk node dp. #1000562853
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-12-26 14:14:25 +08:00
Wu Huocheng
fa86ecde97 feat(cli): add cli for metanode tag.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-22 11:30:50 +08:00
Wu Huocheng
1ff30ad62c fix(master): fix the bug that checking apply id fails.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-22 11:30:50 +08:00
Wu Huocheng
1727527195 chore(master): delete the unused code.#1000511705
The createStoreModeChangePlan is replaced with batchAddLearner.
So delete the unused code that is createStoreModeChangePlan.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-22 11:30:50 +08:00
Wu Huocheng
4345b97fbe feat(master): set zone, nodeset, ip list for learner batch api.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-22 11:30:50 +08:00
zhumingze
bfa9bfd727 fix(master): Add process dp num for manual distribution optimization interface. #1000239501
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-12-18 11:32:10 +08:00
zhumingze
951db45139 fix(master): Fix the issue where the dp num of vol can't reach 10. #1000541227
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-12-18 11:31:53 +08:00
Victor1319
fd9183ec06 feat(raft): enhance logging for peer management and learner recovery. #1000560474
- Add logging for peer removal actions in raft state machine
- Update addMetaReplica and addMetaReplicaLearner methods for logging
- Improve logging in partition FSM operations for member changes
- Enhance logging for learner mode recovery checks and actions
- Modify cluster balance logic to include additional parameters for learners

This commit improves the observability of raft operations and learner recovery processes.

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-12-17 17:36:42 +08:00
Victor1319
5baf97ccb5 feat(cluster): implement learner mode recovery timeout configuration. #1000560474
- Add learnerRecoverTimeoutSeconds parameter for cluster settings
- Update command line interface to accept learner recovery timeout
- Enhance API to parse and handle learnerRecoverTimeoutSeconds
- Modify cluster view to display learnerRecoverTimeoutSeconds status
- Implement logic to set learnerRecoverTimeoutSeconds in cluster

This commit enhances cluster management by allowing configuration of
the learner mode recovery timeout, improving the system's resilience
during decommissioning processes.

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-12-17 14:31:54 +08:00
Victor1319
f4511216a8 feat(cluster): add enableMpDecommissionByLearner parameter for cluster management. #1000539820
- Introduce enableMpDecommissionByLearner flag for cluster settings
- Update command line interface to accept new parameter
- Enhance API to parse and handle enableMpDecommissionByLearner
- Modify cluster view to display enableMpDecommissionByLearner status
- Implement logic to set enableMpDecommissionByLearner in cluster

This commit enhances cluster management capabilities by allowing
configuration of the mp decommissioning behavior.

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-12-17 14:29:06 +08:00
Wu Huocheng
e03c6f6173 chore(master): Optimize the code.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-16 17:41:19 +08:00
leonrayang
5ee498cc38 fix(metanode): Remove code blocks with extensive duplicated logic.#1000511705
Signed-off-by: leonrayang <changliang@oppo.com>
2025-12-15 19:43:23 +08:00
Wu Huocheng
3a3fcb7d82 chore(metanode): rewrite codes according to comments.#1000511705
refine the code with cursor

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-15 16:39:37 +08:00
Wu Huocheng
42ba9c79d6 chore(master): rewrite the codes to make it better.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-15 16:24:01 +08:00
Wu Huocheng
d572932a0a fix(master): fix the bug that can't allocate resource for mp plan.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-15 16:22:39 +08:00
Wu Huocheng
da23a16b7d feat(master): check metapartition md5sum.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-15 16:22:14 +08:00
Wu Huocheng
a89d685533 feat(metanode): get mp inode and dentry md5 sum.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-15 16:20:49 +08:00
Wu Huocheng
3612f75ef4 feat(master): batch create raft learner and promote.#1000511705
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-12-15 16:20:36 +08:00
zhumingze
d4efb36935 feat(master): Add a manual execution interface for distribution optimization.
.#1000239501

Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-12-15 11:19:47 +08:00
zhumingze
363222a82f feat(master): Use cursor to optimize some code in the master. #1000389771
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-12-08 14:39:38 +08:00
Victor1319
177a2dda17 feat(metanode): implement learner addition and promotion functionality. #1000511705
- Add constants for learner operations
- Implement commands to add and promote learners in meta partitions
- Update API routes for learner management
- Enhance logging for learner operations
- Modify data structures to include learner information
- Ensure proper error handling for learner operations

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-12-03 14:44:53 +08:00
Victor1319
bad8d049b5 feat(raft): add Closed method and improve partition handling. #1000432516
- Implement Closed method in Partition interface
- Add Closed method to raft server for status checking
- Enhance onStop method to use goroutines for partition stopping
- Update ApplySnapshot to handle closed partitions gracefully
- Add mock implementations for Closed method in tests

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-11-26 16:59:22 +08:00
Wu Huocheng
6bce72cca2 fix(metanode): seperate uniq checker for store mode.#1000457722
Memory store mode use version 1.
Rocksdb store mode use version 2.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-11-26 10:54:00 +08:00
zhumingze
328dc547c0 feat(sdk): Add meta near read logic. #1000415324
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-11-21 14:04:36 +08:00
zhumingze
b17cac2eb8 fix(master): logic adapts for decommission parts of the dp on the node. #1000441588
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-11-20 15:13:17 +08:00
Wu Huocheng
5307dafd70 fix(master): estimate meta partition size.#1000423269
Inode size base is 512. Dentry base is 64.

Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-11-19 19:33:43 +08:00
Wu Huocheng
bc07a83b30 chore(master): Set start time into mp plan.#1000449016
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-11-19 19:33:43 +08:00
Victor1319
be6c0e924e fix(client): update method signatures to include additional parameters. #1000451798
- Modify LookupPath to accept a boolean parameter
- Update InodeGet_ll to accept a boolean parameter
- Adjust ReadDirLimit_ll to include a boolean parameter
- Enhance MarkDecommissionStatus to include additional string parameter

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-11-19 14:52:13 +08:00
shuqiang-zheng
5e9cd5c9fd fix(master): fixed an issue where disks successfully decommissioned were re-marked .
close: #1000412387

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:24 +08:00
zhumingze
cc38a298af fix(master): Modify the display information of lostDisk status. #1000409772
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-11-18 16:32:24 +08:00
clinx
60d537015d fix(flash): check for a stop event when receiving a pause command
with: #1000394421

Signed-off-by: clinx <chenlin1@oppo.com>
2025-11-18 16:32:24 +08:00
clinx
14658c0660 fix(flash): warm up task set force remote cache true
with: #1000390069

Signed-off-by: clinx <chenlin1@oppo.com>
2025-11-18 16:32:24 +08:00
clinx
3a7285d529 feat(falsh): check file size limits for warm up task
with: #1000383519

Signed-off-by: clinx <chenlin1@oppo.com>
2025-11-18 16:32:24 +08:00
shuqiang-zheng
0fe65f8464 feat(master): Record the decommission term when the dataNode is decommissioned.
close: #1000362788

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:24 +08:00
shuqiang-zheng
2aaf4fa650 fix(master): fix the issue that the check for whether a dp could be marked decommission did not account for cases where the dp was precisely in the decommissionSuccess state.
close: #1000365149

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:04 +08:00
shuqiang-zheng
08e394fd65 fix(master): when querying the decommission progress of disks or nodes, display the weight and decommission type.
close: #1000362788

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:04 +08:00
shuqiang-zheng
459561d379 fix(master): when querying the decommissioning progress of a disk that has not yet been decommissioned, it returns the initial decommissioning status.
close: #1000362788

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:04 +08:00
shuqiang-zheng
5176484855 fix(master): fix an issue where the maximum rollback count determination in dp did not match the actual rollback count.
close: #1000359057

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:04 +08:00
shuqiang-zheng
0a5d9797d2 fix(master): fix the issue of inaccurate disk and node decommission progress due to leader changes.
close: #1000349627

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:04 +08:00
clinx
7cb266dd77 fix(flash): the master distributes the flashkeylimit value
with: #1000356686

Signed-off-by: clinx <chenlin1@oppo.com>
2025-11-18 16:32:04 +08:00
Victor1319
cac7ac5c25 fix(cluster): update getAllDataPartitionWithDiskPathByDataNode to handle discard partitions. #1000355377
- Modify function signature to include ignoreDiscard parameter
- Update logic to skip discarded data partitions when requested
- Ensure compatibility with existing data node retrieval logic

Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-11-18 16:32:03 +08:00
clinx
f97e6dff76 fix(flash): config client flow limit by master
with: #1000347719

Signed-off-by: clinx <chenlin1@oppo.com>
2025-11-18 16:32:03 +08:00
clinx
b0016bad3d fix(flash): pull handleReadTimeout when flashnode start
with: #1000347719

Signed-off-by: clinx <chenlin1@oppo.com>
2025-11-18 16:32:03 +08:00
shuqiang-zheng
0d9aaaeae6 fix(master): Optimize the reporting metrics for bad disks that fail to decommission promptly.
close: #1000336199

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:03 +08:00
shuqiang-zheng
a066ed17aa fix(master): fix an issue where token might consume twice during the decommission repair process of two replicaNum dp, due to the master changing the leader.
close: #1000336199

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-11-18 16:32:03 +08:00