Commit Graph

4935 Commits

Author SHA1 Message Date
xiejian
3f0f1d2b31 refactor(raft): remove raft config resolver, only keep transport resolver
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
5a93938026 refactor(access): move location crc calculate func to common package
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
5a07148126 refactor(raft): fix ut, and some ticker config
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
Cloudstriff
c724be4931 feat(shardnode): add snapshot header support on apply snapshot progress
with #22656329

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
2025-07-08 11:30:14 +08:00
xiejian
9f9cb91540 refactor(raft): simplify SealBlob api args
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
JasonHu520
596e12a83c refactor(blobnode): some changes for code style
with: #22481824 #22481829

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
2025-07-08 11:30:14 +08:00
Cloudstriff
6bfdb3547c fix(blobnode): fix transfer leader without update shard member type
with #22615053

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
2025-07-08 11:30:14 +08:00
JasonHu520
56bfb25102 feat(blobnode): access the client of clustermgr and shardnode
with: #22481824 #22481829

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
2025-07-08 11:30:14 +08:00
JasonHu520
d5e206e18f feat(blobnode): add shard task for worker
1. add some ut shard disk repair task

with: #22481824 #22481829

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
2025-07-08 11:30:14 +08:00
JasonHu520
67277bdf0e feat(scheduler): add shard disk repair manager
1. finish shard task process and shard disk repair

with: #22452167

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
2025-07-08 11:30:14 +08:00
JasonHu520
19e3ec09e4 feat(scheduler): add shard node task
1. Update the scheduler and worker interaction protocols to support new
task types

with: #22417838

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
2025-07-08 11:30:14 +08:00
JasonHu520
afaef8a55c fix(scheduler): fix task update volume failed and task retry duplicated in finish stage
with: #22481824 #22481829

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
6b1a33c26a fix(raft): fix raft transport panic when disk not finish load
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
6a0842c17c fix(shardnode): fix base transport singleflight key conflict
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
d2facf68d9 fix(raft): fix raft manager interface convert problem
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
2a57182e15 feat(shardnode): add shardnode related cli tools
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
ea7e25f25f feat(shardnode): support AllocSlice api, update SealBlob api args
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
4e563e095e refactor(shardnode): replace shardnode.Blob with commom.Blob
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
0e5d3eb8a1 fix(shardnode): fix the problem of incorrect assignment of unit learner
field in addShard process
fix rpc auditlog missing traceID problem, add shardnode log
fix update shard without suid
fix shard task routeVersion check

with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
xiejian
899fb14003 fix(clustermgr): fix cm AllocChunks failure causes concurrency issues
with #22565373

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:14 +08:00
slasher
e270e33102 chore(rpc2): add test server and client
tcp, transport, rpc, rpc2

@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:14 +08:00
slasher
d5e816db40 feat(rpc2): client retry on codec parameter
. #22637531

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:14 +08:00
slasher
fc32992b0b feat(rpc2): audit log on error handler
. #22617142

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:14 +08:00
tangdeyi
0612bfc8ff feat(clustermgr): blobstore cli add catalog cmd
. #22631156

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:14 +08:00
tangdeyi
aa952305d5 fix(clustermgr): remove shard compatibile mode code and record version
. #22416534

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:13 +08:00
xiejian
b5b77a1c97 fix(shardnode): fix test founded problems
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
20176fe835 fix(access): report punished service with threshold
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
f79491f781 feat(rpc2): add any codec for any value
. #22573386

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
xiejian
ce3acc2279 fix(common): rename Location and Slice's json tag
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:13 +08:00
xiejian
c6c97d78b6 fix(shardnode): fix kvstorev2 problems
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:13 +08:00
xiejian
c697fc85dd fix(shardnode): fix raft related problems
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:13 +08:00
xiejian
241781a646 test(shardnode): complete unit test
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:13 +08:00
tangdeyi
c96e84040c feat(clustermgr): add space management
. #22416534

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:13 +08:00
tangdeyi
85e6c02538 feat(clustermgr): add route management
. #22477961

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:13 +08:00
tangdeyi
533caf5fcd feat(clustermgr): add shard management and catalog module
. #22477946

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:13 +08:00
tangdeyi
f20a06c956 feat(clustermgr): add shardnode disk and node management
. #22432395

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:13 +08:00
Cloudstriff
6c8ece4ba6 feat(raft): add multi manager support at one transport only
with #22357426

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
2025-07-08 11:30:13 +08:00
Cloudstriff
1d5cf15406 fix(raft): fix raft transport return nil connection when dial failed
with #22357426

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
2025-07-08 11:30:13 +08:00
tangdeyi
b780eebda6 fix(clustermgr): add leader diskID of shard
. #22556044

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
7d1f94c043 feat(util): defaulter generic with float and integer
. #22575353

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
acc5db3371 feat(rpc2): add cli for simple request
. #22573386

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
ead8eda3c5 feat(rpc2): fixup some code of rpc2
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
xiejian
fe1543e18e feat(shardnode): adapt rpc2 protocol
with #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
7a935eb42d feat(rpc): metrics of auditlog with struct
. #22566070

Benchmark_ParserRow
Benchmark_ParserRow-4         199876    5903 ns/op   2264 B/op    30 allocs/op
Benchmark_ParserAuditlog
Benchmark_ParserAuditlog-4   1000000    1053 ns/op    389 B/op     9 allocs/op

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
08ffaad88d feat(rpc2): transport with rdma connection and writev
. #22550097 #22551076

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
946ec8516f feat(rpc2): transport with context cancel
. #22548427

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
e41f7a497b feat(rpc2): add rpc2 server and client with json
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
ac953debb5 feat(rpc2): add intercepor of auditlog
. #22537366

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
12dd967d35 feat(rpc2): add intercepor of auth
. #22537366

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00
slasher
fb6f396b3e feat(rpc2): change gogo proto stringer
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2025-07-08 11:30:13 +08:00