gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-02 02:00:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,813 Commits 48 Branches 52 Tags 378 MiB
ed72aa7146
Commit Graph

5813 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
mawei029
ed72aa7146 fix(blobnode): happen EIO when handle shard/chunk io, return broken disk 
with: #1000196614

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
af2989efdf chore(blobnode): optimize trace log detail, when occur error in io 
with: #1000195617

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
f42229bf27 feat(clustermgr): add configuration reserved space 
. #1000192943

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
83f0b7c140 feat(blobnode): start mode split blobnode service 
with: #1000159741

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
3197d47f54 feat(blobnode): put nopdata in worker repair 
. #1000180877

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
ed1b827156 feat(common): add zero buffer checker 
cpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
BenchmarkIsZeroBuffer
BenchmarkIsZeroBuffer/1M-zero-all
BenchmarkIsZeroBuffer/1M-zero-all-4        50384       23927   ns/op  0 B/op  0 allocs/op
BenchmarkIsZeroBuffer/1M-not-zero-first
BenchmarkIsZeroBuffer/1M-not-zero-first-4  1000000000      0.4 ns/op  0 B/op  0 allocs/op
BenchmarkIsZeroBuffer/1M-not-zero-mid
BenchmarkIsZeroBuffer/1M-not-zero-mid-4    108770      10617   ns/op  0 B/op  0 allocs/op
BenchmarkIsZeroBuffer/1M-not-zero-last
BenchmarkIsZeroBuffer/1M-not-zero-last-4   48726       22504   ns/op  0 B/op  0 allocs/op

. #1000180877

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
22bb533794 chore(test): change erroris target and err 
. #1000042638

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
49d847fec5 refactor(blobnode): rename worker model file 
with: #1000159720

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
tangdeyi
03cebe99f1 fix(clustermgr): volume allocation optimization 
1. sort vid by health when diskload is not enabled
2. sort vid by health and diskload when diskload is enabled

with #1000151206

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
 2025-10-13 15:29:34 +08:00
JasonHu520
408938ae13 refactor(blobnode): split shard encode process from storage to api 
with: #1000135108

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:34 +08:00
JasonHu520
4f4da7c09d refactor(blobnode): update get shards interface 
with: #1000135108

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:34 +08:00
JasonHu520
2f0b2b163b feat(blobnode): chunk data support batch read 
with: #1000135108

Signed-off-by: slasher <shenjie1@oppo.com>
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:34 +08:00
JasonHu520
ab621790d8 feat(blobnode): chunk migrate and disk repair support download data in batch 
with: #1000119221

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:34 +08:00
JasonHu520
8037070a38 feat(blobnode): add batch shard read interface for blobnode client 
with: #1000118252

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:34 +08:00
JasonHu520
349d757d79 feat(blobnode): add offset to shard info for ListShard and ShardStat interface 
with: #1000118065

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
ab2bb21e70 feat(cli): force write to readonly cluster 
. #1000172452

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
2352cfd075 feat(cli): support sdk in access cli 
. #1000172452

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
a808adf213 test(cli): pretty print access addresses 
. #1000334184

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
0fc7265d79 feat(blobnode): data inspect clean old disk metric 
because already replace a new disk and use another disk id

with: #1000166344

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
cb6818c4d0 refactor(blobnode): data inspect record local incorrect bids log 
with: #1000165799

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
6a9607f583 fix(blobnode): fix false positives data inspect 
with: #1000025911

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
mawei029
80b069d508 refactor(blobnode): optimize io context cancel, and it has become simpler 
with: #1000146037

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
390b3554b4 feat(access): add nopdata in access 
. #1000117047

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
7772b9ee6b feat(blobnode): add nopdata in blobnode 
. #1000117014

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
2300b0f382 feat(util): add generic math max and min 
. #1000153033

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:34 +08:00
tangdeyi
44b3ab643b feat(clustermgr): report broken unit num in volume shard 
with #1000151206

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
 2025-10-13 15:29:34 +08:00
slasher
bc69c2abeb perf(trace): use range function instead copy of slice 
. #23135507

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
0dc16bcc09 perf(trace): cache span start options and id 
Benchmark_Span_Pool
Benchmark_Span_Pool/cached
Benchmark_Span_Pool/cached-4  1656540 724.6 ns/op   48 B/op   1 allocs/op
Benchmark_Span_Pool/nocache
Benchmark_Span_Pool/nocache-4  906054  4233 ns/op 1224 B/op  17 allocs/op

. #23135507

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
ec6ee4a793 perf(trace): cache span to reduce allocation in span tracking 
Benchmark_Span_Pool
Benchmark_Span_Pool/nocache
Benchmark_Span_Pool/nocache-4  506971  6923 ns/op  1520 B/op  32 allocs/op
-->
Benchmark_Span_Pool
Benchmark_Span_Pool/cached
Benchmark_Span_Pool/cached-4   732175  1585 ns/op  654 B/op  16 allocs/op
Benchmark_Span_Pool/nocache
Benchmark_Span_Pool/nocache-4  688984  3551 ns/op 1582 B/op  23 allocs/op

Benchmark_Span_TrackLog
Benchmark_Span_TrackLog/duration-any
Benchmark_Span_TrackLog/duration-any-4    16785505  70.35 ns/op  0 B/op  0 allocs/op
Benchmark_Span_TrackLog/duration-second
Benchmark_Span_TrackLog/duration-second-4 18425104  64.40 ns/op  0 B/op  0 allocs/op
Benchmark_Span_TrackLog/duration-error
Benchmark_Span_TrackLog/duration-error-4  17530114  65.82 ns/op  0 B/op  0 allocs/op

. #23135507

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
ac1e84bf0b perf(trace): avoid to allocate option struct in span tracking 
from:
Benchmark_Span_TrackLog
Benchmark_Span_TrackLog/duration-any
Benchmark_Span_TrackLog/duration-any-4      7892170  457.1 ns/op  40 B/op  3 allocs/op
Benchmark_Span_TrackLog/duration-second
Benchmark_Span_TrackLog/duration-second-4  10384854  113.5 ns/op  28 B/op  3 allocs/op
Benchmark_Span_TrackLog/duration-error
Benchmark_Span_TrackLog/duration-error-4    5301676  500.1 ns/op  80 B/op  6 allocs/op

to:
Benchmark_Span_TrackLog
Benchmark_Span_TrackLog/duration-any
Benchmark_Span_TrackLog/duration-any-4     11378097  97.32 ns/op  16 B/op  1 allocs/op
Benchmark_Span_TrackLog/duration-second
Benchmark_Span_TrackLog/duration-second-4  17787051  211.9 ns/op  24 B/op  3 allocs/op
Benchmark_Span_TrackLog/duration-error
Benchmark_Span_TrackLog/duration-error-4    9427880  189.0 ns/op  40 B/op  3 allocs/op

. #23135507

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
ad4e8b0f9f perf(util): format int to string with function outlining 
BenchmarkUtilFormatInt
BenchmarkUtilFormatInt/format
BenchmarkUtilFormatInt/format-4   100000000  13.48 ns/op  0 B/op  0 allocs/op
BenchmarkUtilFormatInt/strconv
BenchmarkUtilFormatInt/strconv-4   21880332  92.22 ns/op  7 B/op  1 allocs/op

. #23135507

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
mawei029
076de2685d refactor(blobnode): remove unused switch reAddDisk 
with: #1000025913

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
823d3f3a0c pref(rpc2): _gc_ reuse objects in rpc2 
. #23113571

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
e5125b59c4 pref(rpc2): _gc_ reuse objects in transport frames 
. #23113571

cpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
BenchmarkAcceptClose
BenchmarkAcceptClose  276475   5393 ns/op  904 B/op  7 allocs/op
BenchmarkConnSmux
BenchmarkConnSmux      39103  31170 ns/op  121 B/op  0 allocs/op
BenchmarkPipeSmux
BenchmarkPipeSmux      90400  13771 ns/op    6 B/op  0 allocs/op
BenchmarkConnTCP
BenchmarkConnTCP       61194  18310 ns/op    0 B/op  0 allocs/op
BenchmarkRangedWrite
BenchmarkRangedWrite  242226   5230 ns/op   24 B/op  0 allocs/op

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
a793d19ce0 pref(rpc2): _gc_ reuse objects in transport allocator 
. #23113571

cpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
BenchmarkMSB
BenchmarkMSB         172919274   6.938 ns/op  0 B/op  0 allocs/op
BenchmarkAllocator
BenchmarkAllocator    38689362  31.63  ns/op  1 B/op  0 allocs/op
BenchmarkBufferRing
BenchmarkBufferRing     278162    4190 ns/op  0 B/op  0 allocs/op

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
50c3e4b4e8 fix(rpc2): write limited body to reader 
. #1000334184

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
7fcb5c327a feat(blobnode): ranged read with aligned buffer 
. #22951605

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
7f8a4916d8 feat(util): alignment size's head and padding tail 
. #22984144

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
d00703a458 feat(transport): add ranged writer of frame 
. #22922156

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
a2c0dfbadc feat(transport): add flag error for writing to next frame 
. #22922145

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
7fcee0e46f feat(rpc2): aligned checksum buffer in request and response body 
saving bytes in small en-decode body

. #22922123 #22922136

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
d54ac862f5 feat(transport): allocated buffer with frame header and aligned address 
. #22922112

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
8c156972a8 chore(testing): move mock files to testing all 
. #1000145567

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
ad3442a467 chore(testing): move mock files to testing shardnode 
. #1000145567

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
61c2e882fb chore(testing): move mock files to testing clustermgr 
. #1000145567

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
0ab1c039a0 chore(testing): move mock files to testing proxy 
. #1000145567

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00
JasonHu520
e0f47a5037 feat(scheduler): unlock volume support volume epoch 
with: #1000131117

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:33 +08:00
tangdeyi
3cebf9fdbf feat(clustermgr): lock unlock volume support epoch 
with #1000102800

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
 2025-10-13 15:29:33 +08:00
JasonHu520
f2bd5dc417 fix(raft): should check header of snapshot request 
with: #1000100262

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-10-13 15:29:33 +08:00
slasher
6506cab835 feat(common): add extend codemode 
. #1000111154

close .3773 .3767

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-10-13 15:29:33 +08:00