gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-02 02:00:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,791 Commits 48 Branches 52 Tags 378 MiB
14d37c87db
Commit Graph

4791 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Cloudstriff
14d37c87db feat(blobnode): add global config getter for blobnode 
with #1000088261

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
 2025-05-21 15:12:06 +08:00
Cloudstriff
319953c846 feat(clustermgr): add ChunkOversoldRatio and VolumeOverBoughtRatio support 
with #1000088261

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
 2025-05-21 15:12:06 +08:00
baihailong
92360e00a7 feat(docs): add FAQ 
@formatter:off

Signed-off-by: baihailong <baihailong@oppo.com>
 2025-05-21 14:44:31 +08:00
slasher
b7692def89 chore(ci): list all commits and json to local file 
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-05-20 19:24:41 +08:00
heymingwei
ec6ca0af10 fix(blobstore): waiting apply snapshot finish when save new raft log 
Signed-off-by: heymingwei <gongwilliam@163.com>
 2025-05-20 11:38:58 +08:00
slasher
c5389d27ad chore(.git_hub-2): check commit message with letters 
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-05-14 15:38:40 +08:00
zhangjianwei
6365c76349 fix(scheduler): adjust log levels to reduce unnecessary log output for blob_deleter 
case-1: during period blobnode chunk compact, delete ops are not allowed,
        so scheduler bloc_deleter will receive a large number of
        `chunk is compacting` error logs when MarkDelete/Delete.
case-2: during period blobnode chunk migrating, delete ops are not allowed,
        so scheduler blob_deleter will receive a large number of
        `vuid readonly` error logs when MarkDelete/Delete.
Solution:
to reduce log pressure,
set ErrChunkInCompact/ErrReadonlyVUID to warning level

close #3794

Signed-off-by: zhangjianwei <jianwei1216@qq.com>
 2025-05-13 15:36:04 +08:00
yanghonggang
d821466a1f fix(blobstore): rename CodeNoAvaliableVolume to CodeNoCodemodeVolume 
The previous error code 'CodeNoAvaliableVolume' (801) contained a typo
and was semantically ambiguous. To avoid confusion with the similarly
named 'CodeNoAvailableVolume' (917) from Clustermgr, it has been renamed
to 'CodeNoCodemodeVolume'.

Fixes: #3790

Signed-off-by: yanghonggang <yanghonggang_yewu@cmss.chinamobile.com>
 2025-05-13 14:50:05 +08:00
slasher
59df85ac57 feat(dial): add dial testing for blobstore 
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-05-09 10:02:20 +08:00
slasher
be56f5aa66 perf(util): store pointer of bytes in bytespool 
pkg: github.com/cubefs/cubefs/blobstore/util/bytespool
cpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
BenchmarkBytespoolSlice
BenchmarkBytespoolSlice-4      23023845    50.60 ns/op   29 B/op    1 allocs/op
BenchmarkBytespoolPointer
BenchmarkBytespoolPointer-4    60203656    18.60 ns/op    0 B/op    0 allocs/op

@formatter:off

close #3786

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-05-06 17:19:47 +08:00
JasonHu520
ecff057179 docs(docs): update docs for scheduler of blobstore 
Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-04-28 16:46:43 +08:00
mawei029
34e6122824 docs(blobnode): update blobnode config qos docs 
@formatter:off

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 16:46:43 +08:00
mawei029
0aeb9e0adb refactor(blobnode): first wait token, and then read/write io 
with: #1000064393

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
b1fa364a84 fix(blobnode): return user code for writing less body 
. #1000076490

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
8f7c5f1a09 chore(cli): list all broken chunks 
. #23065780

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
640cb1e6ed chore(blobnode): parse shard header at position 
with: #1000070129

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
9b25f1aeb4 refactor(blobnode): config reload dont return json args, only return ok 
with: #1000073970

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
47de7947f4 refactor(blobnode): limit the maximum depth value of qos config 
with: #1000074167

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
d56977c8ca refactor(blobnode): datafile Read return io.ReadCloser, shard body io.ReadCloser 
with: #1000064393

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
9492b9d139 fix(blobnode): fix release bytespool buffer when return reader, add ut concurrence read and write 
with: #1000064393

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
2bd3104251 chore(blobnode): add cost time statistics when datafile delete shard 
with: #23146103

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
82ddae18d4 feat(access): logging slow disk in get and put 
. #1000037209

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
fc8d7649dc feat(blobnode): qos io hot change max concurrence limit count 
with: #23146103

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
5d949cc7d7 docs(blobstore): add document of access directly read shard data 
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
413a35c2c6 refactor(blobnode): qos io delete concurrency, hot change delete max wait count 
with: #23079658

close #3720

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
tangdeyi
543a0b993e fix(clustermgr): fix heartbeat disk err return 
with #23092315

close #3699

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
512e717991 feat(blobnode): write big/small io payload with header and footer 
with: #23079652

close #3719

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
JasonHu520
a1fb8c24f8 feat(scheduler): add rate limit for delete 
with: #23065620

+ #3720

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
dd35b8f085 refactor(blobnode): optimize merge multi small io into single write, optimize read alloc memory 
with: #23079652, #23080447

+ #3719

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
70b557d945 chore(common): sized crc32 with go 1.17 
Add back transport later for TODO

@formatter:off

close #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
9059184293 feat(common): add crc32 sized block coder 
. #23096104

+ #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
9085a00da3 feat(common): avoid crc32 sized coder allocation 
. #23074869

+ #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
b4ed23d627 feat(common): crc32 encoder with append mode 
. #22922161

+ #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
ac2ac1549a feat(common): crc32 checker with fix mode 
return fixed decodable crc body

. #22922161

+ #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
f2cfda4126 feat(common): crc32 decoder with load mode 
return head and data

. #22922161

+ #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
427d0ae09c feat(common): add zero copy and partial content crc32 en-decoder 
. #22922161

+ #3718

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
tangdeyi
3cb3892be7 fix(clustermgr): some minor bug fixes 
1. put diskmgr refresh into loadData
2. fix check inconsistent vid
3. raft http server timeout can be configured
4. applyExpireVolume not double check volume expiration
5. applyAllocVolume not double check volume free size or health

with #23092315

close #3722

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
b31e484c9b fix(access): remove marker of listing broken disk 
. #23065780

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
JasonHu520
42f85990a5 perf(scheduler): optimize delete process 
with: #23065620

close #3721

Signed-off-by: JasonHu520 <huzongchao@oppo.com>
 2025-04-28 11:30:09 +08:00
mawei029
a95dde77bf fix(blobnode): delete local disk after it is repaired, fix access DiskNotFound 
@formatter:off

Signed-off-by: mawei029 <mawei2@oppo.com>
 2025-04-28 11:30:09 +08:00
slasher
0d3ac68d65 feat(access): with other timeout in read only data shard 
. #22782275

close #3717

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-28 11:30:09 +08:00
zhangjianwei
f4c9dac980 docs(cm): fix volume_inspect switch key 
Signed-off-by: zhangjianwei <jianwei1216@qq.com>
 2025-04-16 09:56:26 +08:00
liubingxing
3d51f7b6ba fix(cli): migrate metanode cmd cannot work with default limit mp count 
Signed-off-by: liubingxing <liubbingxing@gmail.com>
 2025-04-15 17:12:55 +08:00
liubingxing
e0ca16e6ad fix(cli): migrate metanode cmd cannot work with default limit mp count 
Signed-off-by: liubingxing <liubbingxing@gmail.com>
 2025-04-15 17:12:55 +08:00
liubingxing
ec394572c8 fix(master): fix chooseZoneNormal when multi zones and each one only has a node 
Signed-off-by: liubingxing <liubbingxing@gmail.com>
 2025-04-11 14:47:42 +08:00
liubingxing
cde83bd84f fix(client): dp near read do not work because NearHosts was replaced by Hosts 
Signed-off-by: liubingxing <liubbingxing@gmail.com>
 2025-04-10 15:28:04 +08:00
FourSpaces
b7635952e0 fix(objectnode): When S3 deletes an object, the empty folder is not recursively deleted 
Signed-off-by: FourSpaces <cheng1483x@gmail.com>
 2025-04-10 11:16:02 +08:00
FourSpaces
9cc87735fe fix(objectnode): S3 Folder not recursively deleted 
Signed-off-by: FourSpaces <cheng1483x@gmail.com>
 2025-04-10 11:16:02 +08:00
slasher
e8adeff5fa chore(build): try to use the newest command docker compose 
@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-09 16:41:55 +08:00
slasher
65889a5213 feat(util): try to wait with context in retry 
cpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
BenchmarkWait
BenchmarkWait/success
BenchmarkWait/success-4     432140625    2.793 ns/op    0 B/op  0 allocs/op
BenchmarkWait/retry-3
BenchmarkWait/retry-3-4      97395789   12.27 ns/op     0 B/op  0 allocs/op
BenchmarkContext
BenchmarkContext/success
BenchmarkContext/success-4  449174043    2.641 ns/op    0 B/op  0 allocs/op
BenchmarkContext/retry-3
BenchmarkContext/retry-3-4    3177039  380.2 ns/op    232 B/op  3 allocs/op

@formatter:off

Closes #3758

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-04-09 16:41:55 +08:00