Commit Graph

336 Commits

Author SHA1 Message Date
Victor1319
dec11132ba refactor(master): support report monitor stats for cluster different media datanode. #22929430
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-07 16:00:30 +08:00
clinx
cfa8134cc7 feat(flashnode): add read and write latency metrics for cache #1000002444
Signed-off-by: clinx <chenlin1@oppo.com>
2025-08-07 16:00:30 +08:00
chihe
91965b0b44 fix(flashnode): use limitRead for read cache
close:#23123752

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:30 +08:00
chihe
9a1adf8212 fix(flashnode): use limitWrite for flashnode instead of disk
close:#23123752

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:30 +08:00
chihe
be79605ea0 fix(flashnode): read data source one by one
close:#23095482

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:30 +08:00
zhumingze
e0b861423d fix(util): Avoid label conflicts during the version metric pushing process. #23104396
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-07 16:00:29 +08:00
chihe
e32d901fb6 fix(flashnode): delete expired file for block cache before unlock
close:#23062373

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:28 +08:00
chihe
d7c377e5d3 fix(client): support 3.5.0 java sdk #23068759
Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:27 +08:00
shuqiang-zheng
3cdea7d89f feat(flashnode): modify the upper limit of cachePercent and the defaultMaxUseRatio of lru.
close:#23080181

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-07 16:00:27 +08:00
shuqiang-zheng
75d6aca05e fix(datanode): fixes for 1.read/write requests still being processed
2.log files not being found after service stopped.

close: #23013195

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-07 16:00:25 +08:00
shuqiang-zheng
101f0e0099 fix(flashnode): fix the problem of high cpu due to frequent gc on flashnode.
close:#22872898

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2025-08-07 15:45:54 +08:00
Wu Huocheng
f00702a0d0 feat(metanode): add set meta partition freeze raft command.#22834621
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-07 15:45:37 +08:00
baihailong
90798863ca fix(flashcache): format some code of flashcache.#22692038
Signed-off-by: baihailong <baihailong@oppo.com>
2025-08-07 15:45:09 +08:00
baihailong
1998c55397 fix(flashnode): add statistical metrics for flashnode.#22559602
Signed-off-by: baihailong <baihailong@oppo.com>
2025-08-07 15:19:30 +08:00
slasher
ecb0b49016 feat(util): add go ping for flash sdk
. #22200563

Signed-off-by: slasher <mcq.sejust@gmail.com>
2025-08-07 15:19:12 +08:00
slasher
297477a2ed chore(util): add util of bloom fliter
. #22200490

Signed-off-by: slasher <mcq.sejust@gmail.com>
2025-08-07 15:18:56 +08:00
slasher
b150a10a36 feat(flashnode): add service of flashnode
. #22115361 #22200518 of #21957840

Signed-off-by: slasher <mcq.sejust@gmail.com>
2025-08-07 15:18:46 +08:00
liubingxing
3d51f7b6ba fix(cli): migrate metanode cmd cannot work with default limit mp count
Signed-off-by: liubingxing <liubbingxing@gmail.com>
2025-04-15 17:12:55 +08:00
yanbin027
f25c5b9cc7 feat(client): "client support aheadread"
Signed-off-by: yanbin027 <yanbin027@ke.com>
2025-03-20 14:18:21 +08:00
slasher
c713d78265 chore(style): fix up semgrep for ip v4 and v6
Signed-off-by: slasher <shenjie1@oppo.com>
2025-02-19 10:05:11 +08:00
slasher
8e76523249 chore(github): upgrade action of upload-artifact
see more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/

Signed-off-by: slasher <shenjie1@oppo.com>
2025-02-08 15:27:14 +08:00
gukaifeng
2e24163a51 fix(metanode): fsync the crc file after wirte
Signed-off-by: gukaifeng <gukaifeng@xiaomi.com>
Signed-off-by: slasher <shenjie1@oppo.com>
2025-01-08 18:01:42 +08:00
Victor1319
8bb1131500 refactor(all): refactor go code with golint. #22906448
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
7c0e8fa470 fix(meta): use a read-write lock to prevent concurrent access to clustername. #22915506
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
dac1a12d0a refactor(master): resolve conflicts when picking to master branch. #22906448
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
c384557efe fix(meta): avoid panic when new export point. #22911241
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
89efec9e52 refactor(util): support delete audit log in log module. #22858505
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
zhumingze
f57815426a fix(util): change clean internal of audit log to 10s #22858505
Signed-off-by: zhumingze <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
zhumingze
498260102c feat(util): set the default rotate size for audit logs to 1G #22858505
Signed-off-by: zhumingze <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
edcad2d620 refactor(lcnode): rename variable name writeGen to leaseExpire. #22855673
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
zhumingze
306132d83d fix(util): modify clean logic of audit log and add test case #22858505
Signed-off-by: zhumingze <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
3a8f5eb3f8 feat(master): support query all client ip from master. #22834669
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
ced8ca0978 feat(masterdata): add config to control whether vol read direct disk. #22818122
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
ab8217379e refactor(datanode): refactor data read and write performance. #22504692
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
3825cad310 refactor(sdk): support limit tiny extent buf size. #22811542
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
a69ba6978f fix(meta): fix unmarshal append inode bug. #22764528
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Reliey
96f4b0101b feat(datanode): add endpoint for setting enablePid in exporter and handle response building #22257509
Signed-off-by: Reliey <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
Reliey
5f3c6e2d4a feat(cli): support reporting and viewing of cluster and vol dimension disk data #22257509
Signed-off-by: Reliey <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
Reliey
3986329ed7 feat(mastercli): support datanode report dpOpLog and diskOpLog infomation. #22257509
Signed-off-by: Reliey <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
Reliey
9ef9459cb3 test(stat): add testcase for statistic log when disk is full. #22637219
Signed-off-by: Reliey <zhumingze@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
a43f96d403 feat(dataclientmaster): support configuring datanode and metanode to forbidden
write operate codes of lower packet protocol version.
       	 close#22531837
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
6e06157948 feat(metalcnode): meta API OpMetaUpdateExtentKeyAfterMigration replies err details to lcNode
lcNode will print it to audit log
	#22327519
	#22385058
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
zhaochenyang
0fe9b02796 fix(lcnode): hybrid cloud optimize lcnode auditlog #22295845
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
2024-12-26 09:24:22 +08:00
zhaochenyang
ed9545c7a7 feat(lcnode): hybrid cloud add lcnode auditlog (#22067931)
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
2024-12-26 09:24:22 +08:00
chihe
94dcb991db feat(metanode): hybrid cloud add audit log for migration
Signed-off-by: chihe <chihe@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
6cafecdb7d feat(masterclientcli): add property volStorageClass and allowedStorageClass to volume
now volType is not uesed while creating volume, but is maintained for compability

Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
fd428b4706 refactor(master): fix confilcts when pick 3.4.0. #22906448
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
1e8ffbeb73 refactor(client): if subdir is empty from cfg file, use cfg from option.
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
Victor1319
448ab8e54f test(data): add testcase for datanode op log when disk is full.
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-12-26 09:24:22 +08:00
leonrayang
78bbb209f2 feat(datanode): enhance the preformace of the stat for disk opertion
Signed-off-by: leonrayang <chl696@sina.com>
2024-12-26 09:24:22 +08:00