Commit Graph

117 Commits

Author SHA1 Message Date
Victor1319
c3ccdfc7e2 refactor(sdk): remove dp cache code from sdk, client, cli. #1000063488
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
Victor1319
a6197cd99a refactor(master): remove dp cache code from master. #1000063482
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2025-08-08 14:59:27 +08:00
zhumingze
8986320af9 feat(mastermeta): support set fileStats thresholds. #1000047869
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:26 +08:00
zhumingze
c8d253aa39 feat(master): mp&dp replica missing check time can be adjusted. #1000044373
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
zhumingze
bfe93e050c feat(datamaster): Supplement test cases related to lost and bad disks. #1000004886
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-08 14:59:25 +08:00
baihailong
c1ee0a0053 fix(sdkmaster): volume remoteCacheReadTimeout parameter changed from seconds to milliseconds.
. #1000024513

Signed-off-by: baihailong <baihailong@oppo.com>
2025-08-08 14:51:45 +08:00
Wu Huocheng
e27923fa1c chore(master): rerange the codes.#22834621
Signed-off-by: Wu Huocheng <wuhuocheng@oppo.com>
2025-08-07 16:00:27 +08:00
zhumingze
55e0a1cade feat(master): Keep discard dp information in master and avoid panic when dp is nil. #23036742
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-07 16:00:27 +08:00
chihe
279817ca50 fix(client): modify default value for remoteCacheReadTimeoutSec
close:#23047275

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 16:00:26 +08:00
chihe
7e294e7f00 fix(master): deduplicate and remove contained for remote cache path
close:#23011064

Signed-off-by: chihe <chihe@oppo.com>
2025-08-07 15:57:57 +08:00
Wu Huocheng
6d881229b4 feat(master): Add freeze and clean empty meta partitions apis.#22834621
Signed-off-by: wu huocheng <wuhuocheng@oppo.com>
2025-08-07 15:45:37 +08:00
zhumingze
410e4545e1 feat(master): Support setting dp and mp cnt limit for different machines #22883514
Signed-off-by: zhumingze <zhumingze@oppo.com>
2025-08-07 15:45:37 +08:00
baihailong
00b082d8d3 fix(flashnode): format some code and fix some test case error.#22692038
Signed-off-by: baihailong <baihailong@oppo.com>
2025-08-07 15:45:25 +08:00
slasher
c98e438814 feat(flash): enable volume config in master
. #22200580

Signed-off-by: slasher <mcq.sejust@gmail.com>
2025-08-07 15:19:11 +08:00
slasher
015776b952 test(master): add test case of flash cache
. #22115371 of 21940685

Signed-off-by: slasher <mcq.sejust@gmail.com>
2025-08-07 15:19:02 +08:00
true1064
7ba5908e16 fix(ci): set master config enableDirectDeleteVol as true
close:#22716686
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
0173aa7e48 fix(master): hybrid cloud fix test cases of creating datapartitions
#22196488
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
3de7e847a9 feat(master): handle compatibility of master when older version upgrade to hybrid cloud:
(1) auto set mediaType of datanode, zone, datapartiton by config legacy "legacyDataMediaType" and persist;
	(2) auto set storageClass of volume by config legacy "legacyDataMediaType" and persist;
	(3) if need to rollback master to older version, need to do nothing, just replace to old master and restart. because hybrid cloud master only add new fields to meta by not changed old fields.

Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
d250c3041a fix(master): check if the cluster has resource to support req.VolStorageClass when create vol
Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
14d7e0e19c feat(masterclientcli): hybrid cloud add property mediaType to datapartition
(1)master: when creating vol, master creates dps of each mediaType according to vol.allowedStorageClass
	(2)client: when select dp to do append write, choose dp with spcific mediaType according to inode's storageClass

Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
true1064
6cafecdb7d feat(masterclientcli): add property volStorageClass and allowedStorageClass to volume
now volType is not uesed while creating volume, but is maintained for compability

Signed-off-by: true1064 <tangjingyu@oppo.com>
2024-12-26 09:24:22 +08:00
NaturalSelect
5365949f7c feat(master): support config decommission settings
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-12-26 09:24:22 +08:00
shuqiang-zheng
0ebf6ffc26 feat(reconstruct): gofmt by gofumpt.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-12-26 09:24:22 +08:00
NaturalSelect
bc5dd77fbd feat(master): support set dp heartbeat timeout
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-22 17:10:59 +08:00
NaturalSelect
6aca325797 feat(master): feat support vol level dp meta repair
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-22 17:10:59 +08:00
NaturalSelect
7a1c60eeea feat(master): support enable or disable auto dp meta repair
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-22 17:10:59 +08:00
W9068822
bf336dc34d feat(master): add getAllDataNodes and getAllMetaNodes apis
Signed-off-by: W9068822 <v-lijianrong1@oppo.com>
2024-07-18 16:57:38 +08:00
Victor1319
4df91ebc6e refactor(all): reslove merge conflicts between 3.3.x and 3.4.0.
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
2024-07-18 16:57:38 +08:00
leonrayang
a9b3d54028 feat(master): Volume update enable crossZone
Signed-off-by: leonrayang <changliang@oppo.com>
2024-07-18 16:57:38 +08:00
shuqiang-zheng
6b63f25071 fix(pprof): Fix for pprof endpoint exposures.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-07-18 16:57:38 +08:00
shuqiang-zheng
9ac7ca9128 enhance(cli):Modifying Volume Deletion Related Error Messages.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-07-18 16:57:38 +08:00
NaturalSelect
625a7a3d62 feat(master): enable audit log by default
NOTE: Please be careful when cherry-picking this commit.

Signed-off-by: NaturalSelect <2145973003@qq.com>
2024-07-18 16:57:38 +08:00
NaturalSelect
d201f99c04 feat(master): support set decommission disk limit
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-18 16:57:38 +08:00
shuqiang-zheng
eed62c913f enhance(log): Modify the threshold triggering log to logLeftSpaceLimitRatio of the total disk space
@formatter:off

Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-07-18 16:57:38 +08:00
NaturalSelect
488ff7434f test(master): fix unit tests
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-05 17:05:43 +08:00
chihe
7a6febec4b feat(cli): add decommissioned disks info for datanode node info display
Signed-off-by: chihe <chihe@oppo.com>
2024-07-05 17:05:43 +08:00
NaturalSelect
3b03341425 feat(master): support max mp cnt limit
@formatter:off

Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-05 17:05:43 +08:00
NaturalSelect
14ce5dc2cd test(master): fix unit tests
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-05 17:05:43 +08:00
NaturalSelect
7329d1f1e8 feat(master): check datapartition status when set dp discard
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-02 16:35:56 +08:00
NaturalSelect
610e6bd457 feat(master): support auto decommission disk and config
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-07-02 16:35:56 +08:00
S9054862
8ad316ecfc feat(data): repair block size support master config
Signed-off-by: S9054862 <huangzhibin1@oppo.com>
2024-07-01 15:09:55 +08:00
slasher
5c7898354f style(all): format all codes with golangci tools
closes #3371

@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2024-05-14 09:54:13 +08:00
shuqiang-zheng
e59f07a084 feat(reconstruct): gofmt by gofumpt.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-03-28 11:43:55 +08:00
shuqiang-zheng
42fa6628f4 enhance(cli): Modifying Volume Deletion Related Error Messages.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-03-28 11:43:55 +08:00
shuqiang-zheng
6d0de0df25 fix(master): Fixed the problem that delayed deleted volumes do not support migration.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-03-28 11:43:55 +08:00
shuqiang-zheng
7be84be6fa fix(master): Fix the bug that when the configured delayDeletionTime is changed.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-03-28 11:43:55 +08:00
shuqiang-zheng
dea6c736c5 feat(master): Add a configuration item to determine if freezing before volume deletion.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-03-28 11:43:55 +08:00
shuqiang-zheng
9c5185e9be feat(master): Freeze the volume when performing volume deletion.
Signed-off-by: shuqiang-zheng <zhengshuqiang@oppo.com>
2024-03-28 11:43:55 +08:00
NaturalSelect
3fefda48fe test(master): fix decommission unit test
Signed-off-by: NaturalSelect <huangzhibin1@oppo.com>
2024-03-27 14:24:50 +08:00
lily-lee
84dea94603 feat(master): compress client/partitions response data
Signed-off-by: lily-lee <lilylee88756@gmail.com>
2024-02-27 10:30:21 +08:00