gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-02 02:00:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,402 Commits 48 Branches 52 Tags 378 MiB
e9f6dca415
Commit Graph

4402 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
chihe
e9f6dca415 feat(metanode): hybrid cloud Delaying deletion of data before migration 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
044a94d26a feat(metanode): hybrid cloud The deletion of inode support hybridcloud 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
4173cc9f4d fix(master): hybrid cloud When updating a volume, ebsBlockSize can be modified if allowedStorageClass contains blobstore. 
Signed-off-by: true1064 <true1063@163.com>
 2024-12-26 09:24:22 +08:00
chihe
0bf7b9b83e fix(client): hybrid cloud update openforwrite for reused streamer 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
zhaochenyang
ae018c58cd refactor(lcnode): hybrid cloud add start time in heartbeat 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
576d4ce771 refactor(clientmetanode): hybrid cloud add debug log to check opMetaExtentsList value 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
7a0f5d43bd fix(metanode): hybrid cloud copy storage class from HybridCouldExtentsMigration of inode when executing migration 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
30b8a98bbf fix(metanode): hybrid cloud Copy WriteGeneration and ForbiddenMigration when getting inode from btree 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
5f5fd78af9 fix(metanode): hybrid cloud fix writeGen rollback to 0 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
zhaochenyang
d207dcc6fe refactor(lcnode): hybrid cloud fix lcnode log 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
94dcb991db feat(metanode): hybrid cloud add audit log for migration 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
d9c01f0af5 fix(client): hybrid cloud Fix some places where the original condition 
should be changed to "vol storageClass is blobstore".
	Adds field volStorageClass to ExtentClient for this purpose.
	Remove field volumeType from ExtentClient by the way.

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
5baafaef59 feat(meta): hybrid cloud provide an API to modify inode's ctime, for debug/test purpose. 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
zhaochenyang
75925990f1 refactor(lcnode): hybrid cloud lifecycle transition 
1. add debug service

Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
a62522f2f5 feat(meta): handle compatibility when old client send create request without field StorageClass 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
3de7e847a9 feat(master): handle compatibility of master when older version upgrade to hybrid cloud: 
(1) auto set mediaType of datanode, zone, datapartiton by config legacy "legacyDataMediaType" and persist;
	(2) auto set storageClass of volume by config legacy "legacyDataMediaType" and persist;
	(3) if need to rollback master to older version, need to do nothing, just replace to old master and restart. because hybrid cloud master only add new fields to meta by not changed old fields.

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
88567fbf65 fix(master): hybrid cloud if vol.volStorageClass is blobStore, add allowedStorageClass is forbidden. 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
6f6ea34f2a fix(master): hybrid cloud amend the way judging if the volume has snapshot version, in update allowedStorageClass procedure. 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
a2c2d4a704 fix(metanode): hybrid cloud assign migration hdd ek in UpdateExtentKeyAfterMigration 
Signed-off-by: chihe <chihe@oppo.com>
 2024-12-26 09:24:22 +08:00
zhaochenyang
4c73097a3d refactor(lcnode): hybrid cloud lifecycle transition 
1. close extent client
2. use StorageClass_Replica_HDD and StorageClass_BlobStore
3. scanner init AllowedStorageClass
4. enhance lifecycle parameter check
5. add extent client for write

Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
eb2d65002b refactor(metanode): hybrid cloud 1.Simplify the logic of updating extent key after migration 
2. reset inode reserved when excute marshall operation
	3. initial ebs client in mp
	4. remove cold vol or hot vol check in client and objectnode

Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
5f4ee4eacb fix(metanode): hybrid cloud fix bug when update extent key after migration 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
505015cd17 refactor(metanode): hybrid cloud add api for get full inode infomation include eks 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
f637d0cc19 fix(metanode): hybrid cloud check sortedEks is nil when excute fsmAppendExtentsWithCheck 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
0bc93129de refactor(client): hybrid cloud only file opened with write request needs to forbidden migration 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
zhaochenyang
d89812aaf0 feat(lcnode): hybrid cloud lifecycle transition 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
284ca0d211 feat(metanodeclient): hybrid cloud support hybridcloud data migration 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
35ed550795 feat(master): hybrid cloud forbidden AdminCreateVersion API because hybrid cloud not support snapshot version yet 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
8a5bc8583a feat(master)๏ผšmake the snapshot version and multiple storageClass mutually exclusive in a volume. 
Because now multiple allowedStorageClass can not support snapshot version.

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
9d2bf3b483 feat(mastercli): add API to add storageClass to volume's allowedStorageClass list 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
5bcc78bb43 fix(master): in create vol procedure, update datapartition view cache right after the creation of datapartitions 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
5936cdfe8d fix(master): amend some logs 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
443199e0b0 fix(client): hybrid cloud only clear dp when volume storage class is blobstore 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
1f227e722c fix(master): in create volume, sort allowedStorageClass after check if append volStorageClass info 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
10685ceba0 fix(master): when updateVol, check that req.VolStorageClass should be in vol.allowedStorageClass 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
d250c3041a fix(master): check if the cluster has resource to support req.VolStorageClass when create vol 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
f04993070b fix(master): some fix about datapartition creation 
(1) while creating dp in vol creation, check dp count by mediaType to judge if success.
	(2) when background periodical check if need to create dp, skip volumes whose createTime is too recently, to avoid background create dp while volume is creating.
	(3) display mediaType of dp in cfs-cli cmd "volume info -d"

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
9d0d2c9fef feat(master): hybrid cloud sort AllowedStorageClass[] while creating volume 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
1cdf8fcd39 fix(metanode): hybrid cloud fix log panic in function of deleteMarkedEBSInodes 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
51554785e3 fix(master): hybrid cloud not set zoneName as default value if not assigned in the create vol request. 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
4815a61444 fix(client): hybrid cloud fix bug about checking media type of dp 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
3373c73b4f fix(master): ybrid cloud not use var pointer dataNode if not got it from cluster, to avoid panic 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
7a7b663b6c feat(mastercli): hybrid cloud vol.volStorageClass can be changed by "cfs-cli volume update" 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
347208f189 refactor(clientmetanode): hybrid cloud check inode/vol storage class when checking vol type 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
1f9dab0eda feat(masterclientpreload): add property CacheDpStorageClass to volume. 
CacheDpStorageClass is used to inform SDK which storageClass to use when access cache dp.

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
92d1d54832 feat(master): volume supports multi mediaType based on the refactored codes from v3.4.0 
Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
14d7e0e19c feat(masterclientcli): hybrid cloud add property mediaType to datapartition 
(1)master: when creating vol, master creates dps of each mediaType according to vol.allowedStorageClass
	(2)client: when select dp to do append write, choose dp with spcific mediaType according to inode's storageClass

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
6cafecdb7d feat(masterclientcli): add property volStorageClass and allowedStorageClass to volume 
now volType is not uesed while creating volume, but is maintained for compability

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
true1064
3081a30168 feat(master): hybrid cloud add mediaType property to zone 
(1) zone mediaType is set as the first add-in datanode's mediaType
	(2) once zone mediaType is set, all datanodes in the zone must be the same mediaType.
	(3) the cmd 'cfs-cli zone info' shows zone mediaType.

Signed-off-by: true1064 <tangjingyu@oppo.com>
 2024-12-26 09:24:22 +08:00
chihe
029d1b359e feat(clientmetanode): hybrid cloud support change inode storageclass when updating ek 
Signed-off-by: chihe <chi.he@oppo.com>
 2024-12-26 09:24:22 +08:00