gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-07 12:27:26 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
3,894 Commits 48 Branches 52 Tags 378 MiB
15ecac6bcd
Commit Graph

32 Commits

Author SHA1 Message Date
zhaochenyang
ca861f4017 fix(lcnode): fix lcnode panic and reduce task allocation time 
#21938894
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-05-13 17:30:54 +08:00
zhaochenyang
a1c3183386 fix(lcnode): snapshot apply on lcnode 
1. add firstDentry may cause panic if channel close
2. reset snapshot verinfos before add verinfo
3. add TaskResults in GetOneTask to avoid adding tasks repeatedly
4. support notify multi snapshot tasks
5. delete snapshot failed retry

Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2024-02-22 14:53:39 +08:00
slasher
5d5e45cb0a chore(util): store current time to atomic value 
BenchmarkGetCurrentTime-4    100000000    11.60 ns/op            0 B/op          0 allocs/op
-->
BenchmarkGetCurrentTime-4    895091701     1.31 ns/op            0 B/op          0 allocs/op

Signed-off-by: slasher <mcq.sejust@gmail.com>
 2024-01-31 19:13:25 +08:00
slasher
8234a93836 chore(lcnode): golangci lint apply into lcnode 
Signed-off-by: slasher <shenjie1@oppo.com>
 2023-12-28 17:13:05 +08:00
zhaochenyang
bd6608b8d0 enhance(lcnode): support multi task 
1. add lcTaskCountLimit
2. delete Scanning and add WorkingCount
3. add lcnode addr in heartbeat resp and scan resp

Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-12-14 11:14:14 +08:00
lily-lee
126cef4bc6 fix(code scan): remove default statements in select added before 
Signed-off-by: lily-lee <lilylee88756@gmail.com>
 2023-12-09 22:32:29 +08:00
lily-lee
a8620dc1e9 fix(code scan): fix code-scanning problems. 
Signed-off-by: lily-lee <lilylee88756@gmail.com>
 2023-11-28 12:32:15 +08:00
zhaochenyang
61b19a1202 fix(lcnode): stop lcnode straightway 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-10-27 14:07:57 +08:00
zhaochenyang
3507c8d9b4 fix(lcnode): close metawrapper in lcnode 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-10-27 14:07:57 +08:00
NaturalSelect
34ec9ff8c6 feat(lc): support full path audit log 
Signed-off-by: NaturalSelect <2145973003@qq.com>
 2023-10-18 14:51:09 +08:00
zhaochenyang
c655eafbea fix(lcnode): fix duplicate sending of the same task and update delete interface 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-10-09 10:18:07 +08:00
leonrayang
2ed714c33c refactor(sdk,metaNode): refactor function name and param mainly relate to snapshot 
Signed-off-by: leonrayang <chl696@sina.com>
 2023-09-05 19:30:39 +08:00
zhaochenyang
95c3c127fb refactor(lcnode): refactor some code about lcnode task manager 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-08-31 16:36:14 +08:00
zhaochenyang
f8c657daad refactor(lcnode): refactor some code in lcnode 
1)refactor some code about snapshot
2)fix some bugs
3)add some ut

Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-08-30 20:05:32 +08:00
zhaochenyang
dcf9551268 fix(master): fix panic that may be caused in lcnode task 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-08-30 20:05:32 +08:00
zhaochenyang
a6d125f5ce feat(lcnode/master): lcnode support bucket lifecycle and vol snapshot delete 
Signed-off-by: zhaochenyang <zhaochenyang@oppo.com>
 2023-08-30 20:05:32 +08:00
leonrayang
3a6e0715d3 fix: go vet not pass 
Signed-off-by: leonrayang <chl696@sina.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
1960b503c4 fix: same task may be dispatched to multiple lcnodes when master leader changed or restarted 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
66d95cb3bb fix: skip reading child files of dir in second phase scanning 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
leonrayang
9c54f81c46 enhance: update read dir params to recognize options from snapshot deletion 
Signed-off-by: leonrayang <chl696@sina.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
820fa2d910 fix: 1.two phase scanning may cause some dirs are not deleted. 2.fix scan task redispatching problem 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
771f53b7dd enhancement: add interface for getting lcnode info from master 
fix: when lcnode is stopped or crashed while doing snapshot version deleting task, and the task will be redispatched to other lcnode by master, in this case, the remaining inodes and dentries of the snapshot version can not be retrieved and deleted correctly

Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
8f9417b523 fix: 1.add parameters for lcnode task routine num. 2.return registration err when max concurrent lcnode reached 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
f576dece63 fix: batchExpirationGetNum and scanCheckInterval config parameters can not load correctly 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
e3826794bd Fix: marker file might be deleted during scanning, while next batch of children of readdirlimit may skip the next file, which would cause some file can not be deleted in the scan task 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
172ff244b4 audit log add debug info 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
leonrayang
8efae62da3 Update: enhance snapshot on tiny extent and split senarios. etc... 
.write on exist extent normally happened and create a new extent to write. but for snapshot scenarios extent be split
and the last extent to append write directly may occur conflict because the position may have the splited extent and effective,
thus we need make a additional jugement for GetEndForAppendW, if split happend then not reuse the extent.
update: add space log for datanode

.Snapshot mod append on tiny extent should use replay extent that may be new

.origin design use the end of old tiny extent id while modAppend, but that process not use get and put ways,
and it's not safe if parrelle. still can be hanged if parrellel already use the tiny extent id

.update tiny modAppend process from modAppend to tinyAppendWrite
.tiny write directly need with follower addrs or else repair later but read may not get it
.inode ek be truncated considered as split 2) code optimizition on createLowerVersion

.extent key be splited and be dropped through extDelCh and freelist, multi ext key belong to one extent
the eks before the last one should just the make a hole on the extent,but the last one should delete the whole extent.

Signed-off-by: leonrayang <chl696@sina.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
916cafd7a8 audit log: update lcnode idle status when processing heartbeat response, fix readdir bugs 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-08-30 20:05:32 +08:00
leonrayang
2f519ff21d enhance: protect verlist info metanode,decommission support snapshot,enhance 2phase version create 
1)reconstruct and clean code
2)fix bugs....

Signed-off-by: leonrayang <chl696@sina.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
e23010ff62 snapshot version deletion support 
Signed-off-by: Patrick Wu <wuchunhuan@gmail.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
c1bd413ab4 optimised S3 lifecycle scanning, mutable concurrent lcnodes, scanning report to grafana 
Signed-off-by: Patrick Wu <wuchunhuan@gmail.com>
 2023-08-30 20:05:32 +08:00
Patrick Wu
29e3110a93 S3 lifecycle 
Signed-off-by: Patrick Wu <wuchunhuan@gmail.com>
 2023-08-30 20:05:32 +08:00