gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-02 02:00:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,677 Commits 48 Branches 52 Tags 378 MiB
d5bb45ea49
Commit Graph

108 Commits

Author SHA1 Message Date
leonrayang
d5bb45ea49 docs. update to cubefs 
Signed-off-by: leonrayang <chl696@sina.com>
 2023-05-26 11:17:10 +08:00
chihe
b3abd72524 [enhance] add default becache filter suffix 
Signed-off-by: chihe <chi.he@oppo.com>
 2023-05-18 10:48:57 +08:00
zhangtianjiong
90336d52db enhance(client):retry to load conf from master when mount 
Signed-off-by: zhangtianjiong <236556116@qq.com>
 2023-05-18 10:48:57 +08:00
Patrick Wu
dc7e5b9885 fix&enhance(meta):transaction bugfix and enhance 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-05-18 10:48:57 +08:00
Patrick Wu
d35fd047c9 feature(client.meata): transaction support 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-05-18 10:48:57 +08:00
leonrayang
108d7af81c enhance.create mount dir if not exist automiclly 
Signed-off-by: leonrayang <chl696@sina.com>
 2023-05-18 10:48:57 +08:00
leonrayang
b525c104a5 Feature: support ip addr acl check 
Signed-off-by: leonrayang <chl696@sina.com>
 2023-05-18 10:48:57 +08:00
Tao Li
6028627579 [Enhancemant] Support fuse request timeout 
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-03-30 13:03:28 +08:00
Patrick Wu
595b016017 fix: audit log prefix info will get lost when re-enabled after disabled 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-03-16 19:43:27 +08:00
tianjiongzhang
a4780b8cf2 enhance(client-blockcache): async read and cache. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2023-03-04 16:13:35 +08:00
Patrick Wu
490b051d6a audit log supports custom prefix, master add request info log 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-03-04 16:13:35 +08:00
Patrick Wu
de36b418cf audit log 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-03-04 16:13:35 +08:00
tianjiongzhang
adef8cf191 feat(client): add block cache sync. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2023-03-04 16:13:35 +08:00
tianjiongzhang
fa2cb61a57 enhance: reduce memory and threads consume for fuse client. 
Signed-off-by: tianjiongzhang <zhangtianjiong@oppo.com>
 2023-03-04 16:13:35 +08:00
tianjiongzhang
191670a99f enhance:optimize blockcache performance. 
1) change http over uds to tcp over uds.
2) add global dentry metadata cache.

Signed-off-by: tianjiongzhang <236556116@qq.com>
 2023-03-04 16:13:35 +08:00
Patrick Wu
5f34fd5102 fix: optimization for dir quota limitation 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-03-04 16:13:35 +08:00
Patrick Wu
f15e2a8d6d feature: add quota limitation for directory children number 
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
 2023-03-04 16:13:35 +08:00
liubingxing
7afd9232bd FIX: fix the wrong fuse subtype 
Signed-off-by: liubingxing <liubbingxing@gmail.com>
 2022-11-25 11:00:32 +08:00
Victor1319
3ec54232e9 fix: remove redundant code 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-10-11 16:15:34 +08:00
leonrayang
9a30b9f884 update: update to CubeFS 
Signed-off-by: leonrayang <chl696@sina.com>
 2022-08-22 18:16:09 +08:00
tianjiongzhang
78a875d05c enhance:disable enableBcache flag. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2022-08-19 15:05:33 +08:00
tianjiongzhang
b9dd0df310 fix:cold vol auto enableBcache flag. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2022-08-10 22:05:45 +08:00
leonrayang
3fdd01cae9 feature: add qos flow limit control process for 2nd cache layer 
1) update master inteface of datanode enable qos
2) master clean client info if long time no report
3) enable qos online shutdown and startup
4) master set client alloc preriod and count of girds hit trigger condition
5) datanode qos limit value set by master and add disable conf and http interface
6) client qos init buffer calculate according on real buffer storage
7) create limiter for both hot and cold volume

Signed-off-by: leonrayang <chl696@sina.com>
 2022-08-10 22:05:45 +08:00
tianjiongzhang
560516a8df enhance:add enableBcache to mountoptions. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2022-08-10 22:05:45 +08:00
tianjiongzhang
0bb82569e7 refactor:open streamer with block cache flag. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2022-08-10 22:05:45 +08:00
Victor1319
f91291619d fix: not open bcache and extent cache when disable metacache 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-08-10 22:05:45 +08:00
tianjiongzhang
d2d6d07138 feat:support local data cache for subdir 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2022-08-10 22:05:45 +08:00
Victor1319
06bb20bbd1 fix: fix meta cache bug 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-08-10 22:05:45 +08:00
Victor1319
a690508c81 feat: support cache meta info in client 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-08-10 22:05:45 +08:00
lixiang12407
3a221d0f06 fix:merge confict. 
Signed-off-by: zhangtianjiong <236556116@qq.com>
 2022-08-10 22:05:45 +08:00
Victor1319
0f0eb9d84a feat: set default push addr for oppo 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-08-10 22:05:45 +08:00
zhangtianjiong
8426459598 feature:support cache block encrypt. 
Signed-off-by: zhangtianjiong <hanhanztj@sina.com>
 2022-08-10 22:05:45 +08:00
Victor1319
32aae63423 feature: add default_permissions in mount option if enablePosixAcl is true 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-07-14 15:38:34 +08:00
leonrayang
cc406ae931 update: gofmt 
Signed-off-by: leonrayang <chl696@sina.com>
 2022-04-13 15:57:55 +08:00
chihe
b0fdefcd47 bugfix: file cache pool not work in hot vol 
Signed-off-by: chihe <chi.he@oppo.com>
 2022-04-13 15:57:55 +08:00
chihe
a4404a3056 bugfix:1.In cold volume, cache dp number seldom less than 10 
2.blobstoreWriter OOM
3.change FS name

Signed-off-by: chihe <chi.he@oppo.com>
 2022-04-13 15:57:55 +08:00
chihe
4c0b8ee1d3 bugfix:1.In cold volume, cache dp number seldom less than 10 
2.blobstoreWriter OOM
3.change FS name

Signed-off-by: chihe <chi.he@oppo.com>
 2022-04-13 15:57:55 +08:00
lixiang12407
878265671a bugfix: fix some cold volume bug 
1. remove ebs-access path check
2. update datapartition after opening a cold file
3. cold files use TinyExtent when offset + size <= 1M

Signed-off-by: lixiang12407 <960754123@qq.com>
 2022-04-13 15:57:55 +08:00
AmazingChi
884dfb4581 enhance: Limit buffer pool size 
Signed-off-by: AmazingChi <hechi1014@126.com>
 2022-04-13 15:57:55 +08:00
tianjiongzhang
94b2744275 feature: client support blobstore 
1. support read/write blobstore
2. multiple cache tier to acceleration data in datalake architecture

Signed-off-by: tianjiongzhang <236556116@qq.com>
Signed-off-by: bboyCH4 <hechi1014@126.com>
Signed-off-by: lixiang12407 <960754123@qq.com>
 2022-04-13 15:57:55 +08:00
leonrayang
2a67ed4104 feature: enable go mod and update vendor 
1. add chubaofs-blobstore client
 2. move origin modified vendor code to depends
    1) raft
    2) spf13
    3) fuse
    3) jacobsa
    4) rocksdb

Signed-off-by: leonrayang <chl696@sina.com>
 2022-04-13 15:57:55 +08:00
Victor1319
d67f9fae02 feature: add default_permissions in mount option if enablePosixAcl is true 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-04-01 11:20:36 +08:00
leonrayang
2970e2c3d1 update. go import path change from chubaofs to cubefs 
Signed-off-by: leonrayang <chl696@sina.com>
 2022-01-25 18:17:02 +08:00
wanglei469
c608deb971 change: append volume to client log path 
When multi client mount on one machine, one client log will be
overwritten by another. To avoid this, we should separate the log path
of each client. To implement this, i attach volume name to the logDir in
the config file.

Besides, previous default logDir is root. I set the default logDir to
/var/log/chubaofs

Signed-off-by: wanglei469 <wanglei469@ke.com>
 2022-01-20 11:48:53 +08:00
Sheng Yong
10ad80f495 fuse: Tell parent error message if error occurs in child 
daemonize uses `SignalOutcome' to pass child's error to parent.

Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
 2022-01-19 16:16:42 +08:00
Sheng Yong
5c20bc510d fuse: Integrate fdstore into cfs-client 
This patch integrate fdstore into cfs-client, and adds 2 more options
for cfs-client:
 -p <port>: pprof port of old cfs-client
 -n: dynamic unix domain socket file name

If we are trying to live upgrade cfs-client, the following command
could be used:

	cfs-client -c <config json file path> -p <pprof port> -r

Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
 2022-01-14 12:54:21 +08:00
Sheng Yong
9392566a4b fuse: Add restore 
This patch adds a new option '-r [UDS]' for restore cfs-client's fuse.
If restore is enabled, cfs-client will not mount a new fuse. Instead,
it receive fuse dev fd from UDS, and load serveNodes and serveHandles
from locale files. After that, cfs-client continue reading requests
from /dev/fuse.

Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
 2022-01-14 12:53:54 +08:00
Sheng Yong
98ac080f2a fuse: Add suspend/resume 
This patch addes two http handler "/suspend" and "/resume".

**suspend**
Suspend has one paremeter, which is a Unix Domain Socket used to send
fd. Suspend stops reading more request from /dev/fuse. It waits all
handling requests to finish, then saves serveNodes and serveHandles to
locale files ("/tmp/ChubaoFS-fuse-Nodes.list" and "/tmp/ChubaoFS-fuse-
Handles.list" by default). After that, cfs-client exits.

**resume**
Resume could stop Suspend before cfs-client exists.

Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
 2022-01-14 12:52:02 +08:00
Sheng Yong
447e213949 fuse: Wait ProfPort ready for use 
This patch adds `waitListenAndServe' to make sure ProfPort is not
in use before timeout (1 min by default). If something goes wrong,
an error is returned to the main goroutine through a channel.

However, if the specific pprof port is in use after timeout, try
choosing a port automatically by system. And warning messages are
printed to show the new port.

Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
 2022-01-14 12:51:50 +08:00
Victor1319
8cd1d1c9cd feature:print pid to file when server started 
Signed-off-by: Victor1319 <834863182@qq.com>
 2022-01-05 14:09:09 +08:00