1. rocksdb upgrade to 6.3.6
2. optimize the build scripts
3. fix build errors
4. add env variable for blobstore unit test
5. update git ignore file to ignore build and test result
6. modify open files limit for unit test
Signed-off-by: yuxiaobo <yuxiaobo@oppo.com>
Signed-off-by: slasher <shenjie1@oppo.com>
Signed-off-by: Victor1319 <834863182@qq.com>
This patch integrate fdstore into cfs-client, and adds 2 more options
for cfs-client:
-p <port>: pprof port of old cfs-client
-n: dynamic unix domain socket file name
If we are trying to live upgrade cfs-client, the following command
could be used:
cfs-client -c <config json file path> -p <pprof port> -r
Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
This patch adds a new option '-r [UDS]' for restore cfs-client's fuse.
If restore is enabled, cfs-client will not mount a new fuse. Instead,
it receive fuse dev fd from UDS, and load serveNodes and serveHandles
from locale files. After that, cfs-client continue reading requests
from /dev/fuse.
Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
This patch addes two http handler "/suspend" and "/resume".
**suspend**
Suspend has one paremeter, which is a Unix Domain Socket used to send
fd. Suspend stops reading more request from /dev/fuse. It waits all
handling requests to finish, then saves serveNodes and serveHandles to
locale files ("/tmp/ChubaoFS-fuse-Nodes.list" and "/tmp/ChubaoFS-fuse-
Handles.list" by default). After that, cfs-client exits.
**resume**
Resume could stop Suspend before cfs-client exists.
Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
LookupRequest gets new child, the child's NodeID should be saved in the
serveHandle but not parent's NodeID.
Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
This bug lead send dp partitions cache from leader to follower be hang and follower recive update expired.
follower will redirect client/partitions request to leader,this can be easy tigger when dp partitions is large such as 10MB.
stable and unstable log entry send in different ways,unstage will send 1 entry if entry large then config.
but stable entries dealling method don't do that and print no message.
leader send to follower dp partititions which make a entry which is large then config size,in some senorio entry is in stable entires
fix bug use the ways of unstable entries dealling.
Signed-off-by: leonrayang <chl696@sina.com>
This commit removes the redundant memcopy when reading raft snapshot.
The redundant memcopy makes reading snapshot very slow which leads to
snapshot send failure.
Signed-off-by: leonrayang <chl696@sina.com>
Reviewd-by: Shuoran Liu <shuoranliu@gmail.com>
This patch introduces an Interface HandleReadDirer, which implements
ReadDir to read partial sub-dirents accroding to request offset and
size.
From now on, readdir could be implemented in two ways:
* ReaddirAll reads all sub-dirents one time
* Readdir reads partial sub-dirents and the caller needs to call
it multple times if more sub-dirents are requested
* If both are implemented, Readdir is used
Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
This commit blocks messages of which the size is larger than 256M
to avoid consuming abnormally large memory due to network port scanners.
Signed-off-by: Victor1319 <834863182@qq.com>
The last Log file of Raft is in an unindexed state, and Raft will traverse
the file to rebuild the index during the startup process.
When the process was shut down abnormally last time (for example, shut down
by kill -9), the log file data may be incomplete.
This improvement allows raft to perform a truncation operation if an incomplete
log entry is encountered during the indexing process of the last log file,
and the log is truncated to the most recent complete and valid log entry.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
The kernel is sending Forget requests without waiting for the responses,
so it is not limited by the value of FUSE_DEFAULT_MAX_BACKGROUND. And
there is a chance that kernel sends tons of Forget requests in a very
short time which runs out of memory.
This commit limits the rate of dealing with Forget requests to optimize
memory consumption.
Signed-off-by: Shuoran Liu <shuoranliu@gmail.com>
