gian/cubefs
1
0
Fork 0
mirror of https://github.com/cubefs/cubefs.git synced 2026-08-02 02:00:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,190 Commits 48 Branches 52 Tags 378 MiB
master
Commit Graph

66 Commits

Author SHA1 Message Date
Victor1319
809d023e9e refactor(vendor): user smux in depends to replace pkg in vendor. #1000193713 
Signed-off-by: Victor1319 <zengxuewei@oppo.com>
 2025-08-11 17:21:16 +08:00
slasher
ecb0b49016 feat(util): add go ping for flash sdk 
. #22200563

Signed-off-by: slasher <mcq.sejust@gmail.com>
 2025-08-07 15:19:12 +08:00
slasher
297477a2ed chore(util): add util of bloom fliter 
. #22200490

Signed-off-by: slasher <mcq.sejust@gmail.com>
 2025-08-07 15:18:56 +08:00
xiejian
0c6d15042d fix(shardnode): 1.remove raft group when apply remove member 
avoid removed member still leader and clear raft data after all raft log applied
2. skip load disk which status is not normal in clustermgr
3.import gomonkey package

with: #1000176986

Signed-off-by: xiejian <xiejian3@oppo.com>
 2025-07-08 11:30:16 +08:00
slasher
917a57b2fd feat(rpc2): add package xxh3 
. github.com/zeebo/xxh3

@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
 2025-07-08 11:30:13 +08:00
Cloudstriff
f3a83ecb47 style(shardnode): code static check and style refactor 
with #22357426

Signed-off-by: Cloudstriff <chenjiongwendao@qq.com>
Signed-off-by: xiejian <xiejian3@oppo.com>
 2025-07-08 11:30:12 +08:00
xiejian
eb1a280bc0 refactor(shardnode): update grpc version 
with: #22409856 of #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
 2025-07-08 11:30:12 +08:00
xiejian
75bc515d0b fix(common): update kvstorev2 
with: #22409822 of #22357426

Signed-off-by: xiejian <xiejian3@oppo.com>
 2025-07-08 11:30:12 +08:00
tangdeyi
b7cade30e6 fix(clustermgr): basic clustermgr dev-branch for shardnode management 
1. fix refactor code and test problem
2. add clustermgr api about shardnode and space

. #22360410

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
 2025-07-08 11:30:12 +08:00
yhjiango
c8e92fb0f0 feat(objectnode): support local and external audit log 
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
 2023-11-01 14:46:14 +08:00
NaturalSelect
c753f80cac chore(all): add gopsutil dependence 
Signed-off-by: NaturalSelect <2145973003@qq.com>
 2023-08-30 20:05:32 +08:00
baijiaruo
b6019dd7df feat(ci): clear code scanning alert by Dockerfile 
Signed-off-by: baijiaruo <baijiaruo@126.com>
 2023-08-04 10:25:02 +08:00
slasher
24e766dd7b chore(package): remove jacobsa's daemonize in depends 
Signed-off-by: slasher <shenjie1@oppo.com>
 2023-06-20 15:00:32 +08:00
pengtianyue
a35b7dcf4a refactor(common): upgrade sarama version to 1.33 
Signed-off-by: pengtianyue <pengtianyue@oppo.com>
 2023-06-02 17:03:10 +08:00
yuxiaobo
ac75137295 build: the build script of blobstore is merged into cubefs 
1. rocksdb upgrade to 6.3.6
2. optimize the build scripts
3. fix build errors
4. add env variable for blobstore unit test
5. update git ignore file to ignore build and test result
6. modify open files limit for unit test

Signed-off-by: yuxiaobo <yuxiaobo@oppo.com>
Signed-off-by: slasher <shenjie1@oppo.com>
Signed-off-by: Victor1319 <834863182@qq.com>
 2023-05-23 15:24:53 +08:00
tianjiongzhang
adef8cf191 feat(client): add block cache sync. 
Signed-off-by: tianjiongzhang <236556116@qq.com>
 2023-03-04 16:13:35 +08:00
leonrayang
d792fb5ee2 update: update vendor referenced by blobstore and other lib 
Signed-off-by: leonrayang <chl696@sina.com>
 2022-04-13 15:57:55 +08:00
leonrayang
2a67ed4104 feature: enable go mod and update vendor 
1. add chubaofs-blobstore client
 2. move origin modified vendor code to depends
    1) raft
    2) spf13
    3) fuse
    3) jacobsa
    4) rocksdb

Signed-off-by: leonrayang <chl696@sina.com>
 2022-04-13 15:57:55 +08:00
Sheng Yong
fa72d3417f fix: Fix logger.Error format 
Fix the following error message:

2022-02-18 12:35:22,026 transport_heartbeat.go๐Ÿ’ฏ [ERROR] [heartbeatTransport] recive message from conn error
%!(EXTRA string=read tcp 192.168.0.31:17330->192.168.0.34:39830: read: connection reset by peer)
^^^^^^^^

Signed-off-by: Sheng Yong <shengyong2021@gmail.com>
 2022-02-18 18:13:05 +08:00
leonrayang
bb3e70cc88 fix.in function of follower read,leader and follower sync message through raft log, if entry too large to send 
This bug lead send dp partitions cache from leader to follower be hang and follower recive update expired.
follower will redirect client/partitions request to leader,this can be easy tigger when dp partitions is large such as 10MB.
stable and unstable log entry send in different ways,unstage will send 1 entry if entry large then config.
but stable entries dealling method don't do that and print no message.
leader send to follower dp partititions which make a entry which is large then config size,in some senorio entry is in stable entires
fix bug use the ways of unstable entries dealling.

Signed-off-by: leonrayang <chl696@sina.com>
 2021-12-14 14:37:46 +08:00
Victor1319
dfb3f91f30 fix: process heartbeat req & resp in async way to avoid dead lock 
Signed-off-by: Victor1319 <834863182@qq.com>
 2021-12-13 19:41:25 +08:00
Victor1319
0906ea75ed enhance: support disk, datanode, metanode decommission 
Signed-off-by: Victor1319 <834863182@qq.com>
 2021-12-13 19:26:45 +08:00
leonrayang
7429730e76 fix: remove redundant memcopy when reading raft snapshot 
This commit removes the redundant memcopy when reading raft snapshot.
The redundant memcopy makes reading snapshot very slow which leads to
snapshot send failure.

Signed-off-by: leonrayang <chl696@sina.com>
Reviewd-by: Shuoran Liu <shuoranliu@gmail.com>
 2021-11-17 15:56:41 +08:00
Victor1319
539d3bfc7b enhance: check size when read data from heartbeat port 
This commit blocks messages of which the size is larger than 256M
to avoid consuming abnormally large memory due to network port scanners.

Signed-off-by: Victor1319 <834863182@qq.com>
 2021-09-15 12:16:41 +08:00
Victor1319
1cf43fb24e enhance: import prometheus push vendor 
Signed-off-by: Victor1319 <834863182@qq.com>
 2021-07-12 14:42:10 +08:00
Mervin
ffd148e745
Merge pull request #1093 from xrefft/bugfix/fix-raft-monitor-panic-error#1069 
Bugfix/fix raft monitor panic error#1069
 2021-02-04 18:31:13 +08:00
Shuoran Liu
b83ec35f79 vendor: introduce willf bitset package 
Signed-off-by: Shuoran Liu <shuoranliu@gmail.com>
 2021-01-28 14:37:33 +08:00
xrefft
39ff7ee12b
fix: raft monitor concurrency unsafe 
raft.monitor is not concurrency safe. So we introduce the Monitor interface as an alternative.

fix #1069

Signed-off-by: xrefft <cserxiao@qq.com>
 2021-01-25 15:36:39 +08:00
Mofei Zhang
abe776d33f Enhancement: auto fix crc-mismatched raft log crc 
The last Log file of Raft is in an unindexed state, and Raft will traverse
the file to rebuild the index during the startup process.

When the process was shut down abnormally last time (for example, shut down
by kill -9), the log file data may be incomplete.

This improvement allows raft to perform a truncation operation if an incomplete
log entry is encountered during the indexing process of the last log file,
and the log is truncated to the most recent complete and valid log entry.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
 2021-01-18 17:17:51 +08:00
Victor1319
0b663cd123 fix data node raftstatus's active value 
Signed-off-by: Victor1319 <834863182@qq.com>
 2020-12-28 15:33:16 +08:00
awzhgw
cf7fbde1f5
Revert "Sync from release-20200720" 2020-11-11 14:23:59 +08:00
awzhgw
b230b7107d Sync changes from release-20200720 branch. 
Signed-off-by: awzhgw <guowl18702995996@gmail.com>
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
 2020-11-09 16:03:42 +08:00
wenjia322
514852b540 enhancement: clean up the raft log when the remaining space is insufficient; limit size of raft logs and rotate 
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
 2020-10-13 18:29:09 +08:00
wenjia322
4a780bb323 fix: limit deleted raft log name 
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
 2020-09-18 18:30:43 +08:00
wenjia322
245720eefe enhancement: only keep raft logs from the past 7 days 
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
 2020-09-18 14:14:42 +08:00
awzhgw
ae1fdf3cf5 Refactor:add log on raftFSM ignored mesg 
Signed-off-by: awzhgw <guowl18702995996@gmail.com>
 2020-08-05 10:16:05 +08:00
ansjsun
f0c164523e add console node 
Signed-off-by: ansjsun <ansj-sun@163.com>
 2020-07-03 12:21:11 +08:00
awzhgw
7fcc849b40 Refactor: optimize delete file speed on dataNode 
Signed-off-by: awzhgw <guowl18702995996@gmail.com>
 2020-06-03 16:21:35 +08:00
zhuhyc
b8cee36807 fix : Redirect error log to file 
Signed-off-by: zhuhyc <zzhniy.163.niy@163.com>
 2020-05-26 10:07:16 +08:00
xuxihao
a8b4075791 enhancement of cli: implement several interfaces; support bash completion; set config file by interactive command line 
Signed-off-by: xuxihao <xuxihao3@jd.com>
 2020-05-12 22:58:08 +08:00
Mofei Zhang
1abdc584fe
Fix: resolve dependencies issue 
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
 2020-03-15 20:05:17 +08:00
Mervin
7fd7f4d5f4
Enhancement: adapt raft to more platform 
Signed-off-by: Mervin <mofei2816@gmail.com>
 2020-03-04 18:16:17 +08:00
awzhgw
d09a832544 Fix: metanode leak memory on DeleteMetaPartition operator 
Signed-off-by: awzhgw <guowl18702995996@gmail.com>
 2020-01-05 17:42:20 +08:00
Mofei Zhang
ea557ef830
merge with master branch and resolve conflict 
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
 2019-12-25 23:34:07 +08:00
Mofei Zhang
d609fedb5c feature: object storage interface 
Add ObjectNode provides S3-compatibile APIs.
Fusion Storage interface expose two interface (POSIX and S3-compatible) for file operation.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
 2019-12-17 17:36:53 +08:00
zhuzhengyi1
331b1b392b enhancement: remove gorequest pkg for license check 
Signed-off-by: zhuzhengyi1 <zhuzhengyi@jd.com>
 2019-12-13 16:31:53 +08:00
jzeng04
60a4977980 Add a general Authentication & Authorization framework for chubaofs. 
Signed-off-by: jzeng4 <jzeng04@gmail.com>
 2019-11-18 09:51:43 +08:00
awzhgw
ced4b8b924 Refactor: master, datanode and metanode Fix dp or mp offline process 
Signed-off-by: awzhgw <guowl18702995996@gmail.com>
 2019-11-12 17:27:14 +08:00
awzhgw
d5febd10da Fix: dataPartition disk error ,not recvoery raft log on new datanode 
Signed-off-by: awzhgw <guowl18702995996@gmail.com>
 2019-09-06 21:06:46 +08:00
Shuoran Liu
b15b78293b fix: fuse directIO read size can exceeds buffer size 
Since jacobasa fuse package uses a fixed-size memory pool for out
messages, it is important to tell fuse kernel module what the maximum
read size is. Otherwise, a large read will cause a panic.

However, max readahead size can only affect buffered read. The kernel
might still send a large directIO read in spite of the max readahead
size. We need to specify "max_read" option during mount.

Signed-off-by: Shuoran Liu <shuoranliu@gmail.com>
 2019-08-22 16:16:56 +08:00