Commit Graph

52 Commits

Author SHA1 Message Date
slasher
5c7898354f style(all): format all codes with golangci tools
closes #3371

@formatter:off

Signed-off-by: slasher <shenjie1@oppo.com>
2024-05-14 09:54:13 +08:00
yhjiango
135da4ec9f refactor(object): add CubeFS's statement comment
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2024-01-10 14:58:30 +08:00
lily-lee
a8620dc1e9 fix(code scan): fix code-scanning problems.
Signed-off-by: lily-lee <lilylee88756@gmail.com>
2023-11-28 12:32:15 +08:00
tangdeyi
af996a6299 fix(objectnode): s3api perm is compatible with fuse perm
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-11-13 15:12:30 +08:00
yhjiango
4401624f96 feat(objectnode): s3 sts and signature auth
1. sts federation token
2. signature auth code refactor

Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-08-30 20:05:32 +08:00
tangdeyi
e172d8b716 fix(objectnode): reduce fuse authentication check
Signed-off-by: tangdeyi <deyitang94@gmail.com>
2023-08-09 14:01:52 +08:00
yhjiango
2040ce00a9 fix(objectnode): fix bucket policy and location processing
1. Fix the problem of returning 500 internal errors in PutBucketPolicy.
2. Add related parameter verification in GetBucketLocation.

Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-07-20 10:58:50 +08:00
tangdeyi
9fcbc52cd4 fix(objectnode): policy check fix
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
yhjiango
d30cdd4dea fix(objectnode): ignore the non-existence error of dir acl
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
7df19f730a fix(objectnode): volume view cache
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
yhjiango
0a4830dd93 fix(objectnode): fix the list multipart uploads and dir object inaccessible
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
ac93148820 fix(objectnode): policy & acl check err handle
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
862fde677a fix(objectnode):
1. MultipartUpload with nonexistent uploadId return NoSuchUploadID
2. NoSuchBucket err handle

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
yhjiango
ac977dd5d5 fix(objectnode): modify the log print information
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-05-18 10:48:57 +08:00
yhjiango
0affc88413 fix(objectnode): fully compatible with s3
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
831ea00d5d fix(objectnode): enhance s3 compatibility
1.CORS supports preflight request;
2.S3-compatible err reply;
3.Small parts upload bugfix;
4.Specific status code when panic;
5.UploadPartCopy;

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
9f0d7810a0 feature: bucket policy fully compatible with s3
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
Patrick Wu
c8bf55ae69 S3 bugfix
Signed-off-by: Patrick Wu <wuchunhuan@oppo.com>
2023-03-04 16:13:35 +08:00
leonrayang
9a30b9f884 update: update to CubeFS
Signed-off-by: leonrayang <chl696@sina.com>
2022-08-22 18:16:09 +08:00
leonrayang
2970e2c3d1 update. go import path change from chubaofs to cubefs
Signed-off-by: leonrayang <chl696@sina.com>
2022-01-25 18:17:02 +08:00
zhuzhengyi001
4875a7009a Enhancement: add subdir authorized check for objectnode
Signed-off-by: zhuzhengyi001 <zhuzhengyi001@ke.com>
2021-08-27 19:29:23 +08:00
Shuoran Liu
1054c00fd8 feature: introduce subdir permissions
Originally the permission control for users are on volume level. This
commit introduces subdir level permission control.

Signed-off-by: Shuoran Liu <shuoranliu@gmail.com>
2020-08-14 20:48:05 +08:00
wenjia322
5b695c61d4 feature: add bucket acl/policy test cases
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-07-14 16:17:36 +08:00
wenjia322
65b3cc7742 fix: bucket policy marshal error
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-07-14 14:55:38 +08:00
wenjia322
4f1934e4ca fix: bucket acl xml marshal error
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-07-14 14:55:38 +08:00
awzhgw
36d77c14d2
Merge pull request #729 from wenjia322/no_cache_meta
refactor: whether to cache volume oss meta is determined by the configuration parameter 'strict'
2020-07-06 16:08:51 +08:00
Mofei Zhang
879e041f95
Fix: S3 response when volume do not exists
When initializing the volume, if the master explicitly responds that the
specified volume does not exist, it will not retry.

During the processing of the S3 request, both the signature and the
strategy are checked for the existence of the volume. NoSuchBucket
response is returned when the specified volume does not exist.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-07-03 20:34:06 +08:00
wenjia322
6cc377ae1c refactor: choose different 'MetaLoader' by configuration 'strict'
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-07-03 18:05:43 +08:00
zhuzhengyi1
d251bcb9d3 Fix: allow for bucket acl owner operation
Signed-off-by: zhuzhengyi1 <zhuzhengyi@jd.com>
2020-05-19 13:35:14 +08:00
zhuzhengyi1
e1b19601ed Enhancement: allow volome owener policy check when not deny
Signed-off-by: zhuzhengyi1 <zhuzhengyi@jd.com>
2020-04-27 16:24:32 +08:00
Mofei Zhang
43561c35b0
Enhancement: compatible with older version s3 credential
Added compatibility with 1.5 version of credential information for
signature verification and policy check of s3 interface.

In version 1.5, each volume has its own access key, which is stored
in the view information center of the volume to which it belongs.

In version 2.0, user security and authorization are added. Each user
has their own access key, and users can use this access key to operate
all their own volumes.

Increase the compatibility with the old version authentication method
to reduce the impact of online version 1.5 cluster upgrade to 2.0 on
online services.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-04-19 21:24:10 +08:00
Mofei Zhang
7a57837f74
Fix: third-party license information
Adjust the format of the license header information of the third-party
code used to make the format more compliant. The NOTICE file supplements
the third-party code NOTIC information and explains the use of the code.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-04-14 19:29:39 +08:00
zhuzhengyi1
2098c6c86f Fixed: update license
Signed-off-by: zhuzhengyi1 <zhuzhengyi@jd.com>
2020-04-14 16:44:23 +08:00
Mofei Zhang
e37abd0731
Enhancement: reduce blocking under concurrency
The blacklist mechanism has been added to the user store and volume
manager in the object node to solve the adverse effect of a large number
of invalid requests on system performance in a short period of time.

Stream requests to the user store and volume manager in the object node
in a hash bucket manner, reducing the blocking caused by read and write
locks under concurrency.

Abstracted the user store and added an implementation that does not
cache user information. This is enabled through the "strict"
configuration to provide sufficient real-time performance with cluster
information during interface compatibility testing.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-31 19:36:21 +08:00
Mofei Zhang
509d96c3d1
Enhancement: folder operations in S3 APIs
Supports folder operations in the same way as Amazon AWS S3.
  - Get folder meta info by HeadObject interface.
  - Create folder when header specifies Content-Type as
    application/directory and Content-Length as 0 on PutObject interface.
  - Delete an empty folder by DeleteObject interface.
  - The GetObject interface responds normally when the target is an
    existing directory.
  - Differentiate folders and files on the ListObject interface through the
    CommonPrefix property (already supported).

Following object storage interface haven been implemented folder operations:
  - HeadObject
  - GetObject
  - DeleteObject
  - DeleteObjects
  - ListObjects
  - ListObjectsV2

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-30 17:01:45 +08:00
Mofei Zhang
8b3d6aa3c5
Enhancement: multiple enhancements and fixes
Enhancement: validate user policy when mount volume.
Enhancement: when an object node loads a volume, different volumes do not block each other.
Enhancement: when checking policies, check user permissions before checking volume policies to reduce the overhead of loading volumes.
Enhancement: release connection pool when releasing meta wrapper resources.
Enhancement: the object node automatically releases the deleted volume.
Fix: response when deleting non-existing files using Object Storage
Interface.
Fix: url parameter parsing issue of signature algorithm V2.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-22 21:02:28 +08:00
wenjia322
745728a306 refactor: change index of 'UserInfo' from 'ak' to 'userID'
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-03-20 00:03:31 +08:00
Mofei Zhang
b49d89c490
Add: list user with filter command for CLI
Add: list user with filter command for CLI
Enhancement: performance of ListBuckets API
Fix: action identify issue

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-17 10:26:41 +08:00
Mofei Zhang
11d0a4ddf0
Refactor: reorganize permission and actions
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-17 00:11:01 +08:00
Mofei Zhang
d99f147e6a
Refacrot: move actions to command proto package
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-16 08:43:17 +08:00
Mofei Zhang
69f9018293
Add: CLI tool to improve cluster management
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-15 17:30:23 +08:00
Mervin
40d01f0661
Enhancement: support dot character in volume name
Signed-off-by: Mervin <mofei2816@gmail.com>
2020-03-10 20:33:14 +08:00
Mervin
fbbe1e0a7e
Refactor: param parse in object storage interface handlers
Signed-off-by: Mervin <mofei2816@gmail.com>
2020-03-10 00:49:13 +08:00
wenjia322
acaf24101d refactor: move util/oss/user.go to proto/oss_proto.go
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-03-04 14:58:12 +08:00
Mofei Zhang
a410319df2
Fix: policy and acl validation
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-03 18:54:10 +08:00
wenjia322
22944e11cc fix: call action.string
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-02-27 16:56:19 +08:00
wenjia322
f75cfc3f57 fix: revise 'actions' to 'action' in check user policy
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-02-27 16:45:47 +08:00
Mofei Zhang
4979d728b2 Add: configuration for signature ignore actions
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-02-26 00:00:32 +08:00
wenjia322
705a1f6d49 feature: add master api 'deleteVolPolicy'; revise code about authnode
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-02-11 17:12:11 +08:00
wenjia322
6c39ad9a01 fix: user policy check condition
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-01-09 14:11:17 +08:00