Commit Graph

25 Commits

Author SHA1 Message Date
yhjiango
c8e92fb0f0 feat(objectnode): support local and external audit log
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-11-01 14:46:14 +08:00
yhjiango
4401624f96 feat(objectnode): s3 sts and signature auth
1. sts federation token
2. signature auth code refactor

Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-08-30 20:05:32 +08:00
tangdeyi
50fc530c80 fix(objnode): file delete lock
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-08-11 16:39:13 +08:00
tangdeyi
9184d6fcd3 perf(objectnode): improve metadata operation performance
Signed-off-by: tangdeyi <deyitang94@gmail.com>
2023-07-20 10:58:50 +08:00
yhjiango
0a4830dd93 fix(objectnode): fix the list multipart uploads and dir object inaccessible
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
862fde677a fix(objectnode):
1. MultipartUpload with nonexistent uploadId return NoSuchUploadID
2. NoSuchBucket err handle

Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
yhjiango
0affc88413 fix(objectnode): fully compatible with s3
Signed-off-by: yhjiango <jiangyunhua@oppo.com>
2023-05-18 10:48:57 +08:00
tangdeyi
9f0d7810a0 feature: bucket policy fully compatible with s3
Signed-off-by: tangdeyi <tangdeyi@oppo.com>
2023-05-18 10:48:57 +08:00
leonrayang
9a30b9f884 update: update to CubeFS
Signed-off-by: leonrayang <chl696@sina.com>
2022-08-22 18:16:09 +08:00
leonrayang
2970e2c3d1 update. go import path change from chubaofs to cubefs
Signed-off-by: leonrayang <chl696@sina.com>
2022-01-25 18:17:02 +08:00
Mofei Zhang
5c7c9980eb
Fix: precondition parsing issue
Fix the parsing issue of the two preconditions of If-Match and
If-None-Match.

We found that the request sent through the AWS S3 SDK included
double quotes for the values of these two preconditions.

This part is not specified in the AWS S3 API Reference.

Affected APIs: HeadObject, GetObject

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-04-22 17:25:19 +08:00
Mofei Zhang
7a57837f74
Fix: third-party license information
Adjust the format of the license header information of the third-party
code used to make the format more compliant. The NOTICE file supplements
the third-party code NOTIC information and explains the use of the code.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-04-14 19:29:39 +08:00
Mofei Zhang
4edef106d5
Enhancement: add routing for unimplemented APIs
In view of the particularity of the AWS S3 protocol rules, additional
routes are implemented for unimplemented APIs to prevent users from
incorrectly routing to other APIs when requesting unimplemented APIs.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-04-08 11:21:25 +08:00
Mofei Zhang
e37abd0731
Enhancement: reduce blocking under concurrency
The blacklist mechanism has been added to the user store and volume
manager in the object node to solve the adverse effect of a large number
of invalid requests on system performance in a short period of time.

Stream requests to the user store and volume manager in the object node
in a hash bucket manner, reducing the blocking caused by read and write
locks under concurrency.

Abstracted the user store and added an implementation that does not
cache user information. This is enabled through the "strict"
configuration to provide sufficient real-time performance with cluster
information during interface compatibility testing.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-31 19:36:21 +08:00
Mofei Zhang
509d96c3d1
Enhancement: folder operations in S3 APIs
Supports folder operations in the same way as Amazon AWS S3.
  - Get folder meta info by HeadObject interface.
  - Create folder when header specifies Content-Type as
    application/directory and Content-Length as 0 on PutObject interface.
  - Delete an empty folder by DeleteObject interface.
  - The GetObject interface responds normally when the target is an
    existing directory.
  - Differentiate folders and files on the ListObject interface through the
    CommonPrefix property (already supported).

Following object storage interface haven been implemented folder operations:
  - HeadObject
  - GetObject
  - DeleteObject
  - DeleteObjects
  - ListObjects
  - ListObjectsV2

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-30 17:01:45 +08:00
Mofei Zhang
ddfd8b7c3f
Enhancement: improve error parsing for master APIs
Enhancement: improve error parsing for master APIs.
Fix: status code of DelteBucket ans DeleteObjects API.
Fix: url signature parsing of signature algorithm V2.
Add: s3tests framework for compatibility test.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-22 08:31:51 +08:00
Mofei Zhang
11d0a4ddf0
Refactor: reorganize permission and actions
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-17 00:11:01 +08:00
Mofei Zhang
d99f147e6a
Refacrot: move actions to command proto package
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-03-16 08:43:17 +08:00
Mervin
0a095a0012
Refactor: remove unused code
Signed-off-by: Mervin <mofei2816@gmail.com>
2020-03-10 11:55:30 +08:00
Mervin
fbbe1e0a7e
Refactor: param parse in object storage interface handlers
Signed-off-by: Mervin <mofei2816@gmail.com>
2020-03-10 00:49:13 +08:00
Mofei Zhang
aec409ffcf Refactor: using middleware for policy valication.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-02-27 20:36:28 +08:00
Mofei Zhang
4979d728b2 Add: configuration for signature ignore actions
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2020-02-26 00:00:32 +08:00
wenjia322
0ea6447516 feature: cache authnode-info in objectnode and check user policy before calling handle function
Signed-off-by: wenjia322 <buaa1214wwj@126.com>
2020-01-08 18:36:04 +08:00
Mofei Zhang
b4434486c9
fix: unit test building issues and log format pattern issues.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2019-12-20 18:19:01 +08:00
Mofei Zhang
d609fedb5c feature: object storage interface
Add ObjectNode provides S3-compatibile APIs.
Fusion Storage interface expose two interface (POSIX and S3-compatible) for file operation.

Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
2019-12-17 17:36:53 +08:00