1) update master inteface of datanode enable qos
2) master clean client info if long time no report
3) enable qos online shutdown and startup
4) master set client alloc preriod and count of girds hit trigger condition
5) datanode qos limit value set by master and add disable conf and http interface
6) client qos init buffer calculate according on real buffer storage
7) create limiter for both hot and cold volume
Signed-off-by: leonrayang <chl696@sina.com>
Parse parameter 'encoding-type' from request. It this parameter
specified and match with 'url', then encode content keys by using url
encoding function.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
1. Fix: parallel-safe issue in conn pool and metanode conn serve
2. Fix: nil pointer issue when list parts for non-exist uploadId
3. Enhancement: validate read size while read from connnect.
4. Enhancement: clean up useless data on failure while upload part.
5. Enhancement: generate ETag if do not exist when listing.
6. Enhancement: include error message when formatting packet to string.
7. Enhancement: improve compatile with old version SDK.
8. Enhancement: success when retry add exist multipart part.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
Signed-off-by: 张墨飞 <zhangmofei@jd.com>
The PutObject interface supports setting user-defined metadata. The
GetObject and HeadObject interfaces can correctly return the stored
user-defined metadata.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
When ETag is stored, it is versioned by encoding, which can make
the file system interface and object storage interface more closely
integrated.
Referring to the Amazon AWS S3 Developer Guide:
"Amazon S3 response includes an ETag that uniquely identifies the
combined object data. This ETag will not necessarily be an MD5 hash
of the object data."
When a new object is written to ChubaoFS through the object storage
interface, ChubaoFS calculates the ETag value for it, and uses the
modification time of the object as its version information when
encoding and storing.
If this file is modified by the file system interface, the ETag of
this file will be invalid because the version information will be
inconsistent with modification time.
After the object storage interface finds that the ETag is invalid,
it will automatically generate a new ETag matching its modification
time.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
Adjust the format of the license header information of the third-party
code used to make the format more compliant. The NOTICE file supplements
the third-party code NOTIC information and explains the use of the code.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
Implemented the following conditional parameters in the GetObject
and HeadObject interfaces:
- If-Match
- If-Modified-Since
- If-None-Match
- If-Unmodified-Since
Signed-off-by: yuboLee <pangbolee@gmail.com>
In view of the particularity of the AWS S3 protocol rules, additional
routes are implemented for unimplemented APIs to prevent users from
incorrectly routing to other APIs when requesting unimplemented APIs.
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
Supports folder operations in the same way as Amazon AWS S3.
- Get folder meta info by HeadObject interface.
- Create folder when header specifies Content-Type as
application/directory and Content-Length as 0 on PutObject interface.
- Delete an empty folder by DeleteObject interface.
- The GetObject interface responds normally when the target is an
existing directory.
- Differentiate folders and files on the ListObject interface through the
CommonPrefix property (already supported).
Following object storage interface haven been implemented folder operations:
- HeadObject
- GetObject
- DeleteObject
- DeleteObjects
- ListObjects
- ListObjectsV2
Signed-off-by: Mofei Zhang <mofei2816@gmail.com>
