Support S3 unsigned body #3323
Support S3 unsigned body #3323
Conversation
|
Detected 1 possible performance regressions:
|
gems/aws-sdk-core/lib/aws-sdk-core/plugins/checksum_algorithm.rb
Outdated
Show resolved
Hide resolved
richardwang1124
left a comment
richardwang1124
left a comment
There was a problem hiding this comment.
Nice! I think it looks good overall to me. Added a few comments.
gems/aws-sdk-core/lib/aws-sdk-core/plugins/checksum_algorithm.rb
Outdated
Show resolved
Hide resolved
gems/aws-sdk-core/lib/aws-sdk-core/plugins/checksum_algorithm.rb
Outdated
Show resolved
Hide resolved
| end | ||
|
|
||
| # Trailer implementation within JRUBY environment is facing some | ||
| # network issues that will need further investigation |
There was a problem hiding this comment.
maybe link to the possible open issue?
There was a problem hiding this comment.
I'm unsure if these two are related but I think they helped me to narrow down the issue to SSL so i'll link those two.
| @@ -208,6 +208,7 @@ def download_file(destination, bucket:, key:, **options) | |||
| # @option option [Integer] :http_chunk_size (16384) Size in bytes for each chunk when streaming request bodies | |||
| # over HTTP. Controls the buffer size used when sending data to S3. Larger values may improve throughput by | |||
| # reducing the number of network writes, but use more memory. Custom values must be at least 16KB. | |||
| # Only Ruby MRI is supported. | |||
There was a problem hiding this comment.
The value is still being set on the thread and the patch is being applied right? So the low level http chunk size would still be applied in both cruby and jruby?
There was a problem hiding this comment.
Yeah the value is still being set - I could check for JRUBY version here to ensure that it doesn't (unnecessary thread local variable). That being said, Checksum plugin checks at runtime to determine if the thread local need to be used or not.
| if @trailer_io | ||
| return @trailer_io.read(length, buf) | ||
| def read(length = nil, buf = nil) | ||
| return @io.read unless length |
There was a problem hiding this comment.
I'm a little confused about this - this means we won't use AWS chunked encoding when we're not given a length?
There was a problem hiding this comment.
I think I was trying to make set of specific tests to pass.
An actual fix will be if length is not given, I should resort to using the minimum chunk size. I also made some edits to fix the expectations on those tests as well.
3 participants
This PR intends to improve performance for S3's
PutObjectandUploadPart.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
To make sure we include your contribution in the release notes, please make sure to add description entry for your changes in the "unreleased changes" section of the
CHANGELOG.mdfile (at corresponding gem). For the description entry, please make sure it lives in one line and starts withFeatureorIssuein the correct format.For generated code changes, please checkout below instructions first:
https://github.com/aws/aws-sdk-ruby/blob/version-3/CONTRIBUTING.md
Thank you for your contribution!