Support for AWS Signature Version 4.

English Support for Cloud Storage
divv
Posts: 5
Joined: Thu Apr 30, 2020 6:45 am

Support for AWS Signature Version 4.

Post by divv »

Hello,

When trying to upload to an S3 bucket with AWS KMS default encryption enabled (AES256 is expressly forbidden by policy), we receive the following error (formatting mine and info redacted):

Code: Select all

(Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Error>
	<Code>InvalidArgument</Code>
	<Message>Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.</Message>
	<ArgumentName>Authorization</ArgumentName>
	<ArgumentValue>null</ArgumentValue>
	<RequestId>XXXXXXXXXXX</RequestId>
	<HostId>XXXXXXXXXXXXXXX</HostId>
</Error>
at 0000000001660298.CP: TGGetOrPutEX:Bad Request<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><Message>Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.</Message><ArgumentName>Authorization</Argum)
Would it be possible to get the internal S3 client patched to support AWS Signature V4?
https://docs.aws.amazon.com/general/lat ... ion-4.html


The profile is configured for AWS S3 and the Server Side Encryption box is unchecked (because we forbid AES256, and we want to allow default encryption to kick in).

tobias
Posts: 320
Joined: Tue Mar 31, 2020 7:37 pm

Re: Support for AWS Signature Version 4.

Post by tobias »

Hello,
Syncovery automatically uses signature V4 when it has reason to believe it's necessary. Apparently you have a server side setting that Syncovery doesn't know about? Which Syncovery version do you have?

divv
Posts: 5
Joined: Thu Apr 30, 2020 6:45 am

Re: Support for AWS Signature Version 4.

Post by divv »

We are using v8.19, but I've also tested with v8.67a (latest) and 9.00rc2.

Interesting that we are getting this behaviour.


The S3 bucket is configured to use default encryption using a customer managed key.
We have a bucket policy that blocks AES256, and also only allows one kms-key-id to be used.
It works fine with the AWS CLI, we can use the following syntax:

Code: Select all

aws s3 cp file.txt s3://<bucket>/file.txt				        	# Default encryption will kick in
aws s3 cp file.txt s3://<bucket>/file.txt --sse aws:kms --sse-kms-key-id alias/<my_key>	# Specifying the correct KMS key
My expectation was that operation via Syncovery would be equivalent to the first case (i.e. no additional parameters).

tobias
Posts: 320
Joined: Tue Mar 31, 2020 7:37 pm

Re: Support for AWS Signature Version 4.

Post by tobias »

Hello,
Syncovery tries to avoid signature V4 because it is expensive (the source files have to be read twice to calulcate the hash).

If you tell Syncovery explicitly to use KMS, it should work.

To do that, please add the following lines to the Main section of the Syncovery.ini file. Like this:

AWSServerSideEncryptionType=aws:kms
AWSServerSideEncryptionKMSkeyid=arn:aws:kms:us-east-1:123456782225:key/12345678-1234-5678-abcde-123d456d2020

Replace the ID with your KMS key ID.

I will also fix this problem in the next V8/9 updates.

divv
Posts: 5
Joined: Thu Apr 30, 2020 6:45 am

Re: Support for AWS Signature Version 4.

Post by divv »

Hi Tobias,

That makes sense! Especially if you have an enormous amount of files...
We'll try modifying the .ini file.

Thanks for your help.

tobias
Posts: 320
Joined: Tue Mar 31, 2020 7:37 pm

Re: Support for AWS Signature Version 4.

Post by tobias »

Hi,
OK, let me know if it works - if not I'll provide an update that should work without Ini modification.

divv
Posts: 5
Joined: Thu Apr 30, 2020 6:45 am

Re: Support for AWS Signature Version 4.

Post by divv »

Hello,

Yes, updating the ini file worked, although it does make it challenging to deal with multiple S3 buckets. For now it looks like we could get away with it as only one S3 destination requires Server Side Encryption.

A fix would be appreciated, but perhaps non-urgent.

tobias
Posts: 320
Joined: Tue Mar 31, 2020 7:37 pm

Re: Support for AWS Signature Version 4.

Post by tobias »

Hello,
the new version 8.67b should now be able to recognize the need for Signature version 4 even without the additional lines in the Ini file. I am not 100% sure that it works in your set-up though but it should.

divv
Posts: 5
Joined: Thu Apr 30, 2020 6:45 am

Re: Support for AWS Signature Version 4.

Post by divv »

hi Tobias,

Many thanks for the new version, unfortunately we did not find any success with it.
Is there anything I can send you that might help to replicate the setup?

I could probably provide a sanitised CloudFormation template?
And is there a way to export and send you the Syncovery profile?

Regards,
David

tobias
Posts: 320
Joined: Tue Mar 31, 2020 7:37 pm

Re: Support for AWS Signature Version 4.

Post by tobias »

Hello,
yes if you could send instructions on how to set this up in the Amazon bucket or account, that would be great. I don’t normally work with AIM or policies myself.

Alternatively, you could turn on Internet Protocol Logging on the Logs tab sheet of the Program Settings dialog and send sanitized log files to support@syncovery.com.

The Internet Protocol logs are usually several per profile run and are named FTPLOGxxxxxxx. The temp folder can be found by entering %TEMP% in the address bar of Windows Explorer.

Post Reply