Add anonymous access option to example config about scpca-nf HOT 7 CLOSED

I asked the user to do some testing, and interestingly the following worked as expected:

$ aws s3 ls s3://scpca-references/homo_sapiens/ensembl-104/salmon_index/Homo_sapiens.GRCh38.104.spliced_intron.txome --no-sign-request
              PRE Homo_sapiens.GRCh38.104.spliced_intron.txome/

However, this did not:

$ aws s3 ls s3://scpca-references/homo_sapiens/ensembl-104/salmon_index/Homo_sapiens.GRCh38.104.spliced_intron.txome

An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

In other words, a completely anonymous request was able to get a listing, but when sending credentials, access was denied. This does not look like a bad credentials issue, as I think those get a different issue.

I tried adding list permissions to the Authenticated users group (anyone with an AWS account) as well, but that doesn't seem to have made any difference (it was a shot in the dark).

(Side note: I also checked (on my own) that adding --recursive to a request with --no-sign-request worked, which is what will be needed later.)

If anyone with more AWS experience can have a look and maybe provide some thoughts, that would be appreciated. Tagging @arkid15r and @davidsmejia to start.

from scpca-nf.

From the issue description the process seems to be working as intended. As (I assume) the user isn't a part of our AWS account the authenticated requests don't work. And the publicly accessible bucket content should be available with the --no-sign-request option.

Adding the Authenticated users group (anyone with an AWS account) could be an alternative (needs to be tested with/without the Everyone (public access) ACL enabled).

from scpca-nf.

Correct, the user is not part of our AWS account, but I had assumed (perhaps incorrectly) that allow everyone access would include authenticated users.
But just in case I did add the Authenticated users group (anyone with an AWS account) option, and I tested it with a profile of my own (outside ALSF) and it seemed to work as expected (I could list the bucket+prefix). So I am still confused as to why this user is getting AccessDenied

from scpca-nf.

I wonder if it is possible to pass to nextflow an option to use a NULL AWS profile somehow? Somehow the profile being used is blocking access, but we would not want to override AWS profiles by default...

from scpca-nf.

Nextflow has a setting that we might be able to use for anonymous S3 access:
aws.client.anonymous = true

I have asked for testing of this, and we will see if it works

from scpca-nf.

Nextflow has a setting that we might be able to use for anonymous S3 access: aws.client.anonymous = true

I have asked for testing of this, and we will see if it works

This was tested successfully! Updating issue to reflect that this should be added to the example config file.

from scpca-nf.

closed by #206

from scpca-nf.