Add guidelines to list S3 public bucket folder / file structure about 2019hackordatasciencetemplate HOT 4 OPEN

I responded to this in the sagemaker PR: hackoregon/hackoregon-aws-infrastructure#61 (comment)

from 2019hackordatasciencetemplate.

@DingoEatingFuzz being able to list the data files will allow better collaboration between the different teams. The desired workflow is to do

$ aws s3 ls s3://hacko-data-archive

An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

I do not think this will be computationally intensive operation. Why wouldn't you want to allow access?

from 2019hackordatasciencetemplate.

It is not computationally intensive, and I don't mind if data scientists are given access tokens to use with the aws cli tool, the issue is around security.

Even though the S3 bucket is public, you'd have to know the URL you want in order to download anything. If I make the ListObjects operation available to the public, then anyone could crawl through the bucket and clone the whole thing.

I would like to eventually open this up, but not before doing an audit and making sure there is truly nothing in there that we would want exposed.

from 2019hackordatasciencetemplate.

resolution: @karenng-civicsoftware will gather a list of emails of the data scientists needing AWS accounts to list the files on the S3 buckets. Then share it with Michael to create the accounts.

from 2019hackordatasciencetemplate.