Comments (4)
I responded to this in the sagemaker PR: hackoregon/hackoregon-aws-infrastructure#61 (comment)
from 2019hackordatasciencetemplate.
@DingoEatingFuzz being able to list the data files will allow better collaboration between the different teams. The desired workflow is to do
$ aws s3 ls s3://hacko-data-archive
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
I do not think this will be computationally intensive operation. Why wouldn't you want to allow access?
from 2019hackordatasciencetemplate.
It is not computationally intensive, and I don't mind if data scientists are given access tokens to use with the aws cli tool, the issue is around security.
Even though the S3 bucket is public, you'd have to know the URL you want in order to download anything. If I make the ListObjects operation available to the public, then anyone could crawl through the bucket and clone the whole thing.
I would like to eventually open this up, but not before doing an audit and making sure there is truly nothing in there that we would want exposed.
from 2019hackordatasciencetemplate.
resolution: @karenng-civicsoftware will gather a list of emails of the data scientists needing AWS accounts to list the files on the S3 buckets. Then share it with Michael to create the accounts.
from 2019hackordatasciencetemplate.
Related Issues (18)
- Polish Python docker image
- Host tutorial section explaining how to sync up Google Colab and local machine for HackO data science work
- Build Docker image and dockerfile for R users
- Figure out how to track data lineage metadata in Postgres DB / RDS HOT 1
- Polish story card request template to guide data processing / analyses
- Sync up with backend team to come up with recommendations for storing & retrieving data from RDS
- Figure out a recommended tracking system for data science workflow
- Check if RStudio Cloud free edition is a sustainable cloud computing solution for R users
- First check-up on team transportation
- Example story card 1
- refine data & API flowchart
- Fix security vulnerability for this repo in requirements.txt file
- help team transportation to put together a platform request for monitoring their db storage quota HOT 1
- Come up with code example of how to use Metacat with a postgres Docker image
- understand how to make it easier to work with storybook APIs and Python / R code
- explore housing tax lot data processing options with Stephen
- review story card data science codes
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 2019hackordatasciencetemplate.