How to locate the s3 bucket about neurocaas HOT 6 CLOSED

Going to reopen this until I finish documenting s3 usage in the developer docs.

from neurocaas.

Hi Shuonan,

The bucket itself will be specified in your "stack_config_template.json" file- the field "PipelineName" is exactly the name of the created s3 bucket the fields indicated here are paths within that bucket. You should be able to find the bucket named after your PipelineName in S3. Having run Step 6 of the developer guide, you should then be able to find directories within this bucket, with "inputs/", "configs/" and "submissions/" subdirectories, where your input data, configuration files, and submit.json files should go.

from neurocaas.

Thank you, step 6 run successfully with the following information:

Successfully packaged artifacts and wrote output template to file compiled_packaged.yaml.
Execute the following command to deploy the packaged template
sam deploy --template-file ../neurocaas/ncap_iac/ncap_blueprints/bardensr/compiled_packaged.yaml --stack-name <YOUR STACK NAME>


	Deploying with following values
	===============================
	Stack name                 : bardensr
	Region                     : None
	Confirm changeset          : False
	Deployment s3 bucket       : None
	Capabilities               : ["CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND"]
	Parameter overrides        : {}

However, under bardensr I wasn't able to find the 'inputs/' 'configs/' etc:

ls -a ./bardensr/
.				.ipynb_checkpoints		stack_config_template.json
..				compiled_packaged.yaml		stack_config_template.json~
.DS_Store			compiled_template.json		test.ipynb
.aws-sam		        test_resources

am i missing something? thank you!

from neurocaas.

Hi Shuonan,

In order to list the contents of an S3 bucket, please use the aws cli as follows:
aws s3 ls [your bucket name] --recursive
You can likewise use this cli to upload/download files https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html or folders https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html. I will document this more clearly in the developer docs!

from neurocaas.

thank you, that was helpful, closing the issue

from neurocaas.

Closed in newest version of developer docs: "End Goal" section.

from neurocaas.