copy data from amazon s3 account to your cluster -凯发k8网页登录

  • documentation
main content

copy data from amazon s3 account to your cluster

tip

if you have data stored in an amazon s3 bucket, then you can use datastores in matlab to directly access the data without needing any storage on the cluster. for instructions on how to transfer s3 data from any aws account, see . the alternative method to download s3 data using cloud center described below is simpler. however, it comes with the limitation that you can only transfer s3 data from the aws account authorized in cloud center.

to transfer individual files from an amazon s3 bucket to the cluster machines, on the create cluster screen, next to amazon s3 data, click add files. specify which files you want to make available to your cluster machines. you can specify s3 files only when creating your cluster and starting it for the first time.

when the cluster starts up, before the mjs process starts, specified files are copied to /shared/imported on the cluster’s shared file system. see . if any of the files is in gzip, tar, or zip file format, they are automatically expanded in /shared/imported.

note

transferring a large amount of data from your amazon s3 account can cause the cluster to time out during its startup. if your data size exceeds approximately 5 gb, start your cluster without the s3 data transfer, then upload the necessary data to the cluster /shared/persisted folder from a local drive, as described in .

related topics

    网站地图