Downloading Part of Data from OpenNeuro with aws

1

Dear @ChrisGorgolewski,

When I use aws s3 sync --no-sign-request s3://openneuro.org/ds002895 ds002895-download/

the whole dataset downloaded (The entire dataset is very large). If I want only to download part of data for example dwi data of the dataset, how should I do, and how does this code change?

Best,

Mohammad

2

AWS CLI has --exclude and --include options that allows you to specify a file naming pattern to use to limit the files being downloaded. For example:

aws s3 sync --no-sign-request --exclude "*" --include "*_dwi.*" s3://openneuro.org/ds002895 ds002895-download/

More info at https://docs.aws.amazon.com/cli/latest/reference/s3/#use-of-exclude-and-include-filters.

1 Like