Hi,
I downloaded the dataset ds000030 from OpenNeuro and I’m having trouble datalad-getting some of the files in several control subjects, namely sub-10912, 11061, 11122,10788,10171,10189,10321,11019,10206.
For example if I stand in the folder sub-10206 and I try to datalad get . , the following error arises :
sub-10206/func/sub-10206_task-stopsignal_bold.nii.gz (file) [download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable]
[ERROR ] download failed: Requested Range Not Satisfiable
| download failed: Requested Range Not Satisfiable
| download failed: Requested Range Not Satisfiable
| download failed: Requested Range Not Satisfiable [get(/data/datasets/UCLA_LA5_Study/sub-10206/func/sub-10206_task-rest_bold.nii.gz)]
get(error): sub-10206/func/sub-10206_task-rest_bold.nii.gz (file) [download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable]
[ERROR ] download failed: Requested Range Not Satisfiable
| download failed: Requested Range Not Satisfiable
| download failed: Requested Range Not Satisfiable
| download failed: Requested Range Not Satisfiable [get(/data/datasets/UCLA_LA5_Study/sub-10206/func/sub-10206_task-scap_bold.nii.gz)]
get(error): sub-10206/func/sub-10206_task-scap_bold.nii.gz (file) [download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable
download failed: Requested Range Not Satisfiable]
Following some lead on internet, I tried to datalad drop . and datalad get again, but the same error arises.
Thanks in advance for your help
1 Like
how did you “download” this dataset from openneuro? (just cut paste from command line)
please also share output of datalad wtf so we know what version of datalad and git-annex are involved.
FWIW – here is what I tried:
$> datalad clone ///openneuro/ds000030
...
$> cd ds000030
$> cd sub-10206
$> datalad get .
get(ok): sub-10206/func/sub-10206_task-rest_physio.tsv.gz (file) [from s3-PUBLIC...]
get(ok): sub-10206/anat/sub-10206_T1w.nii.gz (file) [from s3-PUBLIC...]
get(ok): sub-10206/func/sub-10206_task-rest_bold.nii.gz (file) [from s3-PUBLIC...]01<00:00, 16.3M Bytes/s]
get(ok): sub-10206/dwi/sub-10206_dwi.nii.gz (file) [from s3-PUBLIC...]
get(ok): sub-10206/func/sub-10206_task-bart_bold.nii.gz (file) [from s3-PUBLIC...]01<00:00, 18.3M Bytes/s]
get(ok): sub-10206/func/sub-10206_task-stopsignal_bold.nii.gz (file) [from s3-PUBLIC...]01, 14.6M Bytes/s]
get(ok): sub-10206/func/sub-10206_task-taskswitch_bold.nii.gz (file) [from s3-PUBLIC...]02, 11.3M Bytes/s]
get(ok): sub-10206/func/sub-10206_task-scap_bold.nii.gz (file) [from s3-PUBLIC...]
get(ok): sub-10206 (directory)
action summary:
get (ok: 9)
datalad get . 16.63s user 1.59s system 392% cpu 4.645 total
also do you know if may be you need some proxy configuration – can you wget or some other way download files on that computer?
Hi @yarikoptic,
Thanks a lot for your reply.
Please find enclosed the datalad wtf output as requested. datalad_wtf.txt (4.5 KB).
As I downloaded the dataset a long time ago, I couldn’t find again which command I used to download the dataset, but for sure it was either datalad clone or datalad install. Furthermore, I can wget files on that computer.
I see that in trying to replicate what I did, you could get all the files. There is two elements I have to explain in more details that might explain the discrepancy :
First, previously to executing datalad get in the folder sub-10206, I had interrupted a datalad get . at the highest level of the dataset (that is getting all the subjects from the dataset). I read this page that made me think this could have caused the issue, that is why I tried to datalad drop and datalad get again as mentioned in my first post.
Secondly, between datalad clone and datalad get of the dataset, we had to reset my computer because of hardware issues we are encountering. The SATA disk on which the dataset resides was not touched that’s why the data were still installed. However, I had to reinstall datalad after the reset.
I’m thus wondering if it’s because of one of the previous points that the datalad get in my computer got corrupted but not in yours ?
Thanks for the the WTF. You are running a quite old datalad - 0.12.4, many issues were fixed since then. git annex is also a bit aged (8.20200226) . I would recommend to upgrade both datalad and git-annex! You are using ubuntu, so if you enable neuro.debian.net and update system wide, or alternatively install conda and install within an environment there. More info: 3. Installation and configuration — The DataLad Handbook
Having said all the below, I finally googled and found download failed: Requested Range Not Satisfiable – fixed in 2020-11-19 so indeed upgrade might help.
it is easy to check if it is due to corrupted dataset – just do a fresh datalad clone into a location nearby and then try to get again.
dataset shouldn’t get corrupted really, but may be indeed a partial key was downloaded and then somehow failed to “continue” to download. You could run datalad clean in that dataset and try datalad get again. Also might be good to do straight git annex get --debug sub-10206/func/sub-10206_task-rest_physio.tsv.gz and see if any debug information is relevant.
1 Like
Thanks a lot for your help @yarikoptic. Updating DataLad and git-annex worked to solve the issue. I had tried already to updgrade DataLad using sudo apt-get install datalad, but I did not know I have to download a newer package using conda or neurodebian. For future visitors of this post, I used conda to do the update.