Hi NeuroStars community,
We have a bit of a problem with a large dataset on OpenNeuro. There were some file check errors because the upload got interrupted. I was able to resolve those, but now I cannot create a new version anymore. When I click on “Add” under “New Changelog”, I get “ApolloError: Bad Request”.
I tried this with Chrome (Build 154.0.8037.58) and Safari (Version 27.0, Build 22625.1.29.11.27) and got the same issue.
Is there anything I can do from my side to resolve this?
Hi Joe. It looks like you deleted a critical file (.gitattributes) at some point, which is going to result in poorly-controlled data management. Would it make sense for me to reset to the latest version and then you can retry updating with a normal .gitattributes?
Did you have issues with the CLI uploader? I’m seeing direct commits by Claude Code with dubious commit messages like “OpenNeuro cannot handle symlinks” that proceed to delete a large number of fairly important FreeSurfer files.
Hi Chris, thank you for your help.
There were quite a few issues with the upload – ~30 pages worth. I tried to make sense of it and thought there were two categories, namely links in the Freesurfer subdirectories and large files from fMRIPrep that might not have uploaded completely. I used Claude Code to compare the data on OpenNeuro with our local copy to remove links and re-upload missing or incomplete files. Usually, I would have been a bit more careful, but we are under time pressure with a revision of an accompanying data description paper.
It would be great if you could reset it and I’ll try uploading again from scratch. Do you have any tips for uploading a large dataset to avoid similar issues the second time around?
@effigies Dear Chris, sorry to keep pestering you. Would you mind resetting the dataset to the latest version? Then, I can try uploading the data again. Thank you in advance!
Apologies for the delay. The dataset has been reset to 1.0.0.
Sorry, it’s not quite resolved yet, I’m afraid. I uploaded the BIDS dataset without the derivatives and created a new version, which seems to have worked. Then, I wanted to add the derivatives folders one by one. However, when I try uploading anything now, I get errors like this:
error: Uncaught (in worker "") (in promise) PermissionDenied: Permission denied (os error 13): copy '/home/ASDPrecision/data/bids/sub-025/dwi/sub-025_acq-PA_dwi.bvec' -> '/tmp/openneuro_cli_openneuro.org_d28bc39cd2c55677/ds007182/sub-025/dwi/sub-025_acq-PA_dwi.bvec'
This file was part of the first upload and looks complete on OpenNeuro.
The command that I used was just this: openneuro upload --dataset ds007182 .
The CLI version I’m using is 4.42.0
MANAGED BY INCF