Downloading data for the Dartbrains online course

[choi]

Hi,

I am trying out this online material on neuroimaging using python, Dartbrains (Course Overview — DartBrains) by Luke Chang from Dartmouth.

I got stuck from pretty much the beginning, where I needed to download the open example data, which is from the Pinel Localizer task (Download Data — DartBrains)

I’ve cloned the data by using Datalad as instructed in the course,

but, when I tried to load a specific nii.gz file using this code,

it didn’t work with this error message.

All the relevant software is up-to-date, including Datalad, python, git.

and the mode of the file looked okay, too.

Can anyone let me know what I should try to fix this?
Any pointers would be much appreciated!

Thanks!
Mingyeong

[Steven]

Hi, it looks like only the Git identifier for the file was downloaded, based on the small memory usage.

Can you try to cd to the directory you downloaded those files and run datalad get <filename>?

Best,
Steven

[choi]

Hi Steven,

Thank you so much!

Actually, that’s where I got confused. I did datalad get many times, but kept getting the same error message and the memory size of the file did not change…

Mingyeong

[yarikoptic]

@choi

Actually, that’s where I got confused. I did datalad get many times, but kept getting the same error message and the memory size of the file did not change…

we would need to see the output what you get while you did that… e.g. do datalad drop and then datalad get on one of those .nii.gz and share output. Also output of datalad wtf -D html_details pasted here could be of great help.

[choi]

Hi Yaroslav,

Thank you so much!

Here are the datalad drop, get, and datalad wtf -D html_details results.
What’s weird is that when I did datalad drop, I got an error message, even though I already datalad get on the file…

[yarikoptic]

NB recommendation for the future – in cases of output from terminal, best to just copy/paste that output as text placing it within “Preformatted text”. That would allow to easily copy/paste specific paths etc.

oh – the filesystem you are using there is exfat. It does not support the critical by git-annex (used by datalad to track data files) feature – symlinks. “In principle” things should still work through “adjusted branch” mode (hence you are in “adjusted/master(unlocked)” branch ATM) but we keep running into issues… Is there a chance you could use e.g. your HOME or other location which is not on exfat?

[choi]

Gosh… I’m so sorry. You’re right. This is my first time posting a coding question. Definitely do that from now on! Thanks so much for the heads up!

And, it worked! I changed to /Users/ directory and redid clone & get commands and put it in the data instance. Also, the file type in the filesystem has been changed to apfs from exFAT.

- filesystem: 
    - CWD: 
      - max_pathlength: 1024
      - mount_opts: ro,local,rootfs,dovolfs,journaled,multilabel
      - path: /Users/mchoi/dartbrain/data/Localizer/derivatives/fmriprep/sub-S01/func
      - type: apfs

I tried to use datalad on my external hard drive before, and it seems it uses exFAT file system, unlike my mac home directory.

Can I ask you one more followup question? Does this mean that datalad primarily does not work on external hard drives or flash drives? I might be wanting to use datalad directly on external hard drives to save memory spaces on my laptop. Is there anyone who uses datalad directly on external hard drives on mac?

[yarikoptic]

as for “does not work” the proper answer should be: “it should work but it would be painful and inefficient”

As for the other question(s) - if you format external drive to use some “modern” POSIX filesystem which has symlinks and locking (e.g. as ext3 or ext4 or btrfs on linux or that may be apfs on OSX) – would be working just fine but you might have trouble using that drive across different operating systems. May be @eknahm had some experiences recently?

edit: I also asked on git-annex website to possibly get an ultimate answer.

[choi]

I see. I am looking forward to the git-annex’s answer, too.
Thank you so much for your explanation. I really appreciate it.
It helped me a ton (actually solved the issue!)!