TL;DR: Can fmriprep work on a dataset whose bids organization is a symlink overlay? If yes, can you help me troubleshoot my issue?
I have created a symlink overlay of a subset of the ukbiobank dataset. I used this code as a starting point for creating the overlay, then customized it to my own file system’s organization.
Where input is the symlinked bids dataset, container is the fmriprep 20.2.0 singularity container, s is the subject ID, and so on.
When running this on a sample subject, I receive the error:
fmriprep: error: Path does not exist: <input>.
Where input is replaced by the directory I fed it above.
The directory certainly exists and the permissions are in order for me to access it. I have run the above command on another dataset that is not symlinked and the command works fine.
When you call Singularity, make sure you bind the symlinked drive to the singularity container. For example, if your data is stored in /path/to/my/BIDS/, then add -B /path to the singularity call. To be safe, do this for both the drive where your symlinks live, and where the actual files are (if they are not on the same drive). Does that make sense?
When you run singularity, you are essentially entering a new terminal that only has the software distributed with the container. You bring nothing to it (except maybe your home ~ drive), including any data on your machine. For singularity to find data on the machine, it has to be explicitly linked to the container, which is what -B does.
To address your error, /data is a relative path, not an absolute path. That is, from where you are running the code, /data is a folder, but if you are anywhere else, /data does not exist. Mount the full path. Also, you only need to mount the highest up drive. So, again if it’s something like /Highest_drive/path/to/data/bids_symlink, then you only need -B /Highest_drive
Hmmm, okay. Can you try a test for me? Enter the fmriprep singularity image in your terminal: singularity shell -B /data $container and see if you can navigate to your data in there?
Also, I see the TMPDIR may not be properly bound to the singularity container either.
Also, as a stylistic tip which you may or may not find helpful, I usually bind the drives and rename them in the singularity container. So I will bind my output directory (usually /path/to/BIDS/derivatives) with the name “output” in the container.
singularity shell -B /data $container worked, I can see all the directories in /data
But singularity shell -B /data/UKBB:input $container gave me the same error. So it looks like there’s an issue with the /data/UKBB drive that contains bids_symlink, it might not be an issue with the symlinked directory.