Hello,
I am new to C-PAC and fMRI data preprocessing. Currently I’m trying to preprocess ABIDE-II to get ROI signals, similar to this one for ABIDE-I: ABIDE Preprocessed
Following documentation here: C-PAC Quickstart - C-PAC 1.8.8.dev1 Beta documentation, I installed all the cpac docker image, and ran the following command:
cpac run s3://fcp-indi/data/Projects/ABIDE2/RawData ./test participant
Then I got this output:
Loading 🐳 Docker
Loading 🐳 fcpindi/c-pac:latest as "joezhao (501)" with these directory bindings:
local Docker mode
----------------------------------------- ----------------------------------------- ------
/etc/passwd /etc/passwd ro
/Users/joezhao/.cpac /Users/joezhao/.cpac rw
/Users/joezhao/Desktop/abide-prep /Users/joezhao/Desktop/abide-prep rw
/Users/joezhao/Desktop/abide-prep/test /Users/joezhao/Desktop/abide-prep/test rw
/Users/joezhao/Desktop/abide-prep/log /Users/joezhao/Desktop/abide-prep/log rw
/Users/joezhao/Desktop/abide-prep/working /Users/joezhao/Desktop/abide-prep/working rw
/Users/joezhao/Desktop/abide-prep/outputs /Users/joezhao/Desktop/abide-prep/outputs rw
Logging messages will refer to the Docker paths.
Skipping bids-validator for S3 datasets...
#### Running C-PAC
Number of participants to run in parallel: 1
Input directory: s3://fcp-indi/data/Projects/ABIDE2/RawData
Output directory: /Users/joezhao/Desktop/abide-prep/test/output
Working directory: /Users/joezhao/Desktop/abide-prep/test/working
Log directory: /Users/joezhao/Desktop/abide-prep/test/log
Remove working directory: True
Available memory: 1.0 (GB)
Available threads: 1
Number of threads for ANTs: 1
Parsing s3://fcp-indi/data/Projects/ABIDE2/RawData..
Connecting to AWS: fcp-indi anonymously...
gathering files from S3 bucket (s3.Bucket(name='fcp-indi')) for data/Projects/ABIDE2/RawData
Did not find data in s3://fcp-indi/data/Projects/ABIDE2/RawData
How can I use minimal default preprocessing of C-PAC to preprocess ABIDE-II and get ROI time-series?
Any suggestions will be really appreciated!! 
Some more details:
While the log says Did not find data in s3:…, I verified the link with
aws s3 ls fcp-indi/data/Projects/ABIDE2/RawData/
And I got this:
PRE ABIDEII-BNI_1/
PRE ABIDEII-EMC_1/
PRE ABIDEII-ETHZ_1/
PRE ABIDEII-GU_1/
PRE ABIDEII-IP_1/
PRE ABIDEII-IU_1/
PRE ABIDEII-KKI_1/
PRE ABIDEII-KUL_3/
PRE ABIDEII-NYU_1/
PRE ABIDEII-NYU_2/
PRE ABIDEII-OHSU_1/
PRE ABIDEII-ONRC_2/
PRE ABIDEII-SDSU_1/
PRE ABIDEII-TCD_1/
PRE ABIDEII-UCD_1/
PRE ABIDEII-UCLA_1/
PRE ABIDEII-UCLA_Long/
PRE ABIDEII-UPSM_Long/
PRE ABIDEII-USM_1/
PRE sidecards/
Hi @joe_zhao, could you provide us with the pipeline config you’re using for this run?
There could also be a naming issue occurring here, which could cause the subject folders in RawData to not be pulled in correctly. Can you confirm that the names of these folders are PRE ABIDEII-*, with a space after ‘PRE’? You may need to remove that space in order for the file names to be read in correctly.
Hello @tamsinrogers,
Thanks for your reply and help! I tried this command and it starts running, for one testing site:
cpac run s3://fcp-indi/data/Projects/ABIDE2/RawData/ABIDEII-BNI_1 ./test participant
From the documentation, I think the pipeline I used is the default pre-config pipeline, as I did not specify that arg:
https://fcp-indi.github.io/docs/latest/user/pipelines/preconfig#default-the-default-pipeline
I have another question, how long does it usually take for processing one sample? Currently for sample sub-29006_ses-1, it runs for more than half an hour, and seems it gets stuck at after this log:
241121-09:34:25,378 nipype.workflow INFO:
[Node] Setting-up "cpac_sub-29006_ses-1.cpac_sub-29006_ses-1.ANTS_T1_to_template_57.anat_mni_ants_register.calc_ants_warp" in "/Users/joezhao/Desktop/abide-prep/test/working/pipeline_cpac-default-pipeline/cpac_sub-29006_ses-1/cpac_sub-29006_ses-1/ANTS_T1_to_template_57/anat_mni_ants_register/calc_ants_warp".
241121-09:34:25,427 nipype.workflow INFO:
[Node] Executing "calc_ants_warp" <nipype.interfaces.utility.wrappers.Function>
241121-09:53:53,469 nipype.workflow INFO:
[Node] Finished "calc_ants_warp", elapsed time 1167.826321s.
241121-09:53:53,519 nipype.workflow INFO:
[Node] Setting-up "cpac_sub-29006_ses-1.cpac_sub-29006_ses-1.ANTS_T1_to_template_symmetric_78.anat_mni_ants_register_symmetric.calc_ants_warp" in "/Users/joezhao/Desktop/abide-prep/test/working/pipeline_cpac-default-pipeline/cpac_sub-29006_ses-1/cpac_sub-29006_ses-1/ANTS_T1_to_template_symmetric_78/anat_mni_ants_register_symmetric/calc_ants_warp".
241121-09:53:53,545 nipype.workflow INFO:
[Node] Executing "calc_ants_warp" <nipype.interfaces.utility.wrappers.Function>
I used default configuration for computation resources:
Available memory: 1.0 (GB)
Available threads: 1
Number of threads for ANTs: 1
Again, thank you very much for your help!