Sorry for the confusion–the first couple times I tried this, I thought I was using 1.4.1rc4, but turns out underneath I was still actually using 1.4.1rc2… 
so very very sorry for the waste of time! Several of these errors above were fixed by that change, including the trouble with using --skull-strip-template MNIPediatricAsym:cohort-2 vs --skull-strip-template MNIPediatricAsym:res-2:cohort-2. It does seem to select res-1 by default now.
However, the error that persists is the OSError: [Errno 28] No space left on device, which happens after the boilerplate and after downloading some templates successfully at the beginning. Are they downloaded to somewhere else later in the workflow in tpl_select? I assume it doesn’t have anything to do with nodes needing more threads than are available?
singularity exec --cleanenv fmriprep.simg python -c "from templateflow.conf import TF_HOME; print(TF_HOME)"
yields
/home/fmriprep/.cache/templateflow
But that dir doesn’t exist outside the container that I can tell:
(base) [utooley@chead Ursula]$ ls /home/fmriprep/
ls: cannot access /home/fmriprep/: No such file or directory
(base) [utooley@chead Ursula]$ ls /home/utooley/fmriprep
ls: cannot access /home/utooley/fmriprep: No such file or directory
Full log trace just in case it’s helpful:
Making sure the input data is BIDS compliant (warnings can be ignored in most cases).
1: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)
Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.
Summary: Available Tasks: Available Modalities:
1175 Files, 32.59GB rest T1w
105 - Subjects number T2w
1 - Session nback bold
TODO: full task name for rest events
TODO: full task name for number
If you have any questions, please post on https://neurostars.org/tags/bids.
190717-16:30:25,949 nipype.workflow IMPORTANT:
Running fMRIPREP version 1.4.1rc4:
* BIDS dataset path: /mnt.
* Participant list: ['CBPD0142'].
* Run identifier: 20190717-163025_cd31c52c-0946-4a90-8ff3-cb475f521884.
Downloading https://templateflow.s3.amazonaws.com/tpl-MNIPediatricAsym/cohort-2/tpl-MNIPediatricAsym_cohort-2_res-1_T1w.nii.gz
4.44kB [00:00, 9.91kB/s]
Downloading https://templateflow.s3.amazonaws.com/tpl-MNIPediatricAsym/cohort-2/tpl-MNIPediatricAsym_cohort-2_res-1_desc-brain_mask.nii.gz
160B [00:00, 7.58kB/s]
190717-16:30:29,818 nipype.workflow IMPORTANT:
Creating bold processing workflow for "/mnt/sub-CBPD0142/func/sub-CBPD0142_task-rest_run-01_bold.nii.gz" (0.07 GB / 67 TRs). Memory resampled/largemem=0.27/0.34 GB.
190717-16:30:29,828 nipype.workflow IMPORTANT:
No single-band-reference found for sub-CBPD0142_task-rest_run-01_bold.nii.gz
Downloading https://templateflow.s3.amazonaws.com/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-02_desc-fMRIPrep_boldref.nii.gz
1.71kB [00:00, 10.3kB/s]
Downloading https://templateflow.s3.amazonaws.com/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-02_desc-brain_mask.nii.gz
29.0B [00:00, 4.49kB/s]
190717-16:30:32,562 nipype.workflow IMPORTANT:
Slice-timing correction will be included.
190717-16:30:32,586 nipype.workflow WARNING:
SDC: no fieldmaps found or they were ignored (/mnt/sub-CBPD0142/func/sub-CBPD0142_task-rest_run-01_bold.nii.gz).
Downloading https://templateflow.s3.amazonaws.com/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_desc-carpet_dseg.nii.gz
441B [00:00, 9.65kB/s]
190717-16:30:33,831 nipype.workflow IMPORTANT:
Creating BOLD surface-sampling workflow.
190717-16:30:39,108 nipype.workflow IMPORTANT:
Works derived from this fMRIPrep execution should include the following boilerplate:
Results included in this manuscript come from preprocessing
performed using *fMRIPrep* 1.4.1rc4
(@fmriprep1; @fmriprep2; RRID:SCR_016216),
which is based on *Nipype* 1.2.0
(@nipype1; @nipype2; RRID:SCR_002502).
Anatomical data preprocessing
: A total of 2 T1-weighted (T1w) images were found within the input
BIDS dataset.
All of them were corrected for intensity non-uniformity (INU)
with `N4BiasFieldCorrection` [@n4], distributed with ANTs 2.2.0 [@ants, RRID:SCR_004757].
The T1w-reference was then skull-stripped with a *Nipype* implementation of
the `antsBrainExtraction.sh` workflow (from ANTs), using MNIPediatricAsym
as target template.
Brain tissue segmentation of cerebrospinal fluid (CSF),
white-matter (WM) and gray-matter (GM) was performed on
the brain-extracted T1w using `fast` [FSL 5.0.9, RRID:SCR_002823,
@fsl_fast].
A T1w-reference map was computed after registration of
2 T1w images (after INU-correction) using
`mri_robust_template` [FreeSurfer 6.0.1, @fs_template].
Brain surfaces were reconstructed using `recon-all` [FreeSurfer 6.0.1,
RRID:SCR_001847, @fs_reconall], and the brain mask estimated
previously was refined with a custom variation of the method to reconcile
ANTs-derived and FreeSurfer-derived segmentations of the cortical
gray-matter of Mindboggle [RRID:SCR_002438, @mindboggle].
Volume-based spatial normalization to two standard spaces (MNIPediatricAsym, MNI152NLin2009cAsym) was performed through
nonlinear registration with `antsRegistration` (ANTs 2.2.0),
using brain-extracted versions of both T1w reference and the T1w template.
The following templates were selected for spatial normalization:
*MNI's unbiased standard MRI template for pediatric data from the 4.5 to 18.5y age range* [@mnipediatricasym, RRID:SCR_008796; TemplateFlow ID: MNIPediatricAsym], *ICBM 152 Nonlinear Asymmetrical template version 2009c* [@mni152nlin2009casym, RRID:SCR_008796; TemplateFlow ID: MNI152NLin2009cAsym].
Functional data preprocessing
: For each of the 1 BOLD runs found per subject (across all
tasks and sessions), the following preprocessing was performed.
First, a reference volume and its skull-stripped version were generated
using a custom methodology of *fMRIPrep*.
The BOLD reference was then co-registered to the T1w reference using
`bbregister` (FreeSurfer) which implements boundary-based registration [@bbr].
Co-registration was configured with nine degrees of freedom to account
for distortions remaining in the BOLD reference.
Head-motion parameters with respect to the BOLD reference
(transformation matrices, and six corresponding rotation and translation
parameters) are estimated before any spatiotemporal filtering using
`mcflirt` [FSL 5.0.9, @mcflirt].
BOLD runs were slice-time corrected using `3dTshift` from
AFNI 20160207 [@afni, RRID:SCR_005927].
The BOLD time-series, were resampled to surfaces on the following
spaces: *fsaverage5*.
The BOLD time-series (including slice-timing correction when applied)
were resampled onto their original, native space by applying
a single, composite transform to correct for head-motion and
susceptibility distortions.
These resampled BOLD time-series will be referred to as *preprocessed
BOLD in original space*, or just *preprocessed BOLD*.
The BOLD time-series were resampled into several standard spaces,
correspondingly generating the following *spatially-normalized,
preprocessed BOLD runs*: MNIPediatricAsym, MNI152NLin2009cAsym.
First, a reference volume and its skull-stripped version were generated
using a custom methodology of *fMRIPrep*.
Several confounding time-series were calculated based on the
*preprocessed BOLD*: framewise displacement (FD), DVARS and
three region-wise global signals.
FD and DVARS are calculated for each functional run, both using their
implementations in *Nipype* [following the definitions by @power_fd_dvars].
The three global signals are extracted within the CSF, the WM, and
the whole-brain masks.
Additionally, a set of physiological regressors were extracted to
allow for component-based noise correction [*CompCor*, @compcor].
Principal components are estimated after high-pass filtering the
*preprocessed BOLD* time-series (using a discrete cosine filter with
128s cut-off) for the two *CompCor* variants: temporal (tCompCor)
and anatomical (aCompCor).
tCompCor components are then calculated from the top 5% variable
voxels within a mask covering the subcortical regions.
This subcortical mask is obtained by heavily eroding the brain mask,
which ensures it does not include cortical GM regions.
For aCompCor, components are calculated within the intersection of
the aforementioned mask and the union of CSF and WM masks calculated
in T1w space, after their projection to the native space of each
functional run (using the inverse BOLD-to-T1w transformation). Components
are also calculated separately within the WM and CSF masks.
For each CompCor decomposition, the *k* components with the largest singular
values are retained, such that the retained components' time series are
sufficient to explain 50 percent of variance across the nuisance mask (CSF,
WM, combined, or temporal). The remaining components are dropped from
consideration.
The head-motion estimates calculated in the correction step were also
placed within the corresponding confounds file.
The confound time series derived from head motion estimates and global
signals were expanded with the inclusion of temporal derivatives and
quadratic terms for each [@confounds_satterthwaite_2013].
Frames that exceeded a threshold of 0.5 mm FD or 1.5 standardised DVARS
were annotated as motion outliers.
All resamplings can be performed with *a single interpolation
step* by composing all the pertinent transformations (i.e. head-motion
transform matrices, susceptibility distortion correction when available,
and co-registrations to anatomical and output spaces).
Gridded (volumetric) resamplings were performed using `antsApplyTransforms` (ANTs),
configured with Lanczos interpolation to minimize the smoothing
effects of other kernels [@lanczos].
Non-gridded (surface) resamplings were performed using `mri_vol2surf`
(FreeSurfer).
Many internal operations of *fMRIPrep* use
*Nilearn* 0.5.2 [@nilearn, RRID:SCR_001362],
mostly within the functional processing workflow.
For more details of the pipeline, see [the section corresponding
to workflows in *fMRIPrep*'s documentation](https://fmriprep.readthedocs.io/en/latest/workflows.html "FMRIPrep's documentation").
### References
pandoc-citeproc: reference mnipediatricasym not found
190717-16:30:52,989 nipype.workflow WARNING:
Some nodes demand for more threads than available (1).
Downloading https://templateflow.s3.amazonaws.com/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_desc-brain_mask.nii.gz
156B [00:00, 10.2kB/s]
Downloading https://templateflow.s3.amazonaws.com/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_T1w.nii.gz
59%|███████████████████████████████████████████████████████▍ | 7.89k/13.4k [00:00<00:00, 11.3kB/s]
190717-16:30:56,566 nipype.workflow WARNING:
[Node] Error on "fmriprep_wf.single_subject_CBPD0142_wf.anat_preproc_wf.anat_norm_wf.tpl_select" (/tmp/utooleyCBPD0142/fmriprep_wf/single_subject_CBPD0142_wf/anat_preproc_wf/anat_norm_wf/_template_MNI152NLin2009cAsym/tpl_select)
190717-16:30:56,569 nipype.workflow ERROR:
Node tpl_select.a1 failed to run on host chead.uphs.upenn.edu.
190717-16:30:56,624 nipype.workflow ERROR:
Saving crash info to /mnt/derivatives/fmriprep_test/fmriprep/sub-CBPD0142/log/20190717-163025_cd31c52c-0946-4a90-8ff3-cb475f521884/crash-20190717-163056-utooley-tpl_select.a1-ead924f9-336a-47bd-b588-a9ac70bd705a.txt
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 144, in _s3_get
f.write(data)
OSError: [Errno 28] No space left on device
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 316, in _send_procs_to_workers
self.procs[jobid].run(updatehash=updatehash)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 472, in run
result = self._run_interface(execute=True)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 563, in _run_interface
return self._run_command(execute)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_command
result = self._interface.run(cwd=outdir)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 375, in run
runtime = self._run_interface(runtime)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/utility/wrappers.py", line 144, in _run_interface
out = function_handle(**args)
File "<string>", line 5, in _get_template
File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/utils/misc.py", line 50, in get_template_specs
tpl_target_path = get_template(in_template, **template_spec)
File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 39, in get
_s3_get(filepath)
File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 144, in _s3_get
f.write(data)
OSError: [Errno 28] No space left on device
190717-16:30:58,821 nipype.workflow INFO:
[Node] Setting-up "fmriprep_wf.single_subject_CBPD0142_wf.func_preproc_task_rest_run_01_wf.bold_reference_wf.gen_ref" in "/tmp/utooleyCBPD0142/fmriprep_wf/single_subject_CBPD0142_wf/func_preproc_task_rest_run_01_wf/bold_reference_wf/gen_ref".
190717-16:30:58,823 nipype.workflow INFO:
[Node] Running "gen_ref" ("niworkflows.interfaces.registration.EstimateReferenceImage")
190717-16:31:01,310 nipype.interface INFO:
stderr 2019-07-17T16:31:01.310071:++ 3dvolreg: AFNI version=Debian-16.2.07~dfsg.1-5~nd16.04+1 (Jun 12 2017) [64-bit]
190717-16:31:01,310 nipype.interface INFO:
stderr 2019-07-17T16:31:01.310071:++ Authored by: RW Cox
190717-16:31:01,328 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903:*+ WARNING: If you are performing spatial transformations on an oblique dset,
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903: such as /tmp/utooleyCBPD0142/fmriprep_wf/single_subject_CBPD0142_wf/func_preproc_task_rest_run_01_wf/bold_reference_wf/gen_ref/slice.nii.gz,
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903: or viewing/combining it with volumes of differing obliquity,
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903: you should consider running:
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903: 3dWarp -deoblique
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903: on this and other oblique datasets in the same session.
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.328903: See 3dWarp -help for details.
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.329352:++ Oblique dataset:/tmp/utooleyCBPD0142/fmriprep_wf/single_subject_CBPD0142_wf/func_preproc_task_rest_run_01_wf/bold_reference_wf/gen_ref/slice.nii.gz is 14.095265 degrees from plumb.
190717-16:31:01,329 nipype.interface INFO:
stderr 2019-07-17T16:31:01.329928:++ Coarse del was 10, replaced with 7
