Hi @karofinc,
Thank you again for highlighting fmridenoise in a previous thread. I have now made further progress applying it to my own data. However, when I run the following command:
python -m fmridenoise -p ‘/data/project/BEACON/wk_directories/fmridenoise/pipelines/pipeline-24HMP_aCompCor_SpikeReg_4GS.json’ -g ‘/data/project/BEACON/wk_directories/wk_tom_bids/’
I end up with an error message as follows:
/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/sklearn/externals/joblib/init.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
190919-15:37:37,18 nipype.workflow DEBUG:
adding multipath trait: conf_json
190919-15:37:37,19 nipype.workflow DEBUG:
adding multipath trait: conf_raw
190919-15:37:37,19 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,20 nipype.workflow DEBUG:
adding multipath trait: conf_prep
190919-15:37:37,20 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,20 nipype.workflow DEBUG:
adding multipath trait: fmri_prep
190919-15:37:37,20 nipype.workflow DEBUG:
adding multipath trait: fmri_prep_aroma
190919-15:37:37,22 nipype.workflow DEBUG:
adding multipath trait: fmri_denoised
190919-15:37:37,24 nipype.workflow DEBUG:
adding multipath trait: group_conf_summary
190919-15:37:37,24 nipype.workflow DEBUG:
adding multipath trait: group_corr_mat
190919-15:37:37,25 nipype.workflow DEBUG:
Converted the join node Merge field edges_weight trait type from a legal value to any value
190919-15:37:37,25 nipype.workflow DEBUG:
Converted the join node Merge field edges_weight_clean trait type from a legal value to any value
190919-15:37:37,25 nipype.workflow DEBUG:
Converted the join node Merge field exclude_list trait type from a legal value to any value
190919-15:37:37,25 nipype.workflow DEBUG:
Converted the join node Merge field fc_fd_summary trait type from a legal value to any value
190919-15:37:37,26 nipype.workflow DEBUG:
Converted the join node ReportCreator field pipelines trait type from a legal value to a legal value
190919-15:37:37,26 nipype.workflow DEBUG:
Converted the join node ReportCreator field pipelines_names trait type from a legal value to a string
190919-15:37:37,26 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,26 nipype.workflow DEBUG:
adding multipath trait: in_file
190919-15:37:37,27 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,27 nipype.workflow DEBUG:
adding multipath trait: in_file
190919-15:37:37,27 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,28 nipype.workflow DEBUG:
adding multipath trait: in_file
190919-15:37:37,28 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,28 nipype.workflow DEBUG:
adding multipath trait: in_file
190919-15:37:37,29 nipype.workflow DEBUG:
adding multipath trait: entities
190919-15:37:37,29 nipype.workflow DEBUG:
adding multipath trait: in_file
190919-15:37:37,32 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.Denoiser): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.Denoiser): new edge data: {‘connect’: [(‘tr_dict’, ‘tr_dict’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.Denoiser): Edge data exists: {‘connect’: [(‘tr_dict’, ‘tr_dict’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.Denoiser): new edge data: {‘connect’: [(‘tr_dict’, ‘tr_dict’), (‘fmri_prep’, ‘fmri_prep’), (‘fmri_prep_aroma’, ‘fmri_prep_aroma’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.Denoiser): Edge data exists: {‘connect’: [(‘tr_dict’, ‘tr_dict’), (‘fmri_prep’, ‘fmri_prep’), (‘fmri_prep_aroma’, ‘fmri_prep_aroma’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.Denoiser): new edge data: {‘connect’: [(‘tr_dict’, ‘tr_dict’), (‘fmri_prep’, ‘fmri_prep’), (‘fmri_prep_aroma’, ‘fmri_prep_aroma’), (‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ConfPrep): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ConfPrep): new edge data: {‘connect’: [(‘conf_raw’, ‘conf_raw’), (‘conf_json’, ‘conf_json’), (‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_confounds): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_confounds): new edge data: {‘connect’: [(‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_denoise): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_denoise): new edge data: {‘connect’: [(‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_connectivity): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_connectivity): new edge data: {‘connect’: [(‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_carpet_plot): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_carpet_plot): new edge data: {‘connect’: [(‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_matrix_plot): No edge data
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.BidsGrabber, fmridenoise_wf.ds_matrix_plot): new edge data: {‘connect’: [(‘entities’, ‘entities’)]}
190919-15:37:37,33 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ConfPrep): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ConfPrep): new edge data: {‘connect’: [(‘pipeline’, ‘pipeline’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.Denoiser): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.Denoiser): new edge data: {‘connect’: [(‘pipeline’, ‘pipeline’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.GroupConf): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.GroupConf): new edge data: {‘connect’: [(‘conf_summary’, ‘conf_summary’), (‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_denoise): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_denoise): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_connectivity): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_connectivity): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_confounds): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_confounds): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_carpet_plot): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_carpet_plot): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_matrix_plot): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ds_matrix_plot): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.Denoiser): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.Denoiser): new edge data: {‘connect’: [(‘conf_prep’, ‘conf_prep’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.Denoiser, fmridenoise_wf.ConnCalc): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.Denoiser, fmridenoise_wf.ConnCalc): new edge data: {‘connect’: [(‘fmri_denoised’, ‘fmri_denoised’)]}
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.GroupConn): No edge data
190919-15:37:37,34 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.GroupConn): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.GroupConn): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.GroupConn): new edge data: {‘connect’: [(‘corr_mat’, ‘corr_mat’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.ds_confounds): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConfPrep, fmridenoise_wf.ds_confounds): new edge data: {‘connect’: [(‘conf_prep’, ‘in_file’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.Denoiser, fmridenoise_wf.ds_denoise): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.Denoiser, fmridenoise_wf.ds_denoise): new edge data: {‘connect’: [(‘fmri_denoised’, ‘in_file’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.ds_connectivity): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.ds_connectivity): new edge data: {‘connect’: [(‘corr_mat’, ‘in_file’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.ds_carpet_plot): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.ds_carpet_plot): new edge data: {‘connect’: [(‘carpet_plot’, ‘in_file’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.ds_matrix_plot): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.ConnCalc, fmridenoise_wf.ds_matrix_plot): new edge data: {‘connect’: [(‘matrix_plot’, ‘in_file’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.GroupConn, fmridenoise_wf.QualityMeasures): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.GroupConn, fmridenoise_wf.QualityMeasures): new edge data: {‘connect’: [(‘pipeline_name’, ‘pipeline_name’), (‘group_corr_mat’, ‘group_corr_mat’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.GroupConf, fmridenoise_wf.QualityMeasures): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.GroupConf, fmridenoise_wf.QualityMeasures): new edge data: {‘connect’: [(‘group_conf_summary’, ‘group_conf_summary’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.QualityMeasures, fmridenoise_wf.Merge): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.QualityMeasures, fmridenoise_wf.Merge): new edge data: {‘connect’: [(‘fc_fd_summary’, ‘fc_fd_summary’), (‘edges_weight’, ‘edges_weight’), (‘edges_weight_clean’, ‘edges_weight_clean’), (‘exclude_list’, ‘exclude_list’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.Merge, fmridenoise_wf.PipelinesQC): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.Merge, fmridenoise_wf.PipelinesQC): new edge data: {‘connect’: [(‘fc_fd_summary’, ‘fc_fd_summary’), (‘edges_weight’, ‘edges_weight’), (‘edges_weight_clean’, ‘edges_weight_clean’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.Merge, fmridenoise_wf.ReportCreator): No edge data
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.Merge, fmridenoise_wf.ReportCreator): new edge data: {‘connect’: [(‘exclude_list’, ‘excluded_subjects’)]}
190919-15:37:37,35 nipype.workflow DEBUG:
(fmridenoise_wf.PipelinesQC, fmridenoise_wf.ReportCreator): No edge data
190919-15:37:37,36 nipype.workflow DEBUG:
(fmridenoise_wf.PipelinesQC, fmridenoise_wf.ReportCreator): new edge data: {‘connect’: [(‘plot_pipeline_edges_density’, ‘plot_pipeline_edges_density’), (‘plot_pipelines_edges_density_no_high_motion’, ‘plot_pipelines_edges_density_no_high_motion’), (‘plot_pipelines_fc_fd_pearson’, ‘plot_pipelines_fc_fd_pearson’), (‘plot_pipelines_fc_fd_uncorr’, ‘plot_pipelines_fc_fd_uncorr’), (‘plot_pipelines_distance_dependence’, ‘plot_pipelines_distance_dependence’)]}
190919-15:37:37,36 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ReportCreator): No edge data
190919-15:37:37,36 nipype.workflow DEBUG:
(fmridenoise_wf.PipelineSelector, fmridenoise_wf.ReportCreator): new edge data: {‘connect’: [(‘pipeline’, ‘pipelines’), (‘pipeline_name’, ‘pipelines_names’)]}
190919-15:37:37,170 nipype.workflow DEBUG:
Creating flat graph for workflow: fmridenoise_wf
190919-15:37:37,174 nipype.workflow DEBUG:
expanding workflow: fmridenoise_wf
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.PipelineSelector
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.BidsGrabber
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ConfPrep
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ds_confounds
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.Denoiser
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ds_denoise
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ConnCalc
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ds_matrix_plot
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ds_carpet_plot
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ds_connectivity
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.GroupConn
190919-15:37:37,175 nipype.workflow DEBUG:
processing node: fmridenoise_wf.GroupConf
190919-15:37:37,176 nipype.workflow DEBUG:
processing node: fmridenoise_wf.QualityMeasures
190919-15:37:37,176 nipype.workflow DEBUG:
processing node: fmridenoise_wf.Merge
190919-15:37:37,176 nipype.workflow DEBUG:
processing node: fmridenoise_wf.PipelinesQC
190919-15:37:37,176 nipype.workflow DEBUG:
processing node: fmridenoise_wf.ReportCreator
190919-15:37:37,176 nipype.workflow DEBUG:
finished expanding workflow: fmridenoise_wf
190919-15:37:37,176 nipype.workflow INFO:
Workflow fmridenoise_wf settings: [‘check’, ‘execution’, ‘logging’, ‘monitoring’]
190919-15:37:37,179 nipype.workflow DEBUG:
PE: expanding iterables
190919-15:37:37,179 nipype.workflow DEBUG:
Detected iterable nodes [fmridenoise_wf.PipelineSelector]
190919-15:37:37,180 nipype.workflow DEBUG:
Expanding the iterable node fmridenoise_wf.PipelineSelector…
190919-15:37:37,180 nipype.workflow DEBUG:
Excised the fmridenoise_wf.QualityMeasures -> fmridenoise_wf.Merge join node in-edge.
190919-15:37:37,180 nipype.workflow DEBUG:
Excised the fmridenoise_wf.Merge -> fmridenoise_wf.ReportCreator join node in-edge.
190919-15:37:37,180 nipype.workflow DEBUG:
Excised the fmridenoise_wf.PipelinesQC -> fmridenoise_wf.ReportCreator join node in-edge.
190919-15:37:37,180 nipype.workflow DEBUG:
Excised the fmridenoise_wf.PipelineSelector -> fmridenoise_wf.ReportCreator join node in-edge.
190919-15:37:37,180 nipype.workflow DEBUG:
node: fmridenoise_wf.PipelineSelector iterables: {‘pipeline_path’: <function _standardize_iterables..make_field_func.. at 0x7f1a53945268>}
190919-15:37:37,180 nipype.workflow DEBUG:
(‘subnodes:’, [fmridenoise_wf.PipelineSelector, fmridenoise_wf.ConfPrep, fmridenoise_wf.GroupConf, fmridenoise_wf.QualityMeasures, fmridenoise_wf.Denoiser, fmridenoise_wf.ConnCalc, fmridenoise_wf.GroupConn, fmridenoise_wf.ds_connectivity, fmridenoise_wf.ds_carpet_plot, fmridenoise_wf.ds_matrix_plot, fmridenoise_wf.ds_denoise, fmridenoise_wf.ds_confounds])
190919-15:37:37,183 nipype.workflow DEBUG:
[Node] PipelineSelector - setting input pipeline_path = /data/project/BEACON/wk_directories/fmridenoise/pipelines/pipeline-24HMP_aCompCor_SpikeReg_4GS.json
190919-15:37:37,183 nipype.workflow DEBUG:
Parameterization: paramstr=pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json
190919-15:37:37,184 nipype.workflow DEBUG:
The join node fmridenoise_wf.Merge input fmridenoise_wf.QualityMeasures was expanded to 1 nodes.
190919-15:37:37,184 nipype.workflow DEBUG:
Added the fmridenoise_wf.Merge join item fields {‘fc_fd_summary’: ‘fc_fd_summaryJ1’, ‘edges_weight’: ‘edges_weightJ1’, ‘edges_weight_clean’: ‘edges_weight_cleanJ1’, ‘exclude_list’: ‘exclude_listJ1’}.
190919-15:37:37,184 nipype.workflow DEBUG:
Qualified the fmridenoise_wf.QualityMeasures -> fmridenoise_wf.Merge join field fc_fd_summary as fc_fd_summaryJ1.
190919-15:37:37,184 nipype.workflow DEBUG:
Qualified the fmridenoise_wf.QualityMeasures -> fmridenoise_wf.Merge join field edges_weight as edges_weightJ1.
190919-15:37:37,184 nipype.workflow DEBUG:
Qualified the fmridenoise_wf.QualityMeasures -> fmridenoise_wf.Merge join field edges_weight_clean as edges_weight_cleanJ1.
190919-15:37:37,184 nipype.workflow DEBUG:
Qualified the fmridenoise_wf.QualityMeasures -> fmridenoise_wf.Merge join field exclude_list as exclude_listJ1.
190919-15:37:37,184 nipype.workflow DEBUG:
Connected the join node fmridenoise_wf.Merge subgraph to the expanded join point fmridenoise_wf.QualityMeasures
190919-15:37:37,185 nipype.workflow DEBUG:
The join node fmridenoise_wf.ReportCreator input fmridenoise_wf.PipelinesQC was expanded to 1 nodes.
190919-15:37:37,185 nipype.workflow DEBUG:
The join node fmridenoise_wf.ReportCreator input fmridenoise_wf.PipelineSelector was expanded to 1 nodes.
190919-15:37:37,185 nipype.workflow DEBUG:
Added the fmridenoise_wf.ReportCreator join item fields {‘pipelines’: ‘pipelinesJ1’, ‘pipelines_names’: ‘pipelines_namesJ1’}.
190919-15:37:37,185 nipype.workflow DEBUG:
Connected the join node fmridenoise_wf.ReportCreator subgraph to the expanded join point fmridenoise_wf.PipelinesQC
190919-15:37:37,185 nipype.workflow DEBUG:
Qualified the fmridenoise_wf.PipelineSelector -> fmridenoise_wf.ReportCreator join field pipelines as pipelinesJ1.
190919-15:37:37,185 nipype.workflow DEBUG:
Qualified the fmridenoise_wf.PipelineSelector -> fmridenoise_wf.ReportCreator join field pipelines_names as pipelines_namesJ1.
190919-15:37:37,185 nipype.workflow DEBUG:
Connected the join node fmridenoise_wf.ReportCreator subgraph to the expanded join point fmridenoise_wf.PipelineSelector
190919-15:37:37,186 nipype.workflow DEBUG:
PE: expanding iterables … done
190919-15:37:37,188 nipype.workflow DEBUG:
Performing depth first search
190919-15:37:37,190 nipype.workflow INFO:
Running serially.
190919-15:37:37,191 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector”.
190919-15:37:37,191 nipype.workflow DEBUG:
Setting node inputs
190919-15:37:37,191 nipype.workflow DEBUG:
[Node] Hashes: [(‘pipeline_path’, (’/data/project/BEACON/wk_directories/fmridenoise/pipelines/pipeline-24HMP_aCompCor_SpikeReg_4GS.json’, ‘da70df8cd31b76db20e16a6ba57b6b77’))], ec197b0ed6eb6589b8527cbdf8d5d596, /tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_0xec197b0ed6eb6589b8527cbdf8d5d596.json, [’/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_0xb16c48fde59a84dd4cc3958e91e9b884.json’]
190919-15:37:37,191 nipype.workflow INFO:
[Node] Outdated cache found for “fmridenoise_wf.PipelineSelector”.
190919-15:37:37,191 nipype.workflow DEBUG:
[Node] Old/new hashes = b16c48fde59a84dd4cc3958e91e9b884/ec197b0ed6eb6589b8527cbdf8d5d596
190919-15:37:37,192 nipype.workflow DEBUG:
Dictionaries had differing keys:
* keys not presently seen: {‘needed_outputs’}
190919-15:37:37,192 nipype.utils DEBUG:
Removing contents of /tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector
190919-15:37:37,192 nipype.utils DEBUG:
Creating directory /tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_report
190919-15:37:37,193 nipype.workflow DEBUG:
[Node] Writing pre-exec report to “/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_report/report.rst”
190919-15:37:37,194 nipype.workflow INFO:
[Node] Running “PipelineSelector” (“fmridenoise.interfaces.pipeline_selector.PipelineSelector”)
190919-15:37:37,195 nipype.interface DEBUG:
Creating a ResourceMonitor on a PipelineSelector interface, PID=48358.
190919-15:37:37,491 nipype.workflow DEBUG:
Needed files: /data/project/BEACON/wk_directories/fmridenoise/pipelines/pipeline-24HMP_aCompCor_SpikeReg_4GS.json;/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_0xec197b0ed6eb6589b8527cbdf8d5d596_unfinished.json;/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_inputs.pklz;/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_node.pklz;/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/.proc-48358_time-1568903857.195214_freq-1.00
190919-15:37:37,491 nipype.workflow DEBUG:
Needed dirs: /tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_report
190919-15:37:37,491 nipype.workflow DEBUG:
Removing files:
190919-15:37:37,492 nipype.workflow DEBUG:
saved results in /tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/result_PipelineSelector.pklz
190919-15:37:37,492 nipype.workflow DEBUG:
[Node] Writing post-exec report to “/tmp/fmridenoise/fmridenoise_wf/pipeline_path…data…project…BEACON…wk_directories…fmridenoise…pipelines…pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector/_report/report.rst”
190919-15:37:37,493 nipype.workflow DEBUG:
Aggregate: False
190919-15:37:37,494 nipype.workflow INFO:
[Node] Finished “fmridenoise_wf.PipelineSelector”.
190919-15:37:37,494 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.BidsGrabber” in “/tmp/fmridenoise/fmridenoise_wf/BidsGrabber”.
190919-15:37:37,494 nipype.workflow DEBUG:
Setting node inputs
190919-15:37:37,494 nipype.workflow DEBUG:
[Node] Hashes: [(‘bids_dir’, ‘/data/project/BEACON/wk_directories/wk_tom_bids/’), (‘derivatives’, [’/data/project/BEACON/wk_directories/wk_tom_bids/derivatives/fmriprep’]), (‘ica_aroma’, True)], 1793ffca8cc1465446929dfcc8b0414e, /tmp/fmridenoise/fmridenoise_wf/BidsGrabber/_0x1793ffca8cc1465446929dfcc8b0414e.json, []
190919-15:37:37,494 nipype.workflow DEBUG:
[Node] No hashfiles found in “/tmp/fmridenoise/fmridenoise_wf/BidsGrabber”.
190919-15:37:37,494 nipype.utils DEBUG:
Removing contents of /tmp/fmridenoise/fmridenoise_wf/BidsGrabber
190919-15:37:37,495 nipype.utils DEBUG:
Creating directory /tmp/fmridenoise/fmridenoise_wf/BidsGrabber/_report
190919-15:37:37,495 nipype.workflow DEBUG:
[Node] Writing pre-exec report to “/tmp/fmridenoise/fmridenoise_wf/BidsGrabber/_report/report.rst”
190919-15:37:37,496 nipype.workflow INFO:
[Node] Running “BidsGrabber” (“fmridenoise.interfaces.bids.BIDSGrab”)
190919-15:37:37,496 nipype.interface DEBUG:
Creating a ResourceMonitor on a BIDSGrab interface, PID=48358.
190919-15:38:46,972 nipype.workflow DEBUG:
saved results in /tmp/fmridenoise/fmridenoise_wf/BidsGrabber/result_BidsGrabber.pklz
190919-15:38:46,973 nipype.workflow WARNING:
[Node] Error on “fmridenoise_wf.BidsGrabber” (/tmp/fmridenoise/fmridenoise_wf/BidsGrabber)
190919-15:38:46,974 nipype.workflow DEBUG:
Aggregate: False
190919-15:38:46,974 nipype.workflow DEBUG:
Aggregate: False
190919-15:38:46,974 nipype.workflow DEBUG:
Aggregate: False
190919-15:38:46,975 nipype.workflow DEBUG:
Aggregate: False
190919-15:38:46,975 nipype.workflow ERROR:
Node BidsGrabber failed to run on host login2.nan.kcl.ac.uk.
190919-15:38:46,975 nipype.workflow ERROR:
Saving crash info to /data/project/BEACON/wk_directories/fmridenoise/crash-20190919-153846-k1514616-BidsGrabber-c4bfa51e-3b16-41d9-afc8-b62b94d6fb07.pklz
Traceback (most recent call last):
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/plugins/linear.py”, line 48, in run
node.run(updatehash=updatehash)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 472, in run
result = self._run_interface(execute=True)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 563, in _run_interface
return self._run_command(execute)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 643, in _run_command
result = self._interface.run(cwd=outdir)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 375, in run
runtime = self._run_interface(runtime)
File “/home/k1514616/.local/lib/python3.7/site-packages/fmridenoise/interfaces/bids.py”, line 296, in _run_interface
f"ICA-Aroma file not found for file {fmri_file.path}"
FileNotFoundError: ICA-Aroma file not found for file /data/project/BEACON/wk_directories/wk_tom_bids/derivatives/fmriprep/sub-G1001/func/sub-G1001_task-tom_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
Traceback (most recent call last):
File “/software/system/anaconda/3.7.1/lib/python3.7/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/software/system/anaconda/3.7.1/lib/python3.7/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/home/k1514616/.local/lib/python3.7/site-packages/fmridenoise/main.py”, line 168, in
main()
File “/home/k1514616/.local/lib/python3.7/site-packages/fmridenoise/main.py”, line 164, in main
workflow.run()
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py”, line 599, in run
runner.run(execgraph, updatehash=updatehash, config=self.config)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/plugins/linear.py”, line 48, in run
node.run(updatehash=updatehash)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 472, in run
result = self._run_interface(execute=True)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 563, in _run_interface
return self._run_command(execute)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 643, in _run_command
result = self._interface.run(cwd=outdir)
File “/nan/ceph/network/system/el7/anaconda/3.7.1/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 375, in run
runtime = self._run_interface(runtime)
File “/home/k1514616/.local/lib/python3.7/site-packages/fmridenoise/interfaces/bids.py”, line 296, in _run_interface
f"ICA-Aroma file not found for file {fmri_file.path}"
FileNotFoundError: ICA-Aroma file not found for file /data/project/BEACON/wk_directories/wk_tom_bids/derivatives/fmriprep/sub-G1001/func/sub-G1001_task-tom_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
However, my understanding is that the pipeline-24HMP_aCompCor_SpikeReg_4GS.json pipeline doesn’t run AROMA so I’m not sure why an ICA-Aroma file would be necessary.
Are you able to shed any light on this?
Many thanks!
Monica