AttributeError using fMRIDenoise on fMRIPrep outputs

jshino · September 2, 2019, 7:36am

I am very interested in fMRIDenoise. My data were BIDS formatted and preprocessed in fMRIPrep 1.4.0, with Ubuntu 18.04.3. When running fmridenoise, error messages were shown,

jshino@ubuntu:~$ fmridenoise /home/jshino/BIDS2/
/home/jshino/.local/lib/python3.6/site-packages/sklearn/externals/joblib/init.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
190902-16:13:38,538 nipype.workflow INFO:
Workflow fmridenoise_wf settings: [‘check’, ‘execution’, ‘logging’, ‘monitoring’]
190902-16:13:38,577 nipype.workflow INFO:
Running serially.
190902-16:13:38,578 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-24HMP_aCompCor_SpikeReg_4GS.json/PipelineSelector”.
190902-16:13:38,579 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,579 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,579 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-24HMP_8Phys_SpikeReg.json/PipelineSelector”.
190902-16:13:38,580 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,580 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,580 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-ICA-AROMA_8Phys_4GS.json/PipelineSelector”.
190902-16:13:38,581 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,581 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,581 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-ICA-AROMA_8Phys.json/PipelineSelector”.
190902-16:13:38,581 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,581 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,581 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-24HMP_aCompCor_SpikeReg.json/PipelineSelector”.
190902-16:13:38,582 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,582 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,582 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-24HMP_8Phys_SpikeReg_4GS.json/PipelineSelector”.
190902-16:13:38,583 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,583 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,583 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.PipelineSelector” in “/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise-0.1.1-py3.6.egg..fmridenoise..pipelines..pipeline-Null.json/PipelineSelector”.
190902-16:13:38,583 nipype.workflow INFO:
[Node] Cached “fmridenoise_wf.PipelineSelector” - collecting precomputed outputs
190902-16:13:38,583 nipype.workflow INFO:
[Node] “fmridenoise_wf.PipelineSelector” found cached.
190902-16:13:38,584 nipype.workflow INFO:
[Node] Setting-up “fmridenoise_wf.BidsGrabber” in “/tmp/fmridenoise/fmridenoise_wf/BidsGrabber”.
190902-16:13:38,585 nipype.workflow INFO:
[Node] Running “BidsGrabber” (“fmridenoise.interfaces.bids.BIDSGrab”)
190902-16:13:38,652 nipype.workflow WARNING:
Storing result file without outputs
190902-16:13:38,653 nipype.workflow WARNING:
[Node] Error on “fmridenoise_wf.BidsGrabber” (/tmp/fmridenoise/fmridenoise_wf/BidsGrabber)
190902-16:13:38,658 nipype.workflow ERROR:
Node BidsGrabber failed to run on host ubuntu.
190902-16:13:38,659 nipype.workflow ERROR:
Saving crash info to /home/jshino/crash-20190902-161338-jshino-BidsGrabber-79da2314-3b9f-4e9f-9536-1e5959e115ec.pklz
Traceback (most recent call last):
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/plugins/linear.py”, line 48, in run
node.run(updatehash=updatehash)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 473, in run
result = self._run_interface(execute=True)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 564, in _run_interface
return self._run_command(execute)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 649, in _run_command
result = self._interface.run(cwd=outdir)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/interfaces/base/core.py”, line 376, in run
runtime = self._run_interface(runtime)
File “/home/jshino/.local/lib/python3.6/site-packages/fmridenoise-0.1.1-py3.6.egg/fmridenoise/interfaces/bids.py”, line 247, in _run_interface
entity_bold = fmri_file.get_entities()
File “/home/jshino/.local/lib/python3.6/site-packages/bids/layout/layout.py”, line 65, in getattr
(self.class.name, attr))
AttributeError: BIDSFile object has no attribute named ‘get_entities’

190902-16:13:38,660 nipype.workflow INFO:

190902-16:13:38,660 nipype.workflow ERROR:
could not run node: fmridenoise_wf.BidsGrabber
190902-16:13:38,660 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190902-161338-jshino-BidsGrabber-79da2314-3b9f-4e9f-9536-1e5959e115ec.pklz
190902-16:13:38,660 nipype.workflow INFO:

Traceback (most recent call last):
File “/usr/lib/python3.6/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/usr/lib/python3.6/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/home/jshino/.local/lib/python3.6/site-packages/fmridenoise-0.1.1-py3.6.egg/fmridenoise/main.py”, line 168, in
main()
File “/home/jshino/.local/lib/python3.6/site-packages/fmridenoise-0.1.1-py3.6.egg/fmridenoise/main.py”, line 164, in main
workflow.run()
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/workflows.py”, line 599, in run
runner.run(execgraph, updatehash=updatehash, config=self.config)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/plugins/linear.py”, line 71, in run
report_nodes_not_run(notrun)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/plugins/tools.py”, line 82, in report_nodes_not_run
raise RuntimeError(('Workflow did not execute cleanly. ’
RuntimeError: Workflow did not execute cleanly. Check log for details

Thanks a lot, Jun

karofinc · September 2, 2019, 12:37pm

Hi @jshino! Thank you so much for the interest in the tool and sharing error message!
I just raised an issue related to this: https://github.com/nbraingroup/fmridenoise/issues/22.

What’s your current version of pybids?

I suspect that the newest version of pybids might not install correctly from requirements.txt. We will fix this asap.

jshino · September 3, 2019, 1:02am

Hi @karofinc! Thank you so much for your fast reply!

I did

pip install -r requirements.txt

but, “segmentation fault” error occurred.

Installing collected packages: bz2file, numpy, six, nibabel, pytz, python-dateutil, pandas, scipy, cycler, backports.functools-lru-cache, subprocess32, setuptools, kiwisolver, pyparsing, matplotlib, seaborn, nilearn, pyrsistent, functools32, attrs, jsonschema, traits, futures, pydotplus, future, lxml, isodate, rdflib, decorator, networkx, prov, scandir, pathlib2, click, configparser, neurdflib, funcsigs, pydot, packaging, simplejson, nipype, scikit-learn, sklearn, SQLAlchemy, bids-validator, patsy, docopt, num2words, pybids, MarkupSafe, jinja2
Successfully installed MarkupSafe-1.1.1 SQLAlchemy-1.3.8 attrs-19.1.0 backports.functools-lru-cache-1.5 bids-validator-1.2.4 bz2file-0.98 click-7.0 configparser-3.8.1 cycler-0.10.0 decorator-4.4.0 docopt-0.6.2 funcsigs-1.0.2 functools32-3.2.3.post2 future-0.17.1 futures-3.3.0 isodate-0.6.0 jinja2-2.10.1 jsonschema-3.0.2 kiwisolver-1.1.0 lxml-4.4.1 matplotlib-2.2.4 networkx-2.2 neurdflib-5.0.0.post1 nibabel-2.5.0 nilearn-0.5.2 nipype-1.2.1 num2words-0.5.10 numpy-1.16.5 packaging-19.1 pandas-0.24.2 pathlib2-2.3.4 patsy-0.5.1 prov-1.5.3 pybids-0.9.3 pydot-1.4.1 pydotplus-2.0.2 pyparsing-2.4.2 pyrsistent-0.15.4 python-dateutil-2.8.0 pytz-2019.2 rdflib-4.2.2 scandir-1.10.0 scikit-learn-0.20.4 scipy-1.2.2 seaborn-0.9.0 setuptools-41.2.0 simplejson-3.16.0 six-1.12.0 sklearn-0.0 subprocess32-3.5.4 traits-5.1.2
Segmentation fault (core dumped)

Now, I cannot check the version of “pybids”.

jshino@ubuntu:~$ python
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import pybids
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘pybids’

However, I can import “nipype”, “pandas”, “matplotlib”.

jshino@ubuntu:~$ python
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import nipype
import pandas
import matplotlib
nipype.version
‘1.2.1’

any advises will be appreciated.
many thanks, Jun

jshino · September 3, 2019, 2:35am

Hi, @karofinc!

Now, I tried

sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get install -y python3-pip
sudo -H pip3 install -r /home/jshino/projects/fmridenoise/requirements.txt
sudo -H pip3 install fmridenoise

then

fmridenoise /home/jshino/BIDS2

So, other errors were shown

190903-11:19:26,459 nipype.workflow WARNING:
[Node] Error on “_ConfPrep2” (/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise..pipelines..pipeline-Null.json/ConfPrep/mapflow/_ConfPrep2)
190903-11:19:26,460 nipype.workflow WARNING:
[Node] Error on “fmridenoise_wf.ConfPrep” (/tmp/fmridenoise/fmridenoise_wf/pipeline_path..home..jshino…local..lib..python3.6..site-packages..fmridenoise..pipelines..pipeline-Null.json/ConfPrep)
190903-11:19:26,460 nipype.workflow ERROR:
Node ConfPrep.a6 failed to run on host ubuntu.
190903-11:19:26,460 nipype.workflow ERROR:
Saving crash info to /home/jshino/crash-20190903-111926-jshino-ConfPrep.a6-13ac9675-75fb-48a3-9e3b-143e317c802d.pklz
Traceback (most recent call last):
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/plugins/linear.py”, line 48, in run
node.run(updatehash=updatehash)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 473, in run
result = self._run_interface(execute=True)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 1269, in _run_interface
self.config[‘execution’][‘stop_on_first_crash’])))
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 1191, in _collate_results
(self.name, ‘\n’.join(msg)))
Exception: Subnodes of node: ConfPrep failed:
Subnode 0 failed
Error: Traceback (most recent call last):

(error messages …)

190903-11:19:26,951 nipype.workflow INFO:

190903-11:19:26,951 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a6
190903-11:19:26,951 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a6-13ac9675-75fb-48a3-9e3b-143e317c802d.pklz
190903-11:19:26,951 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a5
190903-11:19:26,951 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a5-0eb4f4c2-8c07-4fa6-97f8-3766ac308fed.pklz
190903-11:19:26,951 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a4
190903-11:19:26,951 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a4-67b74c18-9c86-4fa6-9200-12f8a127107f.pklz
190903-11:19:26,951 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a3
190903-11:19:26,951 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a3-140fc270-e9b3-435f-a998-5656a6cac5a7.pklz
190903-11:19:26,951 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a2
190903-11:19:26,951 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a2-39f3e43e-446f-4ba8-a4b2-c9478feb85d3.pklz
190903-11:19:26,951 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a1
190903-11:19:26,952 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a1-d6e6bf39-a252-464c-a48b-99c80c47c2cc.pklz
190903-11:19:26,952 nipype.workflow ERROR:
could not run node: fmridenoise_wf.ConfPrep.a0
190903-11:19:26,952 nipype.workflow INFO:
crashfile: /home/jshino/crash-20190903-111926-jshino-ConfPrep.a0-d21605cd-75cb-4cbf-84b9-c0fbd2a1da16.pklz
190903-11:19:26,952 nipype.workflow INFO:

Traceback (most recent call last):
File “/usr/lib/python3.6/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/usr/lib/python3.6/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/home/jshino/.local/lib/python3.6/site-packages/fmridenoise/main.py”, line 168, in
main()
File “/home/jshino/.local/lib/python3.6/site-packages/fmridenoise/main.py”, line 164, in main
workflow.run()
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/engine/workflows.py”, line 599, in run
runner.run(execgraph, updatehash=updatehash, config=self.config)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/plugins/linear.py”, line 71, in run
report_nodes_not_run(notrun)
File “/home/jshino/.local/lib/python3.6/site-packages/nipype/pipeline/plugins/tools.py”, line 82, in report_nodes_not_run
raise RuntimeError(('Workflow did not execute cleanly. ’
RuntimeError: Workflow did not execute cleanly. Check log for details

I am very confusing…
thanks a lot.

karofinc · September 3, 2019, 9:06am

@jshino could you send us all crash files? (*.pklz)
Thanks!

jshino · September 3, 2019, 10:10am

@karofinc could you download a zip file (including 7 .pklz files)?
https://www.sugarsync.com/pf/D720540_08778893_7622094

thanks a lot in advance!

karofinc · September 3, 2019, 1:45pm

@jshino thanks! I think we diagnosed a problem. The Issue is discussed here: https://github.com/nbraingroup/fmridenoise/issues/25

Please paste your BIDS folders tree structure for a single subject on the issue’s page. This will help to fix the problem fast.

jshino · September 4, 2019, 2:53am

@karofinc thank you for your fast reply!
I pasted BIDS folders tree at

Is it OK?

karofinc · September 16, 2019, 12:35pm

Hi @jshino! I think we’ve fixed the issue. Can you install the newest GH version of fmridenoise via python setup.py install and let us know if it works?

Thanks a lot!

jshino · September 17, 2019, 2:50am

Hi @karofinc! Thank you very much! I installed fmridenoise via python setup.py install --user.
Unfortunately, BidsGrabber errors occurred.

The errors were (a .txt file):
https://www.sugarsync.com/pf/D720540_08778893_7765811

You can download a crashfile (.pklz) via
https://www.sugarsync.com/pf/D720540_08778893_7765876

Thanks a lot!