Fitlins loader error (NaN inputs)

1

I’m trying to use fitlins for the first time, to run a GLM on data preprocessed with fmriprep. I’m hoping that it will be easier than using FSL for a design in which not all trial types are included in every run (i.e., trial types are conditionalized on participant responses).

I’m trying to start off by running this on a single subject, but I’m getting an error, which seems like it might be due to the issue of some conditions not being present in some runs. It would be great if fitlins could handle having the same design file for all participants, which may include conditions that aren’t present at all for a particular participant. I did try deleting conditions that weren’t present at all for the participant I was trying it on, which didn’t help. Even in that more stripped-down model specification, though, there are some conditions that are present in one run but not the other.

Attached are the full model specification and event specifications (.txt added for Neurostars):
model-1_smdn.json.txt (14.7 KB)
sub-8_task-decision_run-2_events.tsv.txt (2.1 KB) sub-8_task-decision_run-1_events.tsv.txt (2.1 KB)

Here’s is what I ran (with participant ID removed):
singularity run --cleanenv my_images/fitlins-0.9.0.simg EMORLO_BIDS_2 EMORLO_BIDS_fitlins participant -v -v --participant-label {label} --model ~/EMORLO_BIDS_2/models/model-2_smdn.json -d EMORLO_BIDS_fmriprep/ --space MNI152NLin2009cAsym --smoothing 6:run --estimator nistats --drop-missing

Here’s the output:

210312-00:20:57,795 nipype.workflow INFO:
Workflow fitlins_wf settings: [‘check’, ‘execution’, ‘logging’, ‘monitoring’]
210312-00:20:57,822 nipype.workflow INFO:
Running in parallel.
210312-00:20:57,832 nipype.workflow INFO:
[MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 339.92/339.92, Free processors: 40/40.
210312-00:21:00,117 nipype.workflow INFO:
[MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 339.72/339.92, Free processors: 39/40.
Currently running:
* fitlins_wf.loader
210312-00:21:00,223 nipype.workflow INFO:
[Node] Setting-up “fitlins_wf.loader” in “/tmp/tmp9wulfe0_/fitlins_wf/loader”.
210312-00:21:00,253 nipype.workflow INFO:
[Node] Running “loader” (“fitlins.interfaces.bids.LoadBIDSModel”)
210312-00:21:06,840 nipype.workflow WARNING:
Storing result file without outputs
210312-00:21:06,842 nipype.workflow WARNING:
[Node] Error on “fitlins_wf.loader” (/tmp/tmp9wulfe0_/fitlins_wf/loader)
210312-00:21:08,125 nipype.workflow ERROR:
Node loader failed to run on host cubic-login3.
210312-00:21:08,125 nipype.workflow ERROR:
Saving crash info to /cbica/home/cohemic/crash-20210312-002108-cohemic-loader-7e1cab0b-5f02-4afc-9499-7eca02bd731b.txt
Traceback (most recent call last):
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py”, line 148, in init
mp_context=mp_context,
TypeError: init() got an unexpected keyword argument ‘initializer’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py”, line 67, in run_node
result[“result”] = node.run(updatehash=updatehash)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 516, in run
result = self._run_interface(execute=True)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 635, in _run_interface
return self._run_command(execute)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 741, in _run_command
result = self._interface.run(cwd=outdir)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py”, line 434, in run
runtime = self._run_interface(runtime)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/interfaces/bids.py”, line 221, in _run_interface
analysis.setup(drop_na=False, **selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 104, in setup
self.finalize(**selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 112, in finalize
step.setup(input_contrasts, **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 282, in setup
colls = tm.TransformerManager().transform(coll, self.transformations)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/base.py”, line 458, in transform
func(collection, cols, **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/base.py”, line 97, in new
return t.transform()
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/base.py”, line 271, in transform
result = self._transform(data[i], **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/munge.py”, line 143, in _transform
new_cols = pd.get_dummies(data[‘amplitude’], drop_first=False)[levels]
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/frame.py”, line 2986, in getitem
indexer = self.loc._convert_to_indexer(key, axis=1, raise_missing=True)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/indexing.py”, line 1285, in _convert_to_indexer
return self._get_listlike_indexer(obj, axis, **kwargs)[1]
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/indexing.py”, line 1092, in _get_listlike_indexer
keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/indexing.py”, line 1185, in _validate_read_indexer
raise KeyError("{} not in index".format(not_found))
KeyError: ‘[nan] not in index’

210312-00:21:08,144 nipype.workflow INFO:
[MultiProc] Running 0 tasks, and 0 jobs ready. Free memory (GB): 339.92/339.92, Free processors: 40/40.
210312-00:21:10,124 nipype.workflow INFO:

210312-00:21:10,124 nipype.workflow ERROR:
could not run node: fitlins_wf.loader
210312-00:21:10,124 nipype.workflow INFO:
crashfile: /cbica/home/cohemic/crash-20210312-002108-cohemic-loader-7e1cab0b-5f02-4afc-9499-7eca02bd731b.txt
210312-00:21:10,125 nipype.workflow INFO:

Traceback (most recent call last):
File “/opt/miniconda-latest/envs/neuro/bin/fitlins”, line 8, in
sys.exit(main())
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py”, line 294, in main
sys.exit(run_fitlins(sys.argv[1:]))
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py”, line 286, in run_fitlins
analysis.setup(**selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 104, in setup
self.finalize(**selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 112, in finalize
step.setup(input_contrasts, **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 282, in setup
colls = tm.TransformerManager().transform(coll, self.transformations)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/base.py”, line 458, in transform
func(collection, cols, **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/base.py”, line 97, in new
return t.transform()
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/base.py”, line 271, in transform
result = self._transform(data[i], **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/transformations/munge.py”, line 143, in _transform
new_cols = pd.get_dummies(data[‘amplitude’], drop_first=False)[levels]
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/frame.py”, line 2986, in getitem
indexer = self.loc._convert_to_indexer(key, axis=1, raise_missing=True)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/indexing.py”, line 1285, in _convert_to_indexer
return self._get_listlike_indexer(obj, axis, **kwargs)[1]
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/indexing.py”, line 1092, in _get_listlike_indexer
keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/pandas/core/indexing.py”, line 1185, in _validate_read_indexer
raise KeyError("{} not in index".format(not_found))
KeyError: ‘[nan] not in index’

Any advice on what I should be doing differently?

Thanks!
– Michael

2

Hi,
I just wanted to add to this, as I tried running a more stripped down model on a subject who I think should have all of the trial types I’m modeling on every run, and I still get an error. It’s a different error, though. I also tried switching options from “participant” to “run” (not sure which I should be doing).

Here are the event and model files:
sub-15_task-decision_run-2_events.tsv.txt (2.1 KB) sub-15_task-decision_run-1_events.tsv.txt (2.1 KB)model-6_smdn.json.txt (6.1 KB)

This was the command:
singularity run --cleanenv my_images/fitlins-0.9.0.simg EMORLO_BIDS_2 EMORLO_BIDS_fitlins run -v -v --participant-label {redacted} --model ~/EMORLO_BIDS_2/models/model-6_smdn.json -d EMORLO_BIDS_fmriprep/ --space MNI152NLin2009cAsym --smoothing 6:run --estimator nistats --drop-missing

This was the output:

210318-23:52:44,700 nipype.workflow INFO:
Workflow fitlins_wf settings: [‘check’, ‘execution’, ‘logging’, ‘monitoring’]
210318-23:52:44,729 nipype.workflow INFO:
Running in parallel.
210318-23:52:44,743 nipype.workflow INFO:
[MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 339.92/339.92, Free processors: 40/40.
210318-23:52:47,173 nipype.workflow INFO:
[MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 339.72/339.92, Free processors: 39/40.
Currently running:
* fitlins_wf.loader
210318-23:52:47,273 nipype.workflow INFO:
[Node] Setting-up “fitlins_wf.loader” in “/tmp/tmp1isr7g_h/fitlins_wf/loader”.
210318-23:52:47,285 nipype.workflow INFO:
[Node] Running “loader” (“fitlins.interfaces.bids.LoadBIDSModel”)
210318-23:52:53,714 nipype.workflow WARNING:
Storing result file without outputs
210318-23:52:53,715 nipype.workflow WARNING:
[Node] Error on “fitlins_wf.loader” (/tmp/tmp1isr7g_h/fitlins_wf/loader)
210318-23:52:55,183 nipype.workflow ERROR:
Node loader failed to run on host cubic-login3.
210318-23:52:55,183 nipype.workflow ERROR:
Saving crash info to /cbica/home/cohemic/crash-20210318-235255-cohemic-loader-c16e325a-7889-4902-b294-f14c3466d29a.txt
Traceback (most recent call last):
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py”, line 148, in init
mp_context=mp_context,
TypeError: init() got an unexpected keyword argument ‘initializer’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py”, line 67, in run_node
result[“result”] = node.run(updatehash=updatehash)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 516, in run
result = self._run_interface(execute=True)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 635, in _run_interface
return self._run_command(execute)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py”, line 741, in _run_command
result = self._interface.run(cwd=outdir)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py”, line 434, in run
runtime = self._run_interface(runtime)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/interfaces/bids.py”, line 221, in _run_interface
analysis.setup(drop_na=False, **selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 104, in setup
self.finalize(**selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 112, in finalize
step.setup(input_contrasts, **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 280, in setup
coll = merge_collections(colls)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/collections.py”, line 573, in merge_collections
cls = collections[0].class
IndexError: list index out of range

210318-23:52:55,195 nipype.workflow INFO:
[MultiProc] Running 0 tasks, and 0 jobs ready. Free memory (GB): 339.92/339.92, Free processors: 40/40.
210318-23:52:57,180 nipype.workflow INFO:

210318-23:52:57,181 nipype.workflow ERROR:
could not run node: fitlins_wf.loader
210318-23:52:57,181 nipype.workflow INFO:
crashfile: /cbica/home/cohemic/crash-20210318-235255-cohemic-loader-c16e325a-7889-4902-b294-f14c3466d29a.txt
210318-23:52:57,181 nipype.workflow INFO:

Traceback (most recent call last):
File “/opt/miniconda-latest/envs/neuro/bin/fitlins”, line 8, in
sys.exit(main())
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py”, line 294, in main
sys.exit(run_fitlins(sys.argv[1:]))
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py”, line 286, in run_fitlins
analysis.setup(**selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 104, in setup
self.finalize(**selectors)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 112, in finalize
step.setup(input_contrasts, **kwargs)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py”, line 280, in setup
coll = merge_collections(colls)
File “/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/collections.py”, line 573, in merge_collections
cls = collections[0].class
IndexError: list index out of range

3

Hi Michael,

Is there any chance of sharing your data? If not, with a confounds file from fMRIPrep I can probably generate some fake data to debug this.

Best,
Chris

4

Hi Michael,
For your first error, it looks like it’s specifically happening during the Factor transformation.

One thing I noticed with your event files straight away is that when for missing value you simply omit the value in the file. The be fully BIDS compliant, you should have an explicit n/a to represent a missing value. It’s possible that since we are not expecting a pandas NaN (which is probably what is inserted there), that’s why we get the problem.

Try putting n/a explicitly in your event files.

Also, it’s possible that Factoring a variable with n/a is not well supported (haven’t tried that).

Have you had luck running even simpler models on these data?

What about if you simply factor trial_type, and do nothing else. At least to see if such a simple model runs on your data.

I’ll have to take a closer look at your model and event files, but it seems like what you’re trying to do should be supported by fitlins/pyBIDS.

Best,
Alejandro

5

For reference about missing values: Common principles - Brain Imaging Data Structure v1.5.0

6

Thanks – I don’t think it’s solved yet, but this is a helpful start.

I fixed the issue with “n/a” values. (Those corrected files are attached below.) Sadly, I seem to be getting the same error with the full model even after fixing that.

I also tried a simpler model (attached below, as model7). With that, I don’t get the error about nan values. However, I do get a different error saying the following:

ValueError: No variable named ‘trans_x’ found in this collection. Available names are [‘item_memory’, ‘decision’, ‘old_new’, ‘reward_value’, ‘response_time’, ‘trial_type.house’, ‘trial_type.person’].

I assume that means that for some reason, it’s not reading the confounds file properly. I tried fiddling with how the derivatives directory is specified (using “–derivatives” instead of “-d”, and adding “/fmriprep” as a subdirectory there), but to no avail.

Also just to note: my second message earlier (the “list index out of range” error) turned out to be a dumb mistake on my part – I forgot that I had only copied over the event.tsv files for the one subject I was trying to test initially. When I fix that, I still get an error, but it looks like the one I initially posted (“NaN not in index”).

Finally, I’ll attach the confounds file from fMRIPrep. Hopefully that will work for simulating the run! (If you do need the raw data, I can probably upload it on Google Drive and PM you a link, but let’s try the simulated data route first if it’s not too much of a pain.)

Thanks,
Michael

model-7_smdn.json.txt (1.3 KB)

sub-15_task-decision_run-2_events.tsv.txt (2.2 KB) sub-15_task-decision_run-1_events.tsv.txt (2.2 KB) sub-15_ses-01EMO_task-decision_run-1_desc-confounds_timeseries.tsv.txt (1.6 MB) sub-15_ses-01EMO_task-decision_run-2_desc-confounds_timeseries.tsv.txt (1.6 MB)

sub-8_task-decision_run-2_events.tsv.txt (2.2 KB) sub-8_task-decision_run-1_events.tsv.txt (2.2 KB) sub-8_ses-01EMO_task-decision_run-1_desc-confounds_timeseries.tsv.txt (1.4 MB) sub-8_ses-01EMO_task-decision_run-2_desc-confounds_timeseries.tsv.txt (1.4 MB)