Fitlins and BIDS stats subject level model

Summary of what happened:

I am trying to run a subject level model in fitlins for a faces>all localizer. I keep getting errors including:

KeyError: "['extension'] not found in axis"

It is difficult for me to debug this as it takes 45 minutes before fitlins reaches this error. Does anyone know what is wrong with my model? Thank you!

Command used (and if a helper script was used, a link to the helper script or the command generated):

command:

singularity run --cleanenv \
-B /om2 \
/om2/user/jsmentch/fitlins-latest.simg \
/om2/scratch/Mon/jsmentch/cneuromod/hcptrt ./ participant \
-d /om2/scratch/Mon/jsmentch/hcptrt \
-w /om2/scratch/Mon/jsmentch/fitlins/work_subject/ \
-m /om2/scratch/Mon/jsmentch/faces_v_all_sub.json \
--desc-label '' \
--participant-label 01 \
--space fsLR -v --n-cpus 1 --mem-gb 48

BIDS stats model:

{
  "Name": "FacesVsAll",
  "BIDSModelVersion": "1.0.0",
  "Input": {"task":["wm"]},
  "Nodes": [
    {
      "Level": "Run",
      "Name": "run_level",
      "GroupBy": ["run"],
      "Model": {"X":
        ["stim_type.Face",
        "a_comp_cor_00",
        "a_comp_cor_01",
        "a_comp_cor_02",
        "a_comp_cor_03",
        "a_comp_cor_04",
        "cosine*",
        "csf",
        "framewise_displacement",
        "rot_x",
        "rot_y",
        "rot_z",
        "trans_x",
        "trans_y",
        "trans_z"],
      "Type": "glm"},
      "DummyContrasts":{
            "Contrasts":[
               "stim_type.Face"],
            "Test":"t"
      },
      "Transformations":{
        "Transformer":"pybids-transforms-v1",
        "Instructions":[
          {
            "Name": "Factor",
            "Input": ["stim_type"]
          },
          {
              "Name":"Convolve",
              "Input":[
                 "stim_type.Face"
              ],
              "Model":"spm"
           }
          ]
      }
    },
    {
      "Level": "Subject",
      "Name": "subject_level",
      "GroupBy": ["subject", "contrast"],
      "Model": {"X": [1], "Type": "meta"},
      "DummyContrasts": {"Test": "t"}
    }
  ]
}

Version:

fitlins-latest singular container

Environment (Docker, Singularity / Apptainer, custom installation):

Data formatted according to a validatable standard? Please provide the output of the validator:

the BIDS Stats Model Validator didn’t complain

Relevant log outputs (up to 20 lines):

Node: fitlins_wf.loader
Working directory: /om2/scratch/Mon/jsmentch/fitlins/work_subject/fitlins_wf/loader

Node inputs:

database_path = /om2/scratch/Mon/jsmentch/fitlins/work_subject/dbcache
model = {'Name': 'FacesVsAll', 'BIDSModelVersion': '1.0.0', 'Input': {'task': ['wm']}, 'Nodes': [{'Level': 'Run', 'Name': 'run_level', 'GroupBy': ['run'], 'Model': {'X': ['stim_type.Face', 'a_comp_cor_00', 'a_comp_cor_01', 'a_comp_cor_02', 'a_comp_cor_03', 'a_comp_cor_04', 'cosine*', 'csf', 'framewise_displacement', 'rot_x', 'rot_y', 'rot_z', 'trans_x', 'trans_y', 'trans_z'], 'Type': 'glm'}, 'DummyContrasts': {'Contrasts': ['stim_type.Face'], 'Test': 't'}, 'Transformations': {'Transformer': 'pybids-transforms-v1', 'Instructions': [{'Name': 'Factor', 'Input': ['stim_type']}, {'Name': 'Convolve', 'Input': ['stim_type.Face'], 'Model': 'spm'}]}}, {'Level': 'Subject', 'Name': 'subject_level', 'GroupBy': ['subject', 'contrast'], 'Model': {'X': [1], 'Type': 'meta'}, 'DummyContrasts': {'Test': 't'}}]}
selectors = {'desc': None, 'space': 'fsLR', 'subject': ['01']}

Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node loader.

Traceback:
        Traceback (most recent call last):
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run
            runtime = self._run_interface(runtime)
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/fitlins/interfaces/bids.py", line 246, in _run_interface
            graph.load_collections(**selectors)
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/bids/modeling/statsmodels.py", line 198, in load_collections
            collections = self.layout.get_collections(node.level, drop_na=drop_na,
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/bids/layout/layout.py", line 860, in get_collections
            index = load_variables(self, types=types, levels=level,
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/bids/variables/io.py", line 97, in load_variables
            dataset = _load_tsv_variables(layout, t, dataset, scope=scope,
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/bids/variables/io.py", line 463, in _load_tsv_variables
            ent_cols.drop(columns=to_drop, inplace=True)
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
            return func(*args, **kwargs)
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/pandas/core/frame.py", line 4906, in drop
            return super().drop(
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/pandas/core/generic.py", line 4150, in drop
            obj = obj._drop_axis(labels, axis, level=level, errors=errors)
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/pandas/core/generic.py", line 4185, in _drop_axis
            new_axis = axis.drop(labels, errors=errors)
          File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 6017, in drop
            raise KeyError(f"{labels[mask]} not found in axis")
        KeyError: "['extension'] not found in axis"

Screenshots / relevant information:


Hi @jsmentch,

Can you show an example of an events.tsv you are working with?

Can you try putting the HRF convolving node before the GLM node?

Can you try adding --drop-missing to the command?

It looks like you are only modeling the faces. I think it would still be a good idea to model the other tasks, and then do a contrast of faces > each other task.

Also, not related to the error, but do you need to regress out csf if you are using acompcor components?

Best.
Steven

1 Like

Thanks Steven! I will try those adjustments

It looks like you are only modeling the faces. I think it would still be a good idea to model the other tasks, and then do a contrast of faces > each other task.

good point, yes I would like to do that but here I was stripping it down to the simplest model to try to get anything working

an events.tsv:

trial_type	trial	answer	correct_answer	stim_file	target_type	stim_type	onset	duration	response_time	countdown_stim
countdown							0.0	1.997		4
countdown							2.0	1.997		3
countdown							4.0	1.997		2
countdown							6.013	1.984		1
2-Back	1	no_match	no_match	wm/BP_002_BP10_BA.png	nonlure	Body	10.563	0.799	0.799	
2-Back	2	no_match	no_match	wm/BP_001_BP1_BA.png	nonlure	Body	13.096	1.175	1.175	
2-Back	3	no_match	no_match	wm/BP_006_BP2_BA.png	nonlure	Body	15.63	1.4	1.4	
2-Back	4	match	match	wm/BP_001_BP1_BA.png	target	Body	18.163	1.222	1.222	
2-Back	5	no_match	no_match	wm/BP_005_BP19_BA.png	nonlure	Body	20.696	1.135	1.135	
2-Back	6	no_match	no_match	wm/BP_003_BP16_BA.png	nonlure	Body	23.229	1.209	1.209	
2-Back	7		no_match	wm/BP_006_BP2_BA.png	lure	Body	25.762	0.0	0.0	
2-Back	8	no_match	no_match	wm/BP_004_BP18_BA.png	nonlure	Body	28.295	1.141	1.141	
2-Back	9	match	no_match	wm/BP_005_BP19_BA.png	lure	Body	30.829	0.67	0.67	
2-Back	10	match	match	wm/BP_004_BP18_BA.png	target	Body	33.362	1.491	1.491	
Cue				wm/FC_001_M1.png			35.878			
0-Back	1	no_match	no_match	wm/FC_005_M3.png	nonlure	Face	38.396	0.86	0.86	
0-Back	2	match	match	wm/FC_001_M1.png	target	Face	40.929	0.637	0.637	
0-Back	3	no_match	no_match	wm/FC_003_M2.png	nonlure	Face	43.463	1.084	1.084	
0-Back	4	no_match	no_match	wm/FC_004_F2.png	nonlure	Face	45.997	0.647	0.647	
0-Back	5	no_match	no_match	wm/FC_004_F2.png	lure	Face	48.53	0.723	0.723	
0-Back	6	match	match	wm/FC_001_M1.png	target	Face	51.062	0.536	0.536	
0-Back	7	no_match	no_match	wm/FC_006_F3.png	nonlure	Face	53.596	0.582	0.582	
0-Back	8	no_match	no_match	wm/FC_006_F3.png	lure	Face	56.129	0.707	0.707	
0-Back	9	no_match	no_match	wm/FC_003_M2.png	lure	Face	58.662	0.759	0.759	
0-Back	10	no_match	no_match	wm/FC_002_F1.png	nonlure	Face	61.195	0.578	0.578	
2-Back	1	no_match	no_match	wm/TO_001_TOOL1_BA.png	nonlure	Tools	81.226	1.439	1.439	
2-Back	2	no_match	no_match	wm/TO_002_TOOL2_BA.png	nonlure	Tools	83.759	0.907	0.907	
2-Back	3	no_match	no_match	wm/TO_004_TOOL4_BA.png	nonlure	Tools	86.292	1.175	1.175	
2-Back	4	match	match	wm/TO_002_TOOL2_BA.png	target	Tools	88.825	0.732	0.732	
2-Back	5	no_match	no_match	wm/TO_001_TOOL1_BA.png	lure	Tools	91.358	1.079	1.079	
2-Back	6	match	no_match	wm/TO_004_TOOL4_BA.png	lure	Tools	93.892	0.676	0.676	
2-Back	7	no_match	no_match	wm/TO_006_TOOL6_BA.png	nonlure	Tools	96.425	0.812	0.812	
2-Back	8	no_match	no_match	wm/TO_005_TOOL5_BA.png	nonlure	Tools	98.958	1.543	1.543	
2-Back	9	match	match	wm/TO_006_TOOL6_BA.png	target	Tools	101.491	1.125	1.125	
2-Back	10	no_match	no_match	wm/TO_003_TOOL3_BA.png	nonlure	Tools	104.024	0.674	0.674	
Cue				wm/BP_007_BP20_BA.png			106.541			
0-Back	1	match	match	wm/BP_007_BP20_BA.png	target	Body	109.057	1.581	1.581	
0-Back	2	no_match	no_match	wm/BP_011_BP4_BA.png	nonlure	Body	111.591	0.834	0.834	
0-Back	3	no_match	no_match	wm/BP_010_BP3_BA.png	nonlure	Body	114.124	0.697	0.697	
0-Back	4	no_match	no_match	wm/BP_008_BP21_BA.png	nonlure	Body	116.657	0.93	0.93	
0-Back	5	no_match	no_match	wm/BP_011_BP4_BA.png	lure	Body	119.19	0.735	0.735	
0-Back	6	no_match	no_match	wm/BP_009_BP22_BA.png	nonlure	Body	121.723	0.681	0.681	
0-Back	7	no_match	no_match	wm/BP_009_BP22_BA.png	lure	Body	124.256	0.619	0.619	
0-Back	8	no_match	no_match	wm/BP_008_BP21_BA.png	lure	Body	126.79	0.855	0.855	
0-Back	9	match	match	wm/BP_007_BP20_BA.png	target	Body	129.323	1.94	1.94	
0-Back	10	no_match	no_match	wm/BP_012_BP5_BA.png	nonlure	Body	131.856	0.686	0.686	
Cue				wm/PL_001_92012.png			149.372			
0-Back	1	no_match	no_match	wm/PL_003_92016.png	nonlure	Place	151.888	0.733	0.733	
0-Back	2	no_match	no_match	wm/PL_003_92016.png	lure	Place	154.421	0.775	0.775	
0-Back	3	no_match	no_match	wm/PL_006_92025.png	nonlure	Place	156.955	0.721	0.721	
0-Back	4	match	match	wm/PL_001_92012.png	target	Place	159.488	0.705	0.705	
0-Back	5	no_match	no_match	wm/PL_004_92023.png	nonlure	Place	162.021	0.72	0.72	
0-Back	6	no_match	no_match	wm/PL_005_92024.png	nonlure	Place	164.554	0.758	0.758	
0-Back	7	match	match	wm/PL_001_92012.png	target	Place	167.087	0.963	0.963	
0-Back	8	no_match	no_match	wm/PL_002_92014.png	nonlure	Place	169.62	0.652	0.652	
0-Back	9	no_match	no_match	wm/PL_004_92023.png	lure	Place	172.154	0.755	0.755	
0-Back	10	no_match	no_match	wm/PL_006_92025.png	lure	Place	174.687	0.714	0.714	
2-Back	1		no_match	wm/FC_010_F5.png	nonlure	Face	179.722	0.0	0.0	
2-Back	2	no_match	no_match	wm/FC_012_F6.png	nonlure	Face	182.255	0.503	0.503	
2-Back	3	match	match	wm/FC_010_F5.png	target	Face	184.788	0.762	0.762	
2-Back	4	no_match	no_match	wm/FC_008_F4.png	nonlure	Face	187.321	1.708	1.708	
2-Back	5	no_match	no_match	wm/FC_012_F6.png	lure	Face	189.855	1.278	1.278	
2-Back	6	match	match	wm/FC_008_F4.png	target	Face	192.387	0.638	0.638	
2-Back	7	no_match	no_match	wm/FC_007_M4.png	nonlure	Face	194.921	1.105	1.105	
2-Back	8	no_match	no_match	wm/FC_007_M4.png	lure	Face	197.454	1.372	1.372	
2-Back	9	no_match	no_match	wm/FC_009_M5.png	nonlure	Face	199.987	1.352	1.352	
2-Back	10	no_match	no_match	wm/FC_011_M6.png	nonlure	Face	202.519	0.705	0.705	
Cue				wm/TO_007_TOOL7_BA.png			220.034			
0-Back	1	no_match	no_match	wm/TO_010_TOOL10_BA.png	nonlure	Tools	222.551	0.77	0.77	
0-Back	2	no_match	no_match	wm/TO_009_TOOL9_BA.png	nonlure	Tools	225.084	0.724	0.724	
0-Back	3	no_match	no_match	wm/TO_009_TOOL9_BA.png	lure	Tools	227.617	0.598	0.598	
0-Back	4	no_match	no_match	wm/TO_008_TOOL8_BA.png	nonlure	Tools	230.15	0.93	0.93	
0-Back	5	no_match	no_match	wm/TO_011_TOOL11_BA.png	nonlure	Tools	232.683	0.95	0.95	
0-Back	6	no_match	no_match	wm/TO_012_TOOL12_BA.png	nonlure	Tools	235.217	0.635	0.635	
0-Back	7	no_match	no_match	wm/TO_010_TOOL10_BA.png	lure	Tools	237.75	0.468	0.468	
0-Back	8	match	match	wm/TO_007_TOOL7_BA.png	target	Tools	240.283	0.68	0.68	
0-Back	9	no_match	no_match	wm/TO_008_TOOL8_BA.png	lure	Tools	242.816	0.53	0.53	
0-Back	10	match	match	wm/TO_007_TOOL7_BA.png	target	Tools	245.349	0.583	0.583	
2-Back	1	no_match	no_match	wm/PL_007_92026.png	nonlure	Place	250.382	1.597	1.597	
2-Back	2	no_match	no_match	wm/PL_007_92026.png	lure	Place	252.916	1.241	1.241	
2-Back	3	no_match	no_match	wm/PL_010_92038.png	nonlure	Place	255.449	1.103	1.103	
2-Back	4	no_match	no_match	wm/PL_009_92037.png	nonlure	Place	257.982	0.854	0.854	
2-Back	5	match	match	wm/PL_010_92038.png	target	Place	260.515	0.729	0.729	
2-Back	6	match	match	wm/PL_009_92037.png	target	Place	263.048	1.402	1.402	
2-Back	7	no_match	no_match	wm/PL_008_92033.png	nonlure	Place	265.581	0.78	0.78	
2-Back	8	no_match	no_match	wm/PL_012_92047.png	nonlure	Place	268.115	0.769	0.769	
2-Back	9	no_match	no_match	wm/PL_012_92047.png	lure	Place	270.648	0.865	0.865	
2-Back	10	no_match	no_match	wm/PL_011_92044.png	nonlure	Place	273.181	1.357	1.357	

I don’t know that there’s anything wrong with your model. It seems to be a pybids bug, where an “extension” column is expected to be in the pandas dataframe. We then drop them without telling pandas to ignore missing columns.

It’s a bit odd that it’s not present, as extension should be treated like an entity by pybids. So fixing that issue might still lead to other issues.

Did you get a .pklz file? If so, then you can rerun with singularity exec <container> nipypecli crash -r -d <pklz> and get a PDB shell and see what actually is in the dataframe.

1 Like

Thanks Chris! I’m trying to do this.

Somehow I’m not even able to execute nipypecli, i get the error:

singularity exec /om2/user/jsmentch/fitlins-latest.simg nipypecli
FATAL:   "nipypecli": executable file not found in $PATH

Within singularity shell I get the same error. ( I am using apptainer 1.1.7) Do you know how to get nipypecli command in my path in singularity or perhaps I can download nipypecli elsewhere.

The problem is that you need to run nipypecli in the same environment that generated the pklz files, or it’s very unlikely to work. Possibly creating a conda environment with the same Python and nipype versions would be sufficient.

What happens when you run singularity shell and type which fitlins? nipypecli should be in the same directory in the container.

Ah, looks like I used neurodocker. Try:

singularity exec /om2/user/jsmentch/fitlins-latest.simg /neurodocker/startup.sh nipypecli
1 Like

I’m not familiar with PDB but when I run the command on result_loader.pklz there is the following error:

(base) bash-4.4$ ls fitlins/work_subject/fitlins_wf/loader/
_inputs.pklz  _node.pklz  _report  result_loader.pklz

(base) bash-4.4$ singularity exec -B /om2 /om2/user/jsmentch/fitlins-latest.simg  /neurodocker/startup.sh nipypecli crash -r -d fitlins/work_subject/fitlins_wf/loader/result_loader.pklz
Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/bin/nipypecli", line 8, in <module>
    sys.exit(cli())
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/scripts/cli.py", line 96, in crash
    display_crash_file(crashfile, rerun, debug, dir)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/nipype/scripts/crash_files.py", line 53, in display_crash_file
    if "node" in crash_data:
TypeError: argument of type 'InterfaceResult' is not iterable

Sorry, I meant a crashfile. It should be something godawful like crashfile-kljfaslkjdflkasdjklfjas;ldkjfls.pklz. I think it’ll either be in the working or output directory. If it ends in .txt then we’ll need to rerun with a patch to use pklz instead so you can dig in.

Ah! Unfortunately, it is a .txt file. Is there some instructions somewhere on how to rerun with a patch?

Save the following as nipype.cfg:

[execution]
crashfile_format = pklz

Then add -B $CWD/nipype.cfg:/opt/miniconda-latest/envs/neuro/lib/python3.9/site-packages/fitlins/data/nipype.cfg to your singularity commands.

There might be a better way, but it’s been so long since I’ve worked actively on fitlins I don’t really want to figure out the “right” way and then debug when it inevitably goes wrong.

One thing: If you’re taking 45 minutes to reach loader, then I think it’s likely that you’re spending a bunch of time building your pybids database. Add --database-path /some/location. It will still take a while for your next run, but it will be almost instant in future runs.

Thanks, that is working now!

I am fumbling around in pdb trying to find the dataframe in question and what is going on. I’m able to go up and down and print the variables local() and access the result object but I’m a bit lost. If I step forward it crashes and kicks me out. Any pointers on what exactly to look for here or how to get there?

This is the frame you want, if you can get there. You’ll want to look at the ent_cols dataframe.