Confounds going missing with `model.fit` in nilearn

I am performing a task analysis using fMRIPREP and nilearn. For a handful of participants, the confound matrix is showing up as all NaN values, despite the fact that the *_desc-confounds_timeseries.tsv file looks complete.

I created the first-level model with first_level_from_bids(), and am looping over the models with code that looks like

for subject, model, imgs, event, confound in zip(
    sub_labels, models_ord, models_run_imgs_ord, models_events_ord, 
    models_confounds_ord
):
    m = model.fit(imgs, events=event, confounds=confound)

If I ask it to print confound I get:

[    cosine00  rot_x  rot_y  rot_z  trans_x  trans_y  trans_z
0        NaN    NaN    NaN    NaN      NaN      NaN      NaN
1        NaN    NaN    NaN    NaN      NaN      NaN      NaN
...

This is happening for three participants; it’s not always the same scan, and the scans have the expected number of volumes. I am a bit lost on what might be happening here.

Hi @Trevor_Day,

Did fmriprep finish successfully on all these subjects? Is it always the same three subjects that get this behavior? Does the tree structure of those three subjects’ fmriprep outputs look okay? Does the code deliver any warning messages? Can you share the rest of the code that precedes that?

Best,
Steven

As far as I can tell, fmriprep finished successfully and the outputs look correct. The bold Niftis, surface files, and confounds exist for all the runs that exist in the BIDS directory (there should be 4, but one is missing run #4). Is there a specific file to check?

This is the complete error:

Traceback (most recent call last):
  File "/Users/tkmd/code/nilearn//task_analysis.py", line 329, in <module>
    m = model.fit(imgs, events=event, confounds=confound)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tkmd/miniconda3/lib/python3.12/site-packages/nilearn/glm/first_level/first_level.py", line 669, in fit
    design = make_first_level_design_matrix(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tkmd/miniconda3/lib/python3.12/site-packages/nilearn/glm/first_level/design_matrix.py", line 419, in make_first_level_design_matrix
    matrix, _ = full_rank(matrix)
                ^^^^^^^^^^^^^^^^^
  File "/Users/tkmd/miniconda3/lib/python3.12/site-packages/nilearn/glm/_utils.py", line 174, in full_rank
    U, s, V = spl.svd(X, full_matrices=False)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tkmd/miniconda3/lib/python3.12/site-packages/scipy/linalg/_decomp_svd.py", line 108, in svd
    a1 = _asarray_validated(a, check_finite=check_finite)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tkmd/miniconda3/lib/python3.12/site-packages/scipy/_lib/_util.py", line 321, in _asarray_validated
    a = toarray(a)
        ^^^^^^^^^^
  File "/Users/tkmd/miniconda3/lib/python3.12/site-packages/numpy/lib/_function_base_impl.py", line 656, in asarray_chkfinite
    raise ValueError(
ValueError: array must not contain infs or NaNs

There’s a bunch of irrelevant code in my script, but this is the complete first_level_from_bids call:

(

    models,
    models_run_imgs,
    models_events,
    models_confounds,

) = first_level_from_bids(

    bids_dataset,
    task_label=task_label,
    space_label=space_label,
    sub_labels=sub_labels,
    derivatives_folder=derivatives_folder,

    smoothing_fwhm=5.0,
    drift_model=None,

    # Add more things to regression strategy 
    # Set motion threshold to 0.5 mm for task 
    confounds_strategy=settings["confounds_strategy"],
    confounds_motion=settings["confounds_motion"],
    confounds_fd_threshold=settings["confounds_fd_threshold"],

    # Running settings
    # verbose=2,
    minimize_memory=False,

    # Set these values to None to infer
    t_r=None,
    slice_time_ref=None,
    hrf_model = settings["hrf"],

    # job control
    # n_jobs=-2

)

I’m not sure what you mean by “always,” but they fail each time i run it; and none of the others (across two projects and three tasks) are dying with this error.

Thanks!

Hi @Trevor_Day

Can you see where the Nan or infinity is happening in the confounds? I imagine it may be in the first or last value of a column. If so, it’s pretty conventional to just make it the average of the rest of the column, as long as you are transparent about that in publications.

Best,
Steven

Hi @Steven,

Thanks - but maybe I wasn’t clear in my initial post.

I’m currently just using the high pass filter and the 6 basic motion parameters - and the entire confound matrix shows up as NaNs when I print it from nilearn.

The TSV file itself has no n/as in the relevant columns at all.

Hi @Trevor_Day,

Were there any warnings in the steps leading up to that error? I am not sure where models_confounds_ord is defined. Although, this kind of problem might be hard to troubleshoot without getting hands on with the data myself. Just to confirm, if you make a BIDS/fmriprep output dir with just those subjects, does the error persist (and if so can you share it)?

Best,
Steven

No, no errors, I set first_level_models_from_bids() verbosity to 3 and don’t see anything.

When Remi first shared some code with me; they weren’t sorted in alphabetical order, so I
resorted all of them to make sure the label matched the correct data. I think he created a fix, but that code is still floating around in my analysis.

models_orig = [m.subject_label for m in models]
new_order = np.argsort(models_orig).tolist()

models_ord = [models[i] for i in new_order]
models_run_imgs_ord = [models_run_imgs[i] for i in new_order]
models_events_ord = [models_events[i] for i in new_order]
models_confounds_ord = [models_confounds[i] for i in new_order]

This morning, I ran dos2unix and deleted the confounds files and re-ran fmriprep with no success.
I copied the BIDS and derivatives output as you suggested to a new place and received the same error.

Here is the log:

Using nilearn version: 0.10.4
['high_pass', 'motion', 'scrub']
Contrasts:
{'col': 't', 'col - len': 't', 'len': 't', 'len + col': 't', 'len - col': 't'}

Participants:
['TDCh166']

Checking settings ...
Settings OK!

{'confounds_fd_threshold': 0.5,
 'confounds_motion': 'basic',
 'confounds_strategy': ['high_pass', 'motion', 'scrub'],
 'hrf': 'spm'}
Found the following 3 preprocessed BOLD files
 for subject TDCh166
 for filter: [('task', 'sline'), ('space', 'MNI152NLin2009cAsym')]:
 ['/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz', '/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-2_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz', '/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-3_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz']

Found the following 3 events files
 for subject TDCh166
 for filter: [('task', 'sline')]:
 ['/Users/tkmd/Projects/debug_nilearn/bids/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_events.tsv', '/Users/tkmd/Projects/debug_nilearn/bids/sub-TDCh166/func/sub-TDCh166_task-sline_run-2_events.tsv', '/Users/tkmd/Projects/debug_nilearn/bids/sub-TDCh166/func/sub-TDCh166_task-sline_run-3_events.tsv']

Found the following 3 confounds files
 for subject TDCh166
 for filter: [('task', 'sline')]:
 ['/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_desc-confounds_timeseries.tsv', '/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-2_desc-confounds_timeseries.tsv', '/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-3_desc-confounds_timeseries.tsv']


Checking output dir testing/sub-TDCh166/ ...
    /Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz exists
    /Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-2_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz exists
    /Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-3_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz exists
    Found 0 for task-sline len (need 5)
    Found 0 for task-sline col (need 5)
    Found 0 for task-sline lenMinusCol (need 5)
    Found 0 for task-sline colMinusLen (need 5)
    Found 0 for task-sline lenPlusCol (need 5)
  Result of checking testing/sub-TDCh166/ is: NOT OK
TDCh166
FirstLevelModel(drift_model=None, hrf_model='spm', minimize_memory=False,
                slice_time_ref=0.48866666666666664, smoothing_fwhm=5.0,
                subject_label='TDCh166', t_r=3, verbose=4)
['/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',
 '/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-2_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',
 '/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-3_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz']
[   onset  duration trial_type difficulty
0      9         9      instr        NaN
1     18        24        col       easy
2     51         9      instr        NaN
3     60        24        len       easy
4     93         9      instr        NaN
5    102        24        col       easy
6    135         9      instr        NaN
7    144        24        len       easy,
    onset  duration trial_type difficulty
0      9         9      instr        NaN
1     18        24        len       easy
2     51         9      instr        NaN
3     60        24        col       easy
4     93         9      instr        NaN
5    102        24        len       easy
6    135         9      instr        NaN
7    144        24        col       easy,
    onset  duration trial_type difficulty
0      9         9      instr        NaN
1     18        24        col       easy
2     51         9      instr        NaN
3     60        24        len       easy
4     93         9      instr        NaN
5    102        24        col       easy
6    135         9      instr        NaN
7    144        24        len       easy]
[    cosine00     rot_x     rot_y     rot_z   trans_x   trans_y   trans_z
0   0.002905 -0.000586 -0.000407 -0.000512 -0.011132  0.002133 -0.010655
1   0.002405  0.000470 -0.000407 -0.000953 -0.024206  0.075562  0.011215
2   0.001406 -0.001865  0.000244  0.000318  0.026202 -0.024175  0.028371
3  -0.000089  0.000156  0.000438 -0.000570 -0.016689 -0.029033  0.009925
4  -0.002077  0.000021  0.000088  0.000716  0.039716  0.028713 -0.012355
5  -0.004551  0.001803  0.000044  0.001001 -0.013891 -0.053201 -0.026501
6  -0.007504  0.034715 -0.000118  0.001128 -0.054492  0.473855 -0.299136
7  -0.010930 -0.008917 -0.000535 -0.000161  0.081872 -0.118512  0.022246
8  -0.014817  0.004083  0.003468 -0.000368 -0.091523  0.547858  0.550016
9  -0.019157  0.006228 -0.003399 -0.001330  0.173817  0.291950 -0.018679
10 -0.023936 -0.005576  0.002248  0.000760  0.016568 -0.063893 -0.021023
11 -0.029142 -0.004186  0.001200  0.000403  0.034860  0.207336  0.019725
12 -0.034760 -0.002523  0.000946  0.000556  0.014752 -0.021058  0.091258
13 -0.040776 -0.001661 -0.000124  0.000985  0.026459  0.018816 -0.112484
14 -0.047172  0.003172 -0.000236 -0.000057 -0.009048  0.157104 -0.141456
15 -0.053930 -0.005105 -0.001641 -0.001268  0.013267  0.062327  0.005041
16 -0.061034 -0.006684 -0.000645  0.000081  0.042783  0.013254  0.008025
17 -0.068462 -0.005091 -0.000885  0.000488  0.027773  0.056810 -0.001812
18 -0.076195 -0.002316 -0.001149  0.000015 -0.006623 -0.061844 -0.028265
19 -0.084212  0.002849 -0.000679 -0.000611 -0.021132  0.028978 -0.103702
20 -0.092489 -0.001568 -0.000266 -0.000397  0.062027  0.196189 -0.084251
21 -0.101006  0.002373  0.000365 -0.001020 -0.009763  0.129540  0.037122
22 -0.109738 -0.008355 -0.000602 -0.001020 -0.010112  0.143012  0.129365
23 -0.118662 -0.000665 -0.000541  0.000218  0.020161  0.147775 -0.060855
24 -0.127752 -0.001839  0.000547  0.001526  0.026667 -0.164279 -0.039638
25 -0.136985 -0.000299 -0.001999  0.000353 -0.082634  0.415324  0.156655
26 -0.146335  0.005590  0.000751 -0.000391  0.037896 -0.117283 -0.102817
27 -0.155776 -0.004381 -0.003901 -0.000075  0.074240 -0.370815 -0.312230
28 -0.165282  0.002617  0.000748 -0.001044 -0.086120  0.162528  0.120321
29 -0.174827 -0.006614 -0.001969  0.000017  0.067409 -0.429071 -0.310423
30 -0.184385 -0.001421  0.000081 -0.000391  0.053247 -0.124475 -0.102523
31 -0.193931 -0.000020  0.000385  0.000119  0.019713  0.210135  0.038228
32 -0.203437 -0.005306  0.006216  0.000261 -0.131694  0.094159  0.042579
33 -0.212878 -0.003353 -0.008150 -0.000034  0.170979  0.116279  0.077369
34 -0.222227  0.000219  0.001421 -0.000987 -0.062443 -0.050089  0.019880
35 -0.231460 -0.001078 -0.002191 -0.000332  0.035086 -0.017927 -0.048235
36 -0.240551  0.004455 -0.002030 -0.000332 -0.000897 -0.072145  0.104331
37 -0.249474  0.007768 -0.000816 -0.002906 -0.031113  0.083437  0.074113
38 -0.258206  0.012639  0.004615 -0.002906 -0.143194  0.278417  0.054024
39 -0.266723 -0.002513 -0.002262  0.001666  0.098257 -0.363283 -0.491482
40 -0.275001 -0.007536 -0.004399  0.000883  0.079101 -0.017180 -0.212916
41 -0.283017 -0.006562 -0.000513  0.000370  0.040495 -0.022112 -0.001700
42 -0.290750 -0.010015 -0.000944 -0.000247 -0.033071  0.091848  0.009180
43 -0.298179 -0.020426 -0.005208  0.001208  0.123079 -0.422152 -0.375769
44 -0.305282 -0.004851 -0.000784 -0.000284 -0.011016  0.174039  0.072560
45 -0.312041 -0.005343 -0.000211  0.000512  0.038905 -0.082693  0.051066
46 -0.318437 -0.008199  0.001409  0.000300 -0.061633  0.018267  0.152888
47 -0.324452  0.004049 -0.001035 -0.000870 -0.028430  0.046089  0.011142
48 -0.330070 -0.008092 -0.000394  0.001802  0.053442  0.310399  0.331643
49 -0.335276 -0.007777  0.000194 -0.001653 -0.033769  0.050716  0.410164
50 -0.340056  0.000630 -0.000796  0.000909  0.050413 -0.016520 -0.010473
51 -0.344395  0.002709 -0.000370 -0.001262  0.012058  0.062132 -0.063974
52 -0.348283 -0.000484 -0.000620  0.000228  0.011992 -0.148095 -0.012807
53 -0.351708 -0.003427 -0.000534  0.000787  0.011878 -0.154785  0.003982
54 -0.354662 -0.000544 -0.000370  0.000787 -0.013644 -0.164971 -0.046727
55 -0.357136  0.001490 -0.001514 -0.000154  0.011940 -0.119021 -0.219626
56 -0.359123 -0.002770  0.000383  0.000356  0.055240 -0.131498 -0.280985
57 -0.360619 -0.018432 -0.004134 -0.001236  0.030474 -0.995640 -0.652357
58 -0.361618 -0.001575 -0.002216 -0.000438  0.002598  0.161398  0.040405
59 -0.362118 -0.001805  0.000227 -0.000485 -0.056243  0.179370  0.524051,
     cosine00     rot_x     rot_y     rot_z   trans_x   trans_y   trans_z
0   0.001995  0.000999  0.000104  0.000252 -0.002253  0.003171  0.015020
1   0.001495 -0.002049  0.000364  0.000033 -0.008453 -0.041940 -0.017930
2   0.000496 -0.003594  0.001014  0.000356 -0.000842  0.008357 -0.054980
3  -0.000999  0.001469 -0.000853 -0.000070 -0.003306 -0.031187  0.020580
4  -0.002987  0.003175 -0.000629 -0.000571  0.014854  0.061600  0.037310
5  -0.005461 -0.002207  0.000406  0.000532 -0.001521 -0.032588 -0.242630
6  -0.008414 -0.001795  0.001395  0.000548 -0.024587  0.080070 -0.224690
7  -0.011840  0.004130  0.002540  0.001369 -0.054910  0.346250  1.124086
8  -0.015728 -0.002661 -0.000656  0.000885  0.057043 -0.314911 -0.190080
9  -0.020067  0.004136  0.000363 -0.000168 -0.011808 -0.118201 -0.400580
10 -0.024846  0.003524  0.001130  0.000258 -0.031530 -0.153568  0.039460
11 -0.030052 -0.001534  0.000825 -0.000312  0.021848 -0.211999 -0.024060
12 -0.035671 -0.001385  0.000362 -0.000312 -0.024522 -0.173542  0.043510
13 -0.041686  0.019572 -0.003224 -0.002635 -0.055655 -0.228854 -1.055870
14 -0.048082 -0.016018  0.002115  0.002639  0.023194 -0.198596  0.446040
15 -0.054841  0.009935 -0.004713 -0.002237 -0.039868 -0.409177 -1.389160
16 -0.061944 -0.003213  0.000895  0.000063 -0.025008 -0.008964 -0.074440
17 -0.069372 -0.000422  0.000112  0.000063  0.003651 -0.018552  0.197030
18 -0.077105  0.002074  0.003670  0.000285 -0.073447 -0.205581 -0.211580
19 -0.085122  0.002662  0.001229  0.000515  0.053257 -0.039732  0.028710
20 -0.093399  0.003409 -0.001561 -0.002371 -0.048684  0.584896  1.568999
21 -0.101916  0.029383  0.001809 -0.002882 -0.037660 -0.577989 -1.595280
22 -0.110648 -0.009652  0.003805  0.003023 -0.017808  0.079701  1.110243
23 -0.119572  0.006619 -0.004210 -0.001146  0.035840 -0.324252 -1.222770
24 -0.128662 -0.011476  0.002484  0.001360 -0.056476 -0.214890  0.104400
25 -0.137895 -0.006678  0.000018  0.001026 -0.005680  0.016239 -0.162870
26 -0.147245  0.009502 -0.003093 -0.000823  0.093023  0.114185 -0.098130
27 -0.156686 -0.011928  0.001764  0.000935  0.009954 -0.236761  0.040100
28 -0.166192 -0.004695  0.000901  0.000470  0.034805  0.005446 -0.116710
29 -0.175737  0.003032 -0.000848 -0.000188  0.016693  0.020243  0.036580
30 -0.185296 -0.002423  0.000869  0.000012 -0.023303 -0.023442 -0.093310
31 -0.194841 -0.005603  0.001020  0.000871  0.033615 -0.004125 -0.150490
32 -0.204347 -0.002185 -0.000028 -0.000242 -0.008727 -0.057018 -0.102780
33 -0.213788 -0.003776  0.000427 -0.000794 -0.038864  0.037018 -0.005150
34 -0.223137  0.002401 -0.000327  0.000917  0.034453  0.037814  0.075960
35 -0.232370  0.002645  0.004185 -0.000105 -0.124152  0.057050 -0.257870
36 -0.241461 -0.000829 -0.005053 -0.001560  0.053822  0.013066  0.615010
37 -0.250384  0.002025  0.001439  0.001528 -0.006948 -0.177783  0.107810
38 -0.259117  0.005538  0.001159  0.002788  0.051590 -0.023009  0.127360
39 -0.267633 -0.000830 -0.001288 -0.000864 -0.046458 -0.326724 -0.338590
40 -0.275911  0.001425 -0.004897 -0.002098 -0.031106 -0.106477 -0.932460
41 -0.283928 -0.000657  0.006209  0.003286 -0.067565 -0.839779 -1.256790
42 -0.291660  0.001861 -0.001011 -0.001215 -0.010702  0.093402  0.281790
43 -0.299089 -0.011517 -0.006286  0.001540  0.111153 -0.376526 -0.168460
44 -0.306192  0.003083  0.000921 -0.000555 -0.062096 -0.070702  0.027620
45 -0.312951 -0.001161  0.000515 -0.000555  0.018087  0.151541  0.172910
46 -0.319347 -0.019274  0.005440  0.001977 -0.080093 -0.671249 -0.534770
47 -0.325362  0.000690 -0.001451  0.000262  0.035102  0.090311  0.027450
48 -0.330981  0.005873 -0.001451  0.000670 -0.045685  0.043822  0.181710
49 -0.336186  0.014744 -0.001451 -0.002154  0.062245  0.374480  0.420710
50 -0.340966 -0.000156  0.000197 -0.001370 -0.100767 -0.277436 -0.239180
51 -0.345305  0.007675  0.000051 -0.001043 -0.015663 -0.413529 -0.330650
52 -0.349193  0.000078  0.000162  0.001089 -0.005602  0.062662  0.167260
53 -0.352618  0.003699 -0.000714 -0.001069  0.060127 -0.244102 -0.999670
54 -0.355572 -0.006281 -0.000788  0.000827 -0.016054 -0.084490 -0.031400
55 -0.358046 -0.008283  0.002411  0.001553 -0.006290 -0.103336 -0.254220
56 -0.360033 -0.020809  0.000236  0.001301 -0.034358 -0.401234  0.415450
57 -0.361529 -0.001816 -0.003190  0.001077  0.044351 -0.246402 -0.559530
58 -0.362528 -0.002699  0.000046  0.000299 -0.003914 -0.007830 -0.135860
59 -0.363028 -0.001519  0.000199  0.000072 -0.014635 -0.133319 -0.089520,
     cosine00  rot_x  rot_y  rot_z  trans_x  trans_y  trans_z
0        NaN    NaN    NaN    NaN      NaN      NaN      NaN
1        NaN    NaN    NaN    NaN      NaN      NaN      NaN
2        NaN    NaN    NaN    NaN      NaN      NaN      NaN
3        NaN    NaN    NaN    NaN      NaN      NaN      NaN
4        NaN    NaN    NaN    NaN      NaN      NaN      NaN
5        NaN    NaN    NaN    NaN      NaN      NaN      NaN
6        NaN    NaN    NaN    NaN      NaN      NaN      NaN
7        NaN    NaN    NaN    NaN      NaN      NaN      NaN
8        NaN    NaN    NaN    NaN      NaN      NaN      NaN
9        NaN    NaN    NaN    NaN      NaN      NaN      NaN
10       NaN    NaN    NaN    NaN      NaN      NaN      NaN
11       NaN    NaN    NaN    NaN      NaN      NaN      NaN
12       NaN    NaN    NaN    NaN      NaN      NaN      NaN
13       NaN    NaN    NaN    NaN      NaN      NaN      NaN
14       NaN    NaN    NaN    NaN      NaN      NaN      NaN
15       NaN    NaN    NaN    NaN      NaN      NaN      NaN
16       NaN    NaN    NaN    NaN      NaN      NaN      NaN
17       NaN    NaN    NaN    NaN      NaN      NaN      NaN
18       NaN    NaN    NaN    NaN      NaN      NaN      NaN
19       NaN    NaN    NaN    NaN      NaN      NaN      NaN
20       NaN    NaN    NaN    NaN      NaN      NaN      NaN
21       NaN    NaN    NaN    NaN      NaN      NaN      NaN
22       NaN    NaN    NaN    NaN      NaN      NaN      NaN
23       NaN    NaN    NaN    NaN      NaN      NaN      NaN
24       NaN    NaN    NaN    NaN      NaN      NaN      NaN
25       NaN    NaN    NaN    NaN      NaN      NaN      NaN
26       NaN    NaN    NaN    NaN      NaN      NaN      NaN
27       NaN    NaN    NaN    NaN      NaN      NaN      NaN
28       NaN    NaN    NaN    NaN      NaN      NaN      NaN
29       NaN    NaN    NaN    NaN      NaN      NaN      NaN
30       NaN    NaN    NaN    NaN      NaN      NaN      NaN
31       NaN    NaN    NaN    NaN      NaN      NaN      NaN
32       NaN    NaN    NaN    NaN      NaN      NaN      NaN
33       NaN    NaN    NaN    NaN      NaN      NaN      NaN
34       NaN    NaN    NaN    NaN      NaN      NaN      NaN
35       NaN    NaN    NaN    NaN      NaN      NaN      NaN
36       NaN    NaN    NaN    NaN      NaN      NaN      NaN
37       NaN    NaN    NaN    NaN      NaN      NaN      NaN
38       NaN    NaN    NaN    NaN      NaN      NaN      NaN
39       NaN    NaN    NaN    NaN      NaN      NaN      NaN
40       NaN    NaN    NaN    NaN      NaN      NaN      NaN
41       NaN    NaN    NaN    NaN      NaN      NaN      NaN
42       NaN    NaN    NaN    NaN      NaN      NaN      NaN
43       NaN    NaN    NaN    NaN      NaN      NaN      NaN
44       NaN    NaN    NaN    NaN      NaN      NaN      NaN
45       NaN    NaN    NaN    NaN      NaN      NaN      NaN
46       NaN    NaN    NaN    NaN      NaN      NaN      NaN
47       NaN    NaN    NaN    NaN      NaN      NaN      NaN
48       NaN    NaN    NaN    NaN      NaN      NaN      NaN
49       NaN    NaN    NaN    NaN      NaN      NaN      NaN
50       NaN    NaN    NaN    NaN      NaN      NaN      NaN
51       NaN    NaN    NaN    NaN      NaN      NaN      NaN
52       NaN    NaN    NaN    NaN      NaN      NaN      NaN
53       NaN    NaN    NaN    NaN      NaN      NaN      NaN
54       NaN    NaN    NaN    NaN      NaN      NaN      NaN
55       NaN    NaN    NaN    NaN      NaN      NaN      NaN
56       NaN    NaN    NaN    NaN      NaN      NaN      NaN
57       NaN    NaN    NaN    NaN      NaN      NaN      NaN
58       NaN    NaN    NaN    NaN      NaN      NaN      NaN
59       NaN    NaN    NaN    NaN      NaN      NaN      NaN]
Design matrix is 60x7
    7 should be < 60
Design matrix is 60x7
    7 should be < 60
Design matrix is 60x7
    7 should be < 60
[NiftiMasker.fit] Loading data from /Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
[NiftiMasker.fit] Computing the mask
EPI mask computation
[NiftiMasker.fit] Resampling mask
[NiftiMasker.fit] Finished fit
[NiftiMasker.transform_single_imgs] Loading data from Nifti1Image('/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-1_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz')
[NiftiMasker.transform_single_imgs] Smoothing images
[NiftiMasker.transform_single_imgs] Extracting region signals
[NiftiMasker.transform_single_imgs] Cleaning extracted signals
[NiftiMasker.transform_single_imgs] Loading data from Nifti1Image('/Users/tkmd/Projects/debug_nilearn/fmriprep/sub-TDCh166/func/sub-TDCh166_task-sline_run-2_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz')
[NiftiMasker.transform_single_imgs] Smoothing images
[NiftiMasker.transform_single_imgs] Extracting region signals
[NiftiMasker.transform_single_imgs] Cleaning extracted signals

I’ve uploaded the data to Drive, and shared it with you by email (derivatives are uploading as I write this).

Sounds good, let me know when they are done.

It’s complete @Steven

Hi @Trevor_Day

There are no fmriprep outputs I can see.

Best,
Steven

Apologies, selected the wrong directory. Please check now

Hi @Trevor_Day,

I am unable to replicate your error. I do not know how the following are defined:

    confounds_strategy=settings["confounds_strategy"],
    confounds_motion=settings["confounds_motion"],
    confounds_fd_threshold=settings["confounds_fd_threshold"],

That being said, I ran a basic model using the settings I could understand from your code:

(
    models,
    models_run_imgs,
    models_events,
    models_confounds,
) = nilearn.glm.first_level.first_level_from_bids(
bids_path,
derivatives_folder=op.join(bids_path, "derivatives","fmriprep"),
task_label="sline",
smoothing_fwhm=5.0,
drift_model=None,
confounds_strategy=["motion"],
confounds_motion="basic",
t_r=None,
slice_time_ref=None)

and got a set of model confounds with no nans (and other expected outputs). I then tried getting all the confounds by removing the confounds_strategy=["motion"], confounds_motion="basic" arguments, and I get the full dataframe of all confounds as expected. Then I could get the subset of columns needed with something like

motion_keys = ["rot_x", "rot_y", "rot_z", "trans_x", "trans_y", "trans_z"]
cosine_keys = [key for key in models_confounds[0][run_idx].columns if "cosine" in key]
model_confounds_subset = models_confounds[0][run_idx][motion_keys + cosine_keys]

where run_idx is 0-2 for runs 1-3.

For whatever it is worth, I am using Nilearn 0.11.1.

Best,
Steven

Oh I see now. Sorry for omitting those - like you discussed at this post I was doing some testing of what confounds made sense to include so those were set in a bash script.

But, I see now that the FD was high and using the scrub strategy – which you didn’t include – was eliminating all frames. I didn’t think of it because framewise_displacement isn’t included in the confounds matrix.

I will add a check for whether confounds matrix is complete before running the model.

Thanks for your time! and happy new year

1 Like