NetworkBasedStatistic failed to run - what to set edge_key to?

, ,
1

Hi,
I am trying to run NetworkBasedStatistic on dMRI graphs. I get the following error message:

network,pval,files = nbs.run()
*** KeyError: 'number_of_fibers\nInterface NetworkBasedStatistic failed to run. '

I am wondering if I should be doing something about the edge_key option? I tried setting it as “weight” or “value”, but this did not help. I am wondering if I should have set an edge_key when creating the networks, and if so, when?
I would greatly appreciate any advice.

graphlist = []
# Load flies and transform to pck format
for idx,name in enumerate(graphnames):  #graphnames is a list of csv files
    file = cur_path+name
    # Read in CSV file
    graph = genfromtxt(file, delimiter=',')
    # Transform to Networkx graph
    G = nx.from_numpy_matrix(graph)
    out_path = cur_path + graphnames[idx] + '.pck'
    # Save as pck file
    nx.write_gpickle(G, out_path)
    graphlist.append(out_path)

ASD = graphlist[0:23]
control = graphlist[23:len(graphlist)+1]

nbs = cmtk.NetworkBasedStatistic()
nbs.inputs.in_group1 = ASD
nbs.inputs.in_group2 = control
nbs.inputs.edge_key = "weight"
network,pval,files = nbs.run()
2

Hi @dorfschl, do you have exactly the same error when you provide different edge_key? or don’r provide anything?

Any chances that you have a small example available somewhere, so I can run and test it?

3

Hi @djarecka,
thank you for your quick reply! I have to correct myself: I was previously running the code step by step using the python debugger pdb, which produced the error message you see in my first post.
Running the code all at one, I get the following error message for edge_key = "weight"

  File "nbs_try.py", line 38, in <module>
network,pval,files = nbs.run()
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/base.py", line 1081, in run
runtime = self._run_wrapper(runtime)
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/base.py", line 1029, in _run_wrapper
runtime = self._run_interface(runtime)
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/cmtk/nbs.py", line 90, in _run_interface
PVAL, ADJ, _ = nbs.compute_nbs(X, Y, THRESH, K, TAIL)
NameError: name 'nbs' is not defined
Interface NetworkBasedStatistic failed to run.

I went to the respective script and I am guessing it is not importing the cviewer as it should in line 24 of this https://github.com/nipy/nipype/blob/3c3d2e9bc5e5f288b8dc7d13e6d4d2f8a44576f5/nipype/interfaces/cmtk/nbs.py#L27

When I don’t set an edge_key, i.e. run it with the default key, as well as with any other key, I get the following message, where (enter edge_key here) ist the edge_key that I provide.

  File "nbs_try.py", line 36, in <module>
    nbs = cmtk.NetworkBasedStatistic()
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/base.py", line 1081, in run
    runtime = self._run_wrapper(runtime)
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/base.py", line 1029, in _run_wrapper
    runtime = self._run_interface(runtime)
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/cmtk/nbs.py", line 87, in _run_interface
    X = ntwks_to_matrices(self.inputs.in_group1, edge_key)
  File "/usr/local/lib/python3.6/site-packages/nipype/interfaces/cmtk/nbs.py", line 35, in ntwks_to_matrices
    graph[u][v]['weight'] = d[edge_key]  # Setting the edge requested edge value as weight value
KeyError: '(*enter edge_key here*)\nInterface NetworkBasedStatistic failed to run. '

I should add that I am running the code on a Mac - I thought using nypipe I might get around the problem of the connectome viewer suggesting to install on Debian. Thinking that might the problem, I set everything to run on Virtual Box running Debian as described in: http://www.connectomics.org/viewer/documentation/users/installation.html

Running it on that machine, I get the following error message:

brain@neurodebian:~/share$ sudo python nbs_try.py 
Traceback (most recent call last):
  File "nbs_try.py", line 46, in <module>
    network,pval,files = nbs.run()
  File "/usr/lib/python2.7/dist-packages/nipype/interfaces/base.py", line 1033, in run
    runtime = self._run_wrapper(runtime)
  File "/usr/lib/python2.7/dist-packages/nipype/interfaces/base.py", line 983, in _run_wrapper
    runtime = self._run_interface(runtime)
  File "/usr/lib/python2.7/dist-packages/nipype/interfaces/cmtk/nbs.py", line 87, in _run_interface
    PVAL, ADJ, _ = nbs.compute_nbs(X, Y, THRESH, K, TAIL)
  File "/usr/lib/pymodules/python2.7/cviewer/libs/pyconto/groupstatistics/nbs/_nbs.py", line 173, in compute_nbs
    nr_edges_per_component = np.zeros( len(comp_list) )
TypeError: object of type 'generator' has no len()
Interface NetworkBasedStatistic failed to run.

You can reproduce the problem by simply creating random matrices. Replace the readcsv line in the code from above by graph = np.random.rand(69,68).

Thanks a lot for your help!

4

wow, many various errors :wink: I will try to explain all. Thanks for suggesting the easiest way to reproduce your errors, very helpful!

  • “NameError: name ‘nbs’ is not defined”
    you’re right that nipype does not solve the problem of having connectomeviewer. And the error means that the import statement failed, but I agree that the error should be more explanatory, will change it.

  • KeyError: '(enter edge_key here)\nInterface NetworkBasedStatistic failed to run. ’
    You have different error for other edge_keys because it fails earlier. If you use different edge_key, function ntwks_to_matrices expects that your edges have proper attributes, e.g. number_of_fibers, so if your edges don’t have the attribute this line gives error.
    The function you’re using, nx.from_numpy_matrix, returns graph that have weight as attribute (you can check this G.edges(data=True)). You can find here the way how to change the attributes of your edges.
    I’ll write a better error message so it’s easier to understand the problem.

  • TypeError: object of type ‘generator’ has no len()
    I tried to run your example using nuerodebian docker image after installing connectomeviewer and was able to reproduce the error. Unfortunately, this is not nipype error, but inside cviewer package. comp_list is a generator and you can’t ask for len(generator). I’m guessing that it was a list in an older version of networkx, but now I had to change to comp_list = list(netwx.connected_component_subgraphs(G)) in two places to make it work. I can try to write a pull request to the cviewer package, but not sure if anyone is working on it anymore (last changes were 6 years ago).

  • network,pval,files = nbs.run() will fail
    After I changed cviewer code, nbs interface runs, but you can’t ask for network,pval,files like this. You can ask for aa.outputs or aa.outputs.nbs_network, etc.

5

My small PR to connectomviewer were merged: https://github.com/LTS5/connectomeviewer/pull/55