Problem in nimare.extract.download_abstracts(dset, 'xxx')

Software Support
1

Hi,
The program got stuck in step “nimare.extract.download_abstracts(dset, ‘xxx’)”, and showing warning like this:
import imp
WARNING:nimare.utils:Not applying transforms to coordinates in unrecognized space “UNKNOWN”
INFO:nimare.extract.extract:Downloading chunk 1 of 16
Traceback (most recent call last):
File “/usr/python/lib/python3.6/urllib/request.py”, line 1318, in do_open
encode_chunked=req.has_header(‘Transfer-encoding’))
File “/usr/python/lib/python3.6/http/client.py”, line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File “/usr/python/lib/python3.6/http/client.py”, line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File “/usr/python/lib/python3.6/http/client.py”, line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File “/usr/python/lib/python3.6/http/client.py”, line 1026, in _send_output
self.send(msg)
File “/usr/python/lib/python3.6/http/client.py”, line 964, in send
self.connect()
File “/usr/python/lib/python3.6/http/client.py”, line 1392, in connect
super().connect()
File “/usr/python/lib/python3.6/http/client.py”, line 936, in connect
(self.host,self.port), self.timeout, self.source_address)
File “/usr/python/lib/python3.6/socket.py”, line 704, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File “/usr/python/lib/python3.6/socket.py”, line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/prot/NiMARE/Neurosynth/download_neurosynth.py”, line 49, in
dset = nimare.extract.download_abstracts(dset, ‘tsalo006@fiu.edu’)
File “/usr/python/lib/python3.6/site-packages/nimare/extract/extract.py”, line 294, in download_abstracts
retmode=‘text’)
File “/usr/python/lib/python3.6/site-packages/Bio/Entrez/init.py”, line 205, in efetch
return _open(cgi, variables, post=post)
File “/usr/python/lib/python3.6/site-packages/Bio/Entrez/init.py”, line 602, in _open
handle = urlopen(cgi, data=options.encode(“utf8”))
File “/usr/python/lib/python3.6/urllib/request.py”, line 223, in urlopen
return opener.open(url, data, timeout)
File “/usr/python/lib/python3.6/urllib/request.py”, line 526, in open
response = self._open(req, data)
File “/usr/python/lib/python3.6/urllib/request.py”, line 544, in _open
‘_open’, req)
File “/usr/python/lib/python3.6/urllib/request.py”, line 504, in _call_chain
result = func(*args)
File “/usr/python/lib/python3.6/urllib/request.py”, line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
File “/usr/python/lib/python3.6/urllib/request.py”, line 1320, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

Process finished with exit code 1

I don’t understand what’s happening and hope to get directions. Thank you.

Best,
siyang

1 Like
2

I’m not familiar with this error. At a guess though, I’d say that your script might not have internet access. Did you submit this as a job to a scheduler like LSF or SLURM, or did you run it somewhere that definitely has internet access?

Also, could you share the version of Biopython you used?

On a minor note, would you mind editing the post title and your first comment to remove the email address in the function call?

Best,
Taylor

3

I think yes, I’m sure our server has connected to the Internet.
I’m using pycharm with python 3.6.
And I think there’s no LSF or SLURM.
I’ve edited my first comment. Sorry, I didn’t realize that before.

4

Hmm… please try running the following code, but make sure to use your own email address instead of the example one in the code:

from Bio import Entrez, Medline

Entrez.email = 'example@email.com'
h = Entrez.efetch(db='pubmed', id='19304878', rettype='medline', retmode='text')
result = list(Medline.parse(h))[0]
print(result['AU'])

If this fails, then it is possibly a problem with the Biopython (Bio) library, or a connectivity issue.

5

Hello, tsalo,
We mainly explore brain cognitive activities in fmri data, so we’re not familiar with Biopython. Now, I think I should install Bio successfully first, then nimare example can run sequentially. Am I right?

6

Hi Siyang,

It’s possible that Biopython wasn’t installed successfully, or that it’s an old version, but there could be other causes for the error. Still, updating the package will at least allow you to check, so it’s a good next step.

7

Okay, thank you very much, tsalo.

8

Hello, it’s me again. New problem came out. Do you know about these problems?

/usr/python/lib/python3.6/site-packages/scikit_learn-0.20.0-py3.6-linux-x86_64.egg/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module’s documentation for alternative uses
import imp
WARNING:nimare.utils:Not applying transforms to coordinates in unrecognized space “UNKNOWN”
INFO:nimare.extract.extract:Downloading chunk 1 of 16
Traceback (most recent call last):
File “/usr/python/lib/python3.6/urllib/request.py”, line 1318, in do_open
encode_chunked=req.has_header(‘Transfer-encoding’))
File “/usr/python/lib/python3.6/http/client.py”, line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File “/usr/python/lib/python3.6/http/client.py”, line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File “/usr/python/lib/python3.6/http/client.py”, line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File “/usr/python/lib/python3.6/http/client.py”, line 1026, in _send_output
self.send(msg)
File “/usr/python/lib/python3.6/http/client.py”, line 964, in send
self.connect()
File “/usr/python/lib/python3.6/http/client.py”, line 1392, in connect
super().connect()
File “/usr/python/lib/python3.6/http/client.py”, line 936, in connect
(self.host,self.port), self.timeout, self.source_address)
File “/usr/python/lib/python3.6/socket.py”, line 704, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File “/usr/python/lib/python3.6/socket.py”, line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/prot/NiMARE/Neurosynth/download_neurosynth.py”, line 51, in
dset = nimare.extract.download_abstracts(dset, ‘tsalo006@fiu.edu’)
File “/usr/python/lib/python3.6/site-packages/nimare/extract/extract.py”, line 294, in download_abstracts
retmode=‘text’)
File “/usr/python/lib/python3.6/site-packages/Bio/Entrez/init.py”, line 205, in efetch
return _open(cgi, variables, post=post)
File “/usr/python/lib/python3.6/site-packages/Bio/Entrez/init.py”, line 602, in _open
handle = urlopen(cgi, data=options.encode(“utf8”))
File “/usr/python/lib/python3.6/urllib/request.py”, line 223, in urlopen
return opener.open(url, data, timeout)
File “/usr/python/lib/python3.6/urllib/request.py”, line 526, in open
response = self._open(req, data)
File “/usr/python/lib/python3.6/urllib/request.py”, line 544, in _open
‘_open’, req)
File “/usr/python/lib/python3.6/urllib/request.py”, line 504, in _call_chain
result = func(*args)
File “/usr/python/lib/python3.6/urllib/request.py”, line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
File “/usr/python/lib/python3.6/urllib/request.py”, line 1320, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

Process finished with exit code 1

9

That looks like the same issue to me. My guess is that you either don’t have internet access on your cluster’s processing nodes (or wherever you run the script from) or you don’t have access to PubMed wherever you are.

If you’re using NiMARE version 0.0.3, try using this saved Dataset containing Neurosynth and associated abstracts: https://osf.io/jtc6n/. You won’t have to download abstracts from PubMed that way.

10

Okay, thank you. I’ll try this way.

11

Hello, again.
Now I successfully run the first two example scripts.
But the problem is that, I don’t understand what I’m doing in the second script (plot_dataset_io), and that means I still can’t do my own analysis.
So do you have any handbook or explanation in details to answer my questions?

And I’m so confused that what’s the differences between dataset downloaded in this step: dset_dir = nimare.extract.download_nidm_pain() and nidm_pain_dset.json.
Looking forward to you response. Thank you so much.

12

Writing a handbook for NiMARE is a good idea, but it’s not something I’m planning to do until NiMARE is more stable (around version 1.0.0). As it stands, there are too many things that will change from version to version for a handbook to remain relevant for very long.

The examples that are currently available are not sequential or tied together. They simply show a variety of things that can be done with NiMARE.

What is it you’re trying to do? NiMARE is designed to do a lot of different things with meta-analytic data, and generally only a small number of those things will be necessary for a given analysis. If I know what your analysis entails, I can probably point you toward the best examples, functions, etc.

We currently don’t have very solid NIDM Results integration or NeuroVault integration. As a result, we made a NiMARE dataset (nidm_pain_dset.json) with all of the metadata and coordinates, as well as links to the image files. Those statistical images, however, are not stored in the json file. Instead, you need to download them from NeuroVault, which is what nimare.extract.download_nidm_pain() does. The relative paths to the files downloaded with that function are the same as the ones in the json file, so you then just need to use Dataset.update_path() to match the paths to the files in the Dataset to the location of the files on your computer.

The NIDM pain dataset is helpful for testing image-based meta-analysis methods, since it has beta, standard error, z, and t maps for most of its contrasts. However, it’s a very small sample of studies, so I wouldn’t recommend trying to run any publishable analyses on it.

13

Developing a software is quite difficult and need a lot of work, especially when you’re trying to integrate many functions, so I understand and I’m looking forward to the stable version very much.

Actually I’m fresh new in meta analysis, and our event is in a very primary stage. So, at first, I thought the example scripts were tied, and wanted to get some inspirations during the learning. But, as you said, they’re written for showing functions and what nimare can do. And I think it may take lots of time for me to learn it and make sure what we’ll do then.

Now I’ve got an answer for what I asked, and I may consult you later when I meet problems or have some ideas but don’t know how to implement them.

Thank you again.

1 Like