Accessing MRIQC Web Service; errors and questions about downloading approach

I’m trying to download the most recent MRIQC results from the web API and am running into errors. Not sure how to make of these.

I can get a batch of responses, with, e.g.,

curl -X 'GET' 'https://mriqc.nimh.nih.gov/api/v1/T1w?page=1&max_results=50' 

(I think the max_result is capped at 50)

Some pages seem to result in errors consistently, e.g.

$ curl -X 'GET' 'https://mriqc.nimh.nih.gov/api/v1/T1w?page=97&max_results=50'  
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

Other pages result in either 500 or 502

curl -X 'GET' 'https://mriqc.nimh.nih.gov/api/v1/T1w?page=130&max_results=50' 
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.21.3</center>
</body>
</html>

$ curl -X 'GET' 'https://mriqc.nimh.nih.gov/api/v1/T1w?page=130&max_results=50' 
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

So, a couple of questions

  • Are there ways to check the server’s health (e.g., a health check endpoint)?
  • Should I expect to be able to download the full dataset (e.g., without stressing the server)?
  • Is it true that some pages/entries result in errors consistently?

Thanks!

Seems like that was just a bad day; trying again this week and the downloads are proceeding just fine :person_shrugging: