Getting GIfTI surface data into Python

1

I am trying to transition from using AFNI for preprocessing + Matlab functions/scripts for analysis, into using BIDS for preprocessing + python functions/scripts for analysis.

I am currently trying to come up with code that can apply a bunch of ROIs (surface-space, stored in a GIfTI format) to preprocessed time-series data from several runs (also surface-space and GIfTI). Essentially, I would like to produce time-series for each ROI, averaged over the runs and over the ROI nodes.

I am using nilearn’s surface.load_surf_data, but it seems very slow, taking ~16 minutes to load a single hemisphere file, stored locally (~150,000 nodes x 126 TRs).

I have also tried using nibabel.load, something like:

img = nib.load(run_file)
img_data = [x.data for x in img.darrays]
cur_data = np.reshape(img_data,(len(img_data[0]),len(img_data)))

that takes less than two minutes per file, but I wonder if I can do it even faster.

These are steady-state fMRI data, so I could use a command line tool to average the data outside of python, but I would like the ability to look at individual run data.

So:

  1. Does anyone know of a better/faster tool for extracting averages into python from surface data, given some ROI mask?

  2. Does anyone know of a better/faster way to get surface data into python?

I would still be interested in solving (b), even if I figure out (a), since I would like to do whole-brain analysis at some point.

2

  1. Simply changing your GIfTI data type may provide sufficient performance: You may find dramatic performance differences depending on how your GIfTI data is encoded. GIfTI data can be stored as ASCII text, binary data encoded as base64 text, or zipped binary data encoded as base64 text. ASCII should be avoided at all cost, as it generates large files, rounding errors and is slow to read and save. The raw binary data will be the fastest, while the compressed raw will yield the smallest file size.

  2. Using a simpler format may provide sufficient performance: GIfTI is an inherently flexible format, but it is inherently slow. If you use this format you must have reasonable performance expectations. The requirement to encode/decode to base64 for each read/write has a penalty for performance and file size. In my tests, GIfTI (in its Gz) was abouthalf the speed to read and twice the file size as the simpler MZ3.

  3. You may want to consider using GIfTI as an interchange format and a simpler format for all your processing. Its easy to come up with a simple binary format, but you could always adopt MZ3 which is the native format of Surfice. The specification page includes a Blender plugin that shows this format in Python, MRIcroS has Matlab code.

  4. You may want to use Surfice - you can call it from Python and script it using either its own in-built Pascal scripts or you can use Python scripts if the user has Python installed. Pascal scripts are provided with the software, and example Python scripts are here.

  5. While not required, you could use Surfice to exchange data from/to a simpler format and GIfTI. The Surfice Python script shows how you can read a mesh from one format (mz3) and save to others:

    import gl
    gl.resetdefaults()
    gl.meshload(‘BrainMesh_ICBM152Right.mz3’)
    gl.meshsave(’~/ICBM152r.gii’)
    gl.meshsave(’~/ICBM152r.mz3’)

3

Professor Rorden, you rock!

Simply changing my giftis from ascii to base64 encoding cut down load time from 16 minutes to 2-4 seconds (with surface.load_surf_data).

I will look into surfice, and consider using mz3 for even faster loading.

Thanks so much and I hope you are able to stay dry.

4

Perhaps I am missing something, but how do I actually get access to the “gl” module you are importing in your example under (5)? I have installed the surf-ice and microgl dmg images into my Applications (I’m on a Mac).

5

The next release of Surfice will include better Python support, particularly for MacOS. Assuming you are using the current stable release (v1.0.20180622), you can do the following:

  1. Launch Surfice and choose Advanced/Scripting
  2. From scripting window choose File/NewPython
  3. From scripting window choose Script/Run
  4. If the script runs successfully, you are done. You now run your Python scripts - here are some examples. You can also have other Python scripts launch Surfice to run Python scripts by passing a script in the command line or passing the path to a script. All in-built functions are natively compiled and run at high speed.
  5. If step 4 fails, choose the Surfice/Preferences menu item and then press “Advanced”. A text editor opens the critical line is PyLib. I suggest you try setting this to
    PyLib=/System/Library/Frameworks/Python.framework/Versions/Current/lib/
  6. Repeat steps 1…4 - if this still fails you will want to set PyLib to the location of you libpython*.dylib. You may also want to do this if the “Current” version is not your preferred version (on my computer the current version is 2.7, but I prefer 3.6). In my case I have
    PyLib=/Library/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6.dylib
6

I understand now that these lines

import gl
gl.resetdefaults()
gl.meshload(‘BrainMesh_ICBM152Right.mz3’)
gl.meshsave(’~/ICBM152r.gii’)
gl.meshsave(’~/ICBM152r.mz3’)

can only be run from the surfice scripting window, or by sending a command to surfice using subprocess.call. Correct? That was not clear to me at first.

1 Like
7

Correct, you can run the scripts directly from the user interface, or as an argument to the executable. The argument can either be the entire script or the path to the script file. These are all built-in functions that provide the performance of native code running in a valid OpenGL context.

The mz3 web page provides Python code for that simple format (the blender add-on).