Concatenating Large 4-D Images


I’m using nilearn to concatenate 40 large, high-resolution functional images (3.8 GB each, for 40*3.8 = 152 GB total). I’m having a memory issue and I’m wondering if there is any way arround it. Even though I have ~400 GB memory, I’m running out of space in my program after about 12 sessions.

sessions = 40
for i in range(1, sessions + 1):
    if i < 10:
        next_data = image.load_img("betas_session0" + str(i) + ".nii.gz",mask_image)
        if i == 1:
            all_data = next_data
            all_data = nilearn.image.concat_imgs([all_data,next_data],dtype="float16")
        next_data = image.load_img("betas_session" + str(i) + ".nii.gz",mask_image)
        all_data = nilearn.image.concat_imgs([all_data,next_data],dtype="float16")

Is there a more efficient way to perform this task without wasting memory? I apologize if my question is not pointed enough.



Nilearn uses nibabel images under the hood, so it would probably be worth reading up on Images and Memory.

Nibabel also has the concat_images function that has been optimized to try to keep as little extra memory active as possible.

You can also unzip your files. Uncompressed NIfTIs expose their data as memory-mapped arrays, assuming the data is not scaled.