Each SCANS folder has multiple subject folders like 601_106_FMRI-NX1-STT2-PA. There are about 30 subfolders like this within SCANS. Which is why I do SCANS/*/DICOM
The dicom files are within the DICOM folder
There is no folder outside of home directory. I am just running this as a test right now but in the future not sure if this will be an issue.
The pictures aren’t entirely clear to me, looks like something’s might be cut off. In the future, it’s better to paste the text and format it as code with tick marks so it shows up nice
like this.
Dcm2bids might work better if you reorganize into something like scans/subject/acqusition/dicoms, and then you could pass in scans/subject/acquisition as the dicom folder. I do not know what inside of your dicom folders look like, so it is hard for me to help much further and say if your dicom organization looks valid.
The * after SCANS is for each of the subfolders within the SCANS directory and then there is a DICOM folder within each of those directories after which the DICOM directory contains dicom files
Hi. The scanner type is Phillips so I think it only outputs one file for some of them. It does convert to nii.gz when I run through MRIcroGL. I updated my bash script. I’m basically trying to get it to read through each row of the CSV file and where the site is TERBO_Bronx run dcm2bids on that through the config.json file specifications. But I seem to be running into an error with my wildcards. I don’t know why its not reading the dicom directory folders corrects. Here is an example of a log file:
2024-06-26 08:33:55.428 - INFO - dcm2bids.main | — dcm2bids start —
2024-06-26 08:33:55.428 - INFO - dcm2bids.main | Running the following command: /usr/local/bin/dcm2bids -d /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0690827A_20240426-20240426 /SCANS//DICOM/ -p YT0690827A -o /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS -c /fs/ess/PAS2302/Subbi/TURBO/TERBO/Scripts/config.json --clobber
2024-06-26 08:33:55.431 - INFO - dcm2bids.main | OS version: Linux-3.10.0-1160.118.1.el7.x86_64-x86_64-with-glibc2.35
2024-06-26 08:33:55.431 - INFO - dcm2bids.main | Python version: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
2024-06-26 08:33:55.431 - INFO - dcm2bids.main | dcm2bids version: 3.1.0
2024-06-26 08:33:55.444 - INFO - dcm2bids.main | dcm2niix version: v1.0.20230411
2024-06-26 08:33:55.444 - INFO - dcm2bids.main | Checking for software update
2024-06-26 08:33:55.597 - WARNING - tools.check_latest | A newer version exists for dcm2bids: 3.1.1
2024-06-26 08:33:55.597 - WARNING - tools.check_latest | You should update it → GitHub - UNFmontreal/Dcm2Bids: Reorganising NIfTI files from dcm2niix into the Brain Imaging Data Structure.
2024-06-26 08:33:55.662 - WARNING - tools.check_latest | A newer version exists for dcm2niix: v1.0.20240202
2024-06-26 08:33:55.662 - WARNING - tools.check_latest | You should update it → GitHub - rordenlab/dcm2niix: dcm2nii DICOM to NIfTI converter: compiled versions available from NITRC.
2024-06-26 08:33:55.662 - INFO - dcm2bids.main | participant: sub-YT0690827A
2024-06-26 08:33:55.663 - INFO - dcm2bids.main | config: /fs/ess/PAS2302/Subbi/TURBO/TERBO/Scripts/config.json
2024-06-26 08:33:55.663 - INFO - dcm2bids.main | BIDS directory: /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS
2024-06-26 08:33:55.663 - INFO - dcm2bids.main | Auto extract entities: False
2024-06-26 08:33:55.663 - INFO - dcm2bids.main | Validate BIDS: False
It should not be what the bold is like above. This is the path to the dicom directory. dicom_path=“/fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/${folder}/SCANS/*/DICOM”
It needs to read each of the subfolders after SCANS. Why is the wildcard not working?
Read the CSV file and process each row where site is TERBO_Bronx
while IFS=, read -r subject session site folder; do
# Skip the header row
if [ “$subject” == “xnat:mrsessiondata/label” ]; then
continue
fi
# Ensure subject does not contain dashes or underscores
if [[ "$subject" == *"-"* || "$subject" == *"_"* ]]; then
echo "Subject $subject contains dashes or underscores, skipping."
continue
fi
# Only consider rows where the site is TERBO_Bronx
if [ "$site" == "TERBO_Bronx" ]; then
# Ensure the BIDS directory and all subfolders are readable and writable
chmod -R u+rw,g+rw "$bids_dir"
chgrp -R PAS2302 "$bids_dir"
# Construct the DICOM path with a specific session and subject
dicom_path="/fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/${folder}/SCANS/*/DICOM"
# Run dcm2bids for the matching folder
singularity run --cleanenv \
/fs/ess/PAS2302/Subbi/TURBO/TERBO/Scripts/dcm2bids_latest.sif \
dcm2bids \
-d $dicom_path/* \
-p $subject \
-s $session \
-o $bids_dir \
-c /fs/ess/PAS2302/Subbi/TURBO/TERBO/Scripts/cofig.json \
--clobber
echo "Ran dcm2bids for folder: $folder, subject: $subject, session: $session"
fi
Hi @Steven I think I may be a bit confused. Does dcm2bids only work on Nifti files and not on dicoms? I’m trying to do dicom to BIDS conversion and heudiconv does not work for this dataset?
How does the config.file work after nii.gz files are generated if it only works for Nifti files?
It works with dicom, yours are advanced dicoms. Nothing to worry.
I think you are the perfect candidate to test our new docker version.
Give me a couple of minutes and I will share with you this new version so you can test it.
Hi @abore Great! Thanks. If you could pass along the pull command for a sif file. I’m a bit new to Linux/Bash that would be great. My config.json file is at the version top of this convo.
Current error I’m getting is:
subprocess.CalledProcessError: Command ‘[‘dcm2niix’, ‘-b’, ‘y’, ‘-ba’, ‘y’, ‘-z’, ‘y’, ‘-f’, ‘%3s_%f_%p_%t’, ‘-o’, PosixPath(’/fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS/tmp_dcm2bids/sub-YT0691035_ses-1’), PosixPath(‘/fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS/tmp_dcm2bids/sub-YT0691035_ses-1_tmp’)]’ returned non-zero exit status 5.
FileNotFoundError: [Errno 2] No such file or directory: ‘/fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS/tmp_dcm2bids/sub-YT0690459C_ses-1_tmp’
Ran dcm2bids for folder: YT-0690459C_20240508-20240508, subject: YT0690459C, session: 1
My apologies. Feel free to delete the other thread if its taking up space on the community. This is the log file for a single subject.
Subject ID: YT0691035i
Session ID: 1
2024-06-26 14:45:17.434 - INFO - dcm2bids.main | — dcm2bids start —
2024-06-26 14:45:17.434 - INFO - dcm2bids.main | Running the following command: /venv/bin/dcm2bids -d /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1001_104C_B0_map-AP/DICOM/IM_0021 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/101_101_SURVEY_MST/DICOM/IM_0002 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1101_104D_B0_map-PA/DICOM/IM_0025 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1401_108_FMRI-NX1-MID2-PA/DICOM/IM_0030 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1501_104E_B0_map-AP/DICOM/IM_0031 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1601_104F_B0_map-PA/DICOM/IM_0035 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1901_110_FMRI-NX1-NB2-PA/DICOM/IM_0041 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2001_101_SURVEY_MST/DICOM/IM_0044 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/201_102_3min_T1W_3D_TFE/DICOM/IM_0004 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/203_SAGITTAL/DICOM/IM_0005 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/204_CORONAL/DICOM/IM_0006 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2101_104G_B0_map-AP/DICOM/IM_0045 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2201_104H_B0_map-PA/DICOM/IM_0049 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2301_113_DTI_MB4_102d_PA/DICOM/IM_0054 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2303_DIFF_SAG/DICOM/IM_0056 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2304_DIFF_COR/DICOM/IM_0057 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2401_114A_B0_map-AP/DICOM/IM_0058 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2501_114B_B0_map-PA/DICOM/IM_0062 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2801_115_NX1-2/DICOM/IM_0067 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/2901_114C_B0_map-AP/DICOM/IM_0068 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/3001_114D_B0_map-PA/DICOM/IM_0072 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/301_104_B0_map-AP/DICOM/IM_0007 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/3101_103_3D__T2_SAG_RAW/DICOM/IM_0078 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/3103_MPR_COR_0.8_-0.4mm/DICOM/IM_0079 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/3104_MPR_AX_0.8_-0.4mm/DICOM/IM_0080 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/401_104B_B0_map-PA/DICOM/IM_0011 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/601_101_SURVEY_MST/DICOM/IM_0017 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/901_106_FMRI-NX1-STT2-PA/DICOM/IM_0020 -p YT0691035i -s 1 -o /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS -c /fs/ess/PAS2302/Subbi/TURBO/TERBO/Scripts/cofig.json --clobber
2024-06-26 14:45:17.458 - INFO - dcm2bids.main | OS version: Linux-3.10.0-1160.118.1.el7.x86_64-x86_64-with-glibc2.36
2024-06-26 14:45:17.458 - INFO - dcm2bids.main | Python version: 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:38:13) [GCC 12.3.0]
2024-06-26 14:45:17.458 - INFO - dcm2bids.main | dcm2bids version: 3.1.1
2024-06-26 14:45:17.623 - INFO - dcm2bids.main | dcm2niix version: v1.0.20240202
2024-06-26 14:45:17.623 - INFO - dcm2bids.main | Checking for software update
2024-06-26 14:45:17.813 - INFO - tools.check_latest | Currently using the latest version of dcm2bids.
2024-06-26 14:45:17.951 - INFO - tools.check_latest | Currently using the latest version of dcm2niix.
2024-06-26 14:45:17.951 - INFO - dcm2bids.main | participant: sub-YT0691035i
2024-06-26 14:45:17.952 - INFO - dcm2bids.main | session: ses-1
2024-06-26 14:45:17.952 - INFO - dcm2bids.main | config: /fs/ess/PAS2302/Subbi/TURBO/TERBO/Scripts/cofig.json
2024-06-26 14:45:17.953 - INFO - dcm2bids.main | BIDS directory: /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS
2024-06-26 14:45:17.953 - INFO - dcm2bids.main | Auto extract entities: False
2024-06-26 14:45:17.953 - INFO - dcm2bids.main | Validate BIDS: False
2024-06-26 14:45:18.19 - ERROR - dcm2niix_gen.execute |
/fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/YT-0691035i_20240430-20240430/SCANS/1001_104C_B0_map-AP/DICOM/IM_0021 is not a supported file extension.tar, tar.bz2, tar.gz or zip are supported.
2024-06-26 14:45:18.20 - INFO - utils.run_shell_command | Running: dcm2niix -b y -ba y -z y -f %3s_%f_%p_%t -o /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS/tmp_dcm2bids/sub-YT0691035i_ses-1 /fs/ess/PAS2302/Subbi/TURBO/TERBO/data/YT/DICOM/BIDS/tmp_dcm2bids/sub-YT0691035i_ses-1_tmp
@abore When I run the above in the CLI directly it gives me this warning:
WARNING: Environment variable LD_PRELOAD already has value , will not forward new value [/apps/xalt/xalt/lib64/libxalt_init.so] from parent process environment
FATAL: “-d”: executable file not found in $PATH