I am working on converting a large dataset to BIDS format. My usual process to do this is to set up a configuration .json file, run dcm2bids, and check outputs for a few subjects to make sure all imaging modalities of interest were successfully converted and labelled the correct way.
My issue is I have not yet found a way to re-run dcm2bids without the tool deleting and reproducing all nifti files for a subject stored in the tmp_dcm2bids directory. This is very time consuming and I would love to skip this step and only overwrite BIDS outputs in response to changes in my configuration .json. Here is the typical command syntax I am currently running:
dcm2bids -d /sourcedata/sub-100a/session_01/ -p 100a -s 01 -c /sourcedata/study_config_file.json -o /BIDS_output_dir/ --skip_dcm2niix --force_dcm2bids
Versions: dcm2bids 3.1.1, dcm2niix 1.0.20240202, python 3.12.4
Environment: M3 iMac, macOS Sonoma 14.4, virtual environment (conda) clean aside from dcm2bids + dependencies
Below is example output from a specific subject:
Relevant log outputs (up to 20 lines):
INFO | --- dcm2bids start ---
INFO | Running the following command: /opt/miniconda3/envs/bidsconv/bin/dcm2bids -d sub-145a -p 145a -s 04y -c /Volumes/BIDS/CL_PDP/code/20250226_pdp_config_YYC_dcm2bids-v311.json -o /Volumes/catherine_team-1/PDP_BIDS_Batch_2025/PDP_BIDS_26th_overnight_outputs --skip_dcm2niix --force_dcm2bids
INFO | OS version: macOS-14.4-arm64-arm-64bit
INFO | Python version: 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:07:17) [Clang 14.0.6 ]
INFO | dcm2bids version: 3.1.1
INFO | dcm2niix version: v1.0.20240202
INFO | Checking for software update
WARNING | A newer version exists for dcm2bids: 3.2.0
WARNING | You should update it -> https://github.com/UNFmontreal/Dcm2Bids.
WARNING | A newer version exists for dcm2niix: v1.0.20241211
WARNING | You should update it -> https://github.com/rordenlab/dcm2niix.
INFO | participant: sub-145a
INFO | session: ses-04y
INFO | config: /Volumes/BIDS/CL_PDP/code/20250226_pdp_config_YYC_dcm2bids-v311.json
INFO | BIDS directory: /Volumes/catherine_team-1/PDP_BIDS_Batch_2025/PDP_BIDS_26th_overnight_outputs
INFO | Auto extract entities: False
INFO | Validate BIDS: False
WARNING | Previous dcm2bids temporary directory output found:
WARNING | /Volumes/catherine_team-1/PDP_BIDS_Batch_2025/PDP_BIDS_26th_overnight_outputs/tmp_dcm2bids/sub-145a_ses-04y
WARNING | 'force' argument is set to True
WARNING | Cleaning the previous directory and running dcm2bids
After which the tmp_dcm2bids folder for a participant is wiped and dcm2niix reproduces nifti files.
Removal of the --force_dcm2bids flag appears to end dcm2bids execution with no changes to tmp_dcm2bids or BIDS output directories.
I’ve tried mixing and matching the --skip_dcm2niix, --force_dcm2bids, and --clobber flags, but have only had two results: either BIDS outputs are detected and thus not overwritten, or I force dcm2bids which unavoidably reproduces files in the tmp_dcm2bids directory.
Any insight into how to reproduce BIDS outputs without reproducing tmp_dcm2bids files, or any workarounds to shorten the amount of time eaten up by dcm2niix, would be greatly appreciated. Thanks!