Heudicov: Permission denied '/BIDS/derivatives/mriqc/..'

1

Dear experts,

I had an error that seems odd to me when trying to use heudiconv to convert new data to an existing BIDS folder. The data in the BIDS folder had gone through some processes and therefore the derivatives folder is not empty.

The error message was:
PermissionError: [Errno 13] Permission denied: '/base/BIDS/derivatives/mriqc/sub-001/ses-d1/func/sub-001_ses-d1_task-fam_events.tsv'

The code I was used is the same as I had used before:

singularity run --cleanenv -B /home/hcp4715/Data/RepDopa/Nipy:/base /home/hcp4715/docker_images/heudiconv-0.8.0.simg -d /base/Dicom/sub-{subject}/ses-{session}/*/* -o /base/BIDS/ -f /base/heuristic/heuristic_d1.py -s 006 -ss d2 -c dcm2niix -b --overwrite --minmeta

The output was as below:

INFO: Running heudiconv version 0.8.0 latest 0.8.0
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': '006', 'outdir': '/base/BIDS/', 'session': 'd2'}
INFO: Processing 3219 dicoms
INFO: Analyzing 3219 dicoms
INFO: Generated sequence info for 14 studies with 3210 entries total
INFO: Doing conversion using dcm2niix
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold (770 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:17,287 nipype.utils INFO:
	 Running nipype version 1.4.2 (latest: 1.5.0)
INFO: Running nipype version 1.4.2 (latest: 1.5.0)
200818-20:08:17,298 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niix14lo8exq/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niix14lo8exq/convert".
200818-20:08:17,415 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold -o . -s n -v n /tmp/dcm2niix14lo8exq/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold -o . -s n -v n /tmp/dcm2niix14lo8exq/convert
200818-20:08:18,499 nipype.interface INFO:
	 stdout 2020-08-18T20:08:18.499356:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:18.499356:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:18,499 nipype.interface INFO:
	 stdout 2020-08-18T20:08:18.499356:Found 770 DICOM file(s)
INFO: stdout 2020-08-18T20:08:18.499356:Found 770 DICOM file(s)
200818-20:08:18,499 nipype.interface INFO:
	 stdout 2020-08-18T20:08:18.499356:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:18.499356:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:18,499 nipype.interface INFO:
	 stdout 2020-08-18T20:08:18.499356:Convert 770 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold (84x72x60x770)
INFO: stdout 2020-08-18T20:08:18.499356:Convert 770 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold (84x72x60x770)
200818-20:08:22,429 nipype.interface INFO:
	 stdout 2020-08-18T20:08:22.429885:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold.nii"
INFO: stdout 2020-08-18T20:08:22.429885:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold.nii"
200818-20:08:22,430 nipype.interface INFO:
	 stdout 2020-08-18T20:08:22.429885:Conversion required 4.969318 seconds (1.003081 for core code).
INFO: stdout 2020-08-18T20:08:22.429885:Conversion required 4.969318 seconds (1.003081 for core code).
200818-20:08:22,462 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-fam_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold (770 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:22,667 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niixga3vh73h/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niixga3vh73h/convert".
200818-20:08:22,784 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold -o . -s n -v n /tmp/dcm2niixga3vh73h/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold -o . -s n -v n /tmp/dcm2niixga3vh73h/convert
200818-20:08:23,349 nipype.interface INFO:
	 stdout 2020-08-18T20:08:23.349789:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:23.349789:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:23,350 nipype.interface INFO:
	 stdout 2020-08-18T20:08:23.349789:Found 770 DICOM file(s)
INFO: stdout 2020-08-18T20:08:23.349789:Found 770 DICOM file(s)
200818-20:08:23,350 nipype.interface INFO:
	 stdout 2020-08-18T20:08:23.349789:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:23.349789:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:23,350 nipype.interface INFO:
	 stdout 2020-08-18T20:08:23.349789:Convert 770 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold (84x72x60x770)
INFO: stdout 2020-08-18T20:08:23.349789:Convert 770 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold (84x72x60x770)
200818-20:08:27,605 nipype.interface INFO:
	 stdout 2020-08-18T20:08:27.605434:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold.nii"
INFO: stdout 2020-08-18T20:08:27.605434:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold.nii"
200818-20:08:27,605 nipype.interface INFO:
	 stdout 2020-08-18T20:08:27.605434:Conversion required 4.812507 seconds (0.801286 for core code).
INFO: stdout 2020-08-18T20:08:27.605434:Conversion required 4.812507 seconds (0.801286 for core code).
200818-20:08:27,635 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-exp_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-1_bold (26 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:27,834 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niix4z2k51g0/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niix4z2k51g0/convert".
200818-20:08:27,839 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-1_bold -o . -s n -v n /tmp/dcm2niix4z2k51g0/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-1_bold -o . -s n -v n /tmp/dcm2niix4z2k51g0/convert
200818-20:08:27,888 nipype.interface INFO:
	 stdout 2020-08-18T20:08:27.888314:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:27.888314:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:27,888 nipype.interface INFO:
...
	 stdout 2020-08-18T20:08:28.847438:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-2_bold (84x72x60x480)
INFO: stdout 2020-08-18T20:08:28.847438:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-2_bold (84x72x60x480)
200818-20:08:31,444 nipype.interface INFO:
	 stdout 2020-08-18T20:08:31.444456:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-2_bold.nii"
INFO: stdout 2020-08-18T20:08:31.444456:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-2_bold.nii"
200818-20:08:31,444 nipype.interface INFO:
	 stdout 2020-08-18T20:08:31.444456:Conversion required 3.275455 seconds (0.661094 for core code).
INFO: stdout 2020-08-18T20:08:31.444456:Conversion required 3.275455 seconds (0.661094 for core code).
200818-20:08:31,469 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-2_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold (480 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:31,605 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niixnxiwctz7/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niixnxiwctz7/convert".
200818-20:08:31,680 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold -o . -s n -v n /tmp/dcm2niixnxiwctz7/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold -o . -s n -v n /tmp/dcm2niixnxiwctz7/convert
200818-20:08:32,299 nipype.interface INFO:
	 stdout 2020-08-18T20:08:32.299640:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:32.299640:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:32,299 nipype.interface INFO:
	 stdout 2020-08-18T20:08:32.299640:Found 480 DICOM file(s)
INFO: stdout 2020-08-18T20:08:32.299640:Found 480 DICOM file(s)
200818-20:08:32,299 nipype.interface INFO:
	 stdout 2020-08-18T20:08:32.299640:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:32.299640:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:32,300 nipype.interface INFO:
	 stdout 2020-08-18T20:08:32.299640:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold (84x72x60x480)
INFO: stdout 2020-08-18T20:08:32.299640:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold (84x72x60x480)
200818-20:08:34,991 nipype.interface INFO:
	 stdout 2020-08-18T20:08:34.991801:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold.nii"
INFO: stdout 2020-08-18T20:08:34.991801:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold.nii"
200818-20:08:34,991 nipype.interface INFO:
	 stdout 2020-08-18T20:08:34.991801:Conversion required 3.303619 seconds (0.598410 for core code).
INFO: stdout 2020-08-18T20:08:34.991801:Conversion required 3.303619 seconds (0.598410 for core code).
200818-20:08:35,16 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-3_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold (157 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:35,149 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niixbdti8j2m/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niixbdti8j2m/convert".
200818-20:08:35,174 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold -o . -s n -v n /tmp/dcm2niixbdti8j2m/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold -o . -s n -v n /tmp/dcm2niixbdti8j2m/convert
200818-20:08:35,401 nipype.interface INFO:
	 stdout 2020-08-18T20:08:35.401298:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:35.401298:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:35,401 nipype.interface INFO:
	 stdout 2020-08-18T20:08:35.401298:Found 157 DICOM file(s)
INFO: stdout 2020-08-18T20:08:35.401298:Found 157 DICOM file(s)
200818-20:08:35,401 nipype.interface INFO:
	 stdout 2020-08-18T20:08:35.401298:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:35.401298:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:35,401 nipype.interface INFO:
	 stdout 2020-08-18T20:08:35.401298:Convert 157 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold (84x72x60x157)
INFO: stdout 2020-08-18T20:08:35.401298:Convert 157 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold (84x72x60x157)
200818-20:08:36,348 nipype.interface INFO:
	 stdout 2020-08-18T20:08:36.348210:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold.nii"
INFO: stdout 2020-08-18T20:08:36.348210:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold.nii"
200818-20:08:36,348 nipype.interface INFO:
	 stdout 2020-08-18T20:08:36.348210:Conversion required 1.165266 seconds (0.214030 for core code).
INFO: stdout 2020-08-18T20:08:36.348210:Conversion required 1.165266 seconds (0.214030 for core code).
200818-20:08:36,367 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-4_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold (480 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:36,424 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niixmuilo32f/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niixmuilo32f/convert".
200818-20:08:36,500 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold -o . -s n -v n /tmp/dcm2niixmuilo32f/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold -o . -s n -v n /tmp/dcm2niixmuilo32f/convert
200818-20:08:37,154 nipype.interface INFO:
	 stdout 2020-08-18T20:08:37.154434:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:37.154434:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:37,154 nipype.interface INFO:
	 stdout 2020-08-18T20:08:37.154434:Found 480 DICOM file(s)
INFO: stdout 2020-08-18T20:08:37.154434:Found 480 DICOM file(s)
200818-20:08:37,154 nipype.interface INFO:
	 stdout 2020-08-18T20:08:37.154434:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:37.154434:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:37,154 nipype.interface INFO:
	 stdout 2020-08-18T20:08:37.154434:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold (84x72x60x480)
INFO: stdout 2020-08-18T20:08:37.154434:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold (84x72x60x480)
200818-20:08:39,915 nipype.interface INFO:
	 stdout 2020-08-18T20:08:39.915448:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold.nii"
INFO: stdout 2020-08-18T20:08:39.915448:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold.nii"
200818-20:08:39,915 nipype.interface INFO:
	 stdout 2020-08-18T20:08:39.915448:Conversion required 3.405364 seconds (0.633780 for core code).
INFO: stdout 2020-08-18T20:08:39.915448:Conversion required 3.405364 seconds (0.633780 for core code).
200818-20:08:39,940 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-5_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold (175 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:40,69 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niix68neg62t/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niix68neg62t/convert".
200818-20:08:40,97 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold -o . -s n -v n /tmp/dcm2niix68neg62t/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold -o . -s n -v n /tmp/dcm2niix68neg62t/convert
200818-20:08:40,319 nipype.interface INFO:
	 stdout 2020-08-18T20:08:40.319242:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:40.319242:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:40,319 nipype.interface INFO:
	 stdout 2020-08-18T20:08:40.319242:Found 175 DICOM file(s)
INFO: stdout 2020-08-18T20:08:40.319242:Found 175 DICOM file(s)
200818-20:08:40,319 nipype.interface INFO:
	 stdout 2020-08-18T20:08:40.319242:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:40.319242:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:40,319 nipype.interface INFO:
	 stdout 2020-08-18T20:08:40.319242:Convert 175 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold (84x72x60x175)
INFO: stdout 2020-08-18T20:08:40.319242:Convert 175 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold (84x72x60x175)
200818-20:08:41,302 nipype.interface INFO:
	 stdout 2020-08-18T20:08:41.302096:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold.nii"
INFO: stdout 2020-08-18T20:08:41.302096:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold.nii"
200818-20:08:41,302 nipype.interface INFO:
	 stdout 2020-08-18T20:08:41.302096:Conversion required 1.197064 seconds (0.207047 for core code).
INFO: stdout 2020-08-18T20:08:41.302096:Conversion required 1.197064 seconds (0.207047 for core code).
200818-20:08:41,320 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-6_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold (480 DICOMs) -> /base/BIDS/sub-006/ses-d2/func . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:41,378 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niixmaujg8b5/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niixmaujg8b5/convert".
200818-20:08:41,451 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold -o . -s n -v n /tmp/dcm2niixmaujg8b5/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold -o . -s n -v n /tmp/dcm2niixmaujg8b5/convert
200818-20:08:42,81 nipype.interface INFO:
	 stdout 2020-08-18T20:08:42.081791:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:42.081791:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:42,82 nipype.interface INFO:
	 stdout 2020-08-18T20:08:42.081791:Found 480 DICOM file(s)
INFO: stdout 2020-08-18T20:08:42.081791:Found 480 DICOM file(s)
200818-20:08:42,82 nipype.interface INFO:
	 stdout 2020-08-18T20:08:42.081791:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
INFO: stdout 2020-08-18T20:08:42.081791:slices stacked despite varying acquisition numbers (if this is not desired recompile with 'mySegmentByAcq')
200818-20:08:42,82 nipype.interface INFO:
	 stdout 2020-08-18T20:08:42.081791:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold (84x72x60x480)
INFO: stdout 2020-08-18T20:08:42.081791:Convert 480 DICOM as ./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold (84x72x60x480)
200818-20:08:44,687 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.687231:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold.nii"
INFO: stdout 2020-08-18T20:08:44.687231:Compress: "/usr/bin/pigz" -b 960 -n -f -6 "./base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold.nii"
200818-20:08:44,687 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.687231:Conversion required 3.227776 seconds (0.618335 for core code).
INFO: stdout 2020-08-18T20:08:44.687231:Conversion required 3.227776 seconds (0.618335 for core code).
200818-20:08:44,711 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
INFO: Post-treating /base/BIDS/sub-006/ses-d2/func/sub-006_ses-d2_task-rest_run-7_bold.json file
INFO: Converting /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude (100 DICOMs) -> /base/BIDS/sub-006/ses-d2/fmap . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:44,835 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niixc9ibwb8f/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niixc9ibwb8f/convert".
200818-20:08:44,852 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude -o . -s n -v n /tmp/dcm2niixc9ibwb8f/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude -o . -s n -v n /tmp/dcm2niixc9ibwb8f/convert
200818-20:08:44,888 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.888858:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:44.888858:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:44,889 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.888858:Found 100 DICOM file(s)
INFO: stdout 2020-08-18T20:08:44.888858:Found 100 DICOM file(s)
200818-20:08:44,889 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.888858:slices not stacked: echo varies (TE 4.89, 7.35; echo 1, 2). Use 'merge 2D slices' option to force stacking
INFO: stdout 2020-08-18T20:08:44.888858:slices not stacked: echo varies (TE 4.89, 7.35; echo 1, 2). Use 'merge 2D slices' option to force stacking
200818-20:08:44,889 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.888858:Convert 50 DICOM as ./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e1 (64x64x50x1)
INFO: stdout 2020-08-18T20:08:44.888858:Convert 50 DICOM as ./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e1 (64x64x50x1)
200818-20:08:44,899 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.899773:Compress: "/usr/bin/pigz" -n -f -6 "./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e1.nii"
INFO: stdout 2020-08-18T20:08:44.899773:Compress: "/usr/bin/pigz" -n -f -6 "./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e1.nii"
200818-20:08:44,899 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.899773:Convert 50 DICOM as ./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e2 (64x64x50x1)
INFO: stdout 2020-08-18T20:08:44.899773:Convert 50 DICOM as ./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e2 (64x64x50x1)
200818-20:08:44,908 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.908698:Compress: "/usr/bin/pigz" -n -f -6 "./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e2.nii"
INFO: stdout 2020-08-18T20:08:44.908698:Compress: "/usr/bin/pigz" -n -f -6 "./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude_e2.nii"
200818-20:08:44,908 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.908698:Conversion required 0.048438 seconds (0.030482 for core code).
INFO: stdout 2020-08-18T20:08:44.908698:Conversion required 0.048438 seconds (0.030482 for core code).
200818-20:08:44,924 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
WARNING: Failed to find task field in /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude1.json.
WARNING: Failed to find task field in /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_magnitude2.json.
WARNING: For now not embedding BIDS and info generated .nii.gz itself since sequence produced multiple files
INFO: Converting /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff (50 DICOMs) -> /base/BIDS/sub-006/ses-d2/fmap . Converter: dcm2niix . Output types: ('nii.gz',)
200818-20:08:44,931 nipype.workflow INFO:
	 [Node] Setting-up "convert" in "/tmp/dcm2niix7lz8go6q/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niix7lz8go6q/convert".
200818-20:08:44,939 nipype.workflow INFO:
	 [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff -o . -s n -v n /tmp/dcm2niix7lz8go6q/convert
INFO: [Node] Running "convert" ("nipype.interfaces.dcm2nii.Dcm2niix"), a CommandLine Interface with command:
dcm2niix -b y -z y -x n -t n -m n -f /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff -o . -s n -v n /tmp/dcm2niix7lz8go6q/convert
200818-20:08:44,963 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.963252:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
INFO: stdout 2020-08-18T20:08:44.963252:Chris Rorden's dcm2niiX version v1.0.20190410  GCC6.3.0 (64-bit Linux)
200818-20:08:44,963 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.963252:Found 50 DICOM file(s)
INFO: stdout 2020-08-18T20:08:44.963252:Found 50 DICOM file(s)
200818-20:08:44,963 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.963252:Convert 50 DICOM as ./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff_e2_ph (64x64x50x1)
INFO: stdout 2020-08-18T20:08:44.963252:Convert 50 DICOM as ./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff_e2_ph (64x64x50x1)
200818-20:08:44,971 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.971613:Compress: "/usr/bin/pigz" -n -f -6 "./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff_e2_ph.nii"
INFO: stdout 2020-08-18T20:08:44.971613:Compress: "/usr/bin/pigz" -n -f -6 "./base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff_e2_ph.nii"
200818-20:08:44,971 nipype.interface INFO:
	 stdout 2020-08-18T20:08:44.971613:Conversion required 0.024234 seconds (0.016168 for core code).
INFO: stdout 2020-08-18T20:08:44.971613:Conversion required 0.024234 seconds (0.016168 for core code).
200818-20:08:44,986 nipype.workflow INFO:
	 [Node] Finished "convert".
INFO: [Node] Finished "convert".
WARNING: Failed to find task field in /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff.json.
INFO: Post-treating /base/BIDS/sub-006/ses-d2/fmap/sub-006_ses-d2_phasediff.json file
INFO: Lock 139697194964304 acquired on /base/BIDS/heudiconv.lock
INFO: Populating template files under /base/BIDS/
INFO: Lock 139697194964304 released on /base/BIDS/heudiconv.lock
Traceback (most recent call last):
  File "/opt/miniconda-latest/bin/heudiconv", line 11, in <module>
    load_entry_point('heudiconv', 'console_scripts', 'heudiconv')()
  File "/src/heudiconv/heudiconv/cli/run.py", line 134, in main
    process_args(args)
  File "/src/heudiconv/heudiconv/cli/run.py", line 352, in process_args
    grouping=args.grouping,)
  File "/src/heudiconv/heudiconv/convert.py", line 233, in prep_conversion
    getattr(heuristic, 'DEFAULT_FIELDS', {}))
  File "/src/heudiconv/heudiconv/bids.py", line 77, in populate_bids_templates
    populate_aggregated_jsons(path)
  File "/src/heudiconv/heudiconv/bids.py", line 144, in populate_aggregated_jsons
    with open(events_file, 'w') as f:
PermissionError: [Errno 13] Permission denied: '/base/BIDS/derivatives/mriqc/sub-001/ses-d1/func/sub-001_ses-d1_task-fam_events.tsv'

this error was odd to me because the code I was using was not relevant to mriqc, why should it try to access to that folder in the first place?

2

This looks like a bug in HeuDiConv. It could be assuming that everything in the output directory is something it wrote, and so crawling it without considering the case that it might encounter something foreign. I would open an issue on https://github.com/nipy/heudiconv/.

1 Like
3

You are right, I moved all other folders outside the BIDS/derivatives folder except scanner, no error then.

4

I’m having a similar issue and looked through the Github issues to see if this had been resolved, but didn’t see anything that seemed relevant.

Basically, running Heudiconv touches all or the first (not sure which) derivatives sub-directories and makes files labeled: sub-CBPD0048_ses-03_task-rest_run-02_events.tsv for each subject in that directory, despite having run it on only one subject (not sub-CBPD0048). Very confusing, as I believe I used the same Heudiconv container on a different HPC and didn’t have this issue, but perhaps it was because these derivatives directories didn’t exist the previous times I ran Heudiconv (because that is the first preprocessing step).

5

yeap, a bug – hopefully this fix would resolve it ENH+BF: find_files to take list of topdirs, look for _bold only under sub-* directories by yarikoptic · Pull Request #496 · nipy/heudiconv · GitHub . Haven’t seen this message, and no issue AFAIK was filed as the wise @effigies recommended (or have I missed it too? ;))

1 Like
6

Thanks so much for the fix! Will wait for a new release/container (since I’m running through Singularity) and test it out.

7

I’m actually not sure this is at all related to this issue, since i haven’t been able to use a new singularity container with this fix (doesn’t look like any new ones have been released since end of 2020) to see if it addresses it.

However, I realized upon trying to BIDS-validate that I’m also having this issue, where heudiconv is adding an empty CogAtlasID field to all the top-level task-<taskname>_bold.json files, which then is an error in BIDS-validation when I don’t actually have a CogAtlasID associated with the task…seems like unintended behavior when I don’t have a CogAtlasID in any lower-level files…I don’t want to have to delete it from top-level files every time after I use heudiconv to convert a new scan, it’s very tedious…is this intended? Is this part of touching all the files that would be fixed by this fix?

8

well, even though I would also dislike a very “opinionated” software telling me what todo, I am afraid here I would prefer to leave it as is and keep placing TODO CogAtlasID if none is provided or remove at the top level, or when it is not consistently (the same _task- files per subject have the same CogAtlasID) present across all subjects/runs for that task.

My rationale/line defense: CogAtlasID is indeed just a RECOMMENDED field in BIDS, so a task... .json file without it is indeed perfectly “legit”! But I consider it somewhat unfortunately that BIDS is not mandating any way to assign an unambiguous identifier to the task label. Such absence of a machine-readable and unique identifier for the task hinders the utility of BIDS datasets. With my heudiconv hat on I would like to help fix this situation by “strongly” encouraging its users to spend those 1-3 minutes necessary to look up the closest matching task on Cognitive Atlas and if somehow it is not yet there (among 783! currently already present), suggest a new one.

Altogether, instead of keeping removing it, I would strongly recommend adding it, and your dataset and its future users would only be grateful, and I think it would be those 3 minutes “very well spent” :wink:

1 Like
9

Your rationale makes sense to me, despite the absence of BIDS standard forcing the choice! Though my knee-jerk reaction was “why is heudiconv/BIDS changing on me, and making things harder?” I think that was unwarranted.

Since adding the top-level task CogAtlasID one time fixes the BIDS validation error, it’s worth it. And it additionally forced me to check out the Cognitive Atlas, which I had been meaning to do…thanks!

1 Like