Running an older version of C-PAC (V1.3.0)

Summary of what happened:

Hello All, I am trying to replicate some analysis done on the HBN data with C-PAC, I have the pipeline configuration file for that analysis, which states that it was running on C-PAC V1.3.0. I am currently using the latest container with C-PAC V1.8.6 using singularity. Unfortunately, the pipeline exits and throws an error. I have also tried running my container with the default pipeline of C-PAC V1.3.0, which also exited with a different error. I can provide you with the specific configuration files and errors if necessary.
but more importantly, Is there a way to perhaps pull an old container that has C-PAC version 1.3.0 so that I can use it for replication?
Thanks in Advance!

Command used:

singularity run -B /home/ammar/cpac_inputs:/bids_dataset -B /home/ammar/cpac_outputs:/outputs -B /tmp:/tmp c-pac-latest.simg /bids_dataset /outputs participant --pipeline-file /home/ammar/cpac_inputs/pipeline_config.yml

and similarly with the other config file:

singularity run -B /home/ammar/cpac_inputs:/bids_dataset -B /home/ammar/cpac_outputs:/outputs -B /tmp:/tmp c-pac-latest.simg /bids_dataset /outputs participant --pipeline-file /home/ammar/cpac_inputs/pipeline_config_cpac_default_1.3.0.yml

Version:

C-PAC Version 1.8.6
Singularity Version 2.6.1-dist

Environment : Singularity


C-PAC Version 1.3.0 is actually the oldest version for which we have a published container. You can pull a Singularity version like

singularity pull docker://fcpindi/c-pac:release-v1.3.0

C-PAC’s pipeline configuration file schema changes (usually in small ways) with each release, including a complete overhaul in version 1.8.0, so if you’re looking to replicate a prior analysis, using the same version is probably the way to go.


If you do want to upgrade a configuration to a recent version of C-PAC, there is a page in the documentation that articulates the mappings from a C-PAC 1.7.2 pipeline configuration to a C-PAC 1.8.0 configuration.

In the newer versions, you don’t need to configure options you aren’t using. You can start a pipeline configuration with

FROM: blank

to set every option to default to be inactive, then fill out just what you want to be configured.

Unless you’re adding options that didn’t exist in 1.3.0 to your configuration, most of the keys should be the same (or nearly the same) in 1.7.2 as in 1.3.0, so the update-mappings should mostly be applicable (the changes from version to version are typically additive).

Thanks a lot for your help!

I successfully pulled the old image (C-PAC 1.3.0), but then using it to run the replication analysis, or using it with the default pipeline, produce the same error.

commands used:

run using the default config file

singularity run -B /home/ammar/cpac_inputs:/bids_dataset -B /home/ammar/cpac_outputs:/outputs -B /tmp:/tmp c-pac-release-v1.3.0.simg /bids_dataset /outputs participant

run using config file for replication

singularity run -B /home/ammar/cpac_inputs:/bids_dataset -B /home/ammar/cpac_outputs:/outputs -B /tmp:/tmp c-pac-release-v1.3.0.simg /bids_dataset /outputs participant --pipeline_file /home/ammar/cpac_inputs/replication_pipeline.yml

full error:

Running BIDS validator
/usr/lib/node_modules/bids-validator/node_modules/yargs/lib/command.js:22
    globalMiddleware.push(...middlewares)
                          ^^^

SyntaxError: Unexpected token ...
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:373:25)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/usr/lib/node_modules/bids-validator/node_modules/yargs/lib/argsert.js:2:17)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)

#### Running C-PAC
Number of participants to run in parallel: 1
Input directory: /bids_dataset
Output directory: /outputs/output
Working directory: /scratch/working
Crash directory: /outputs/crash
Log directory: /outputs/log
Remove working directory: True
Available memory: 6.0 (GB)
Available threads: 1
Number of threads for ANTs: 1
/usr/local/miniconda/lib/python2.7/site-packages/matplotlib/__init__.py:1405: UserWarning: 
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

  warnings.warn(_use_error_msg)
Traceback (most recent call last):
  File "/code/run.py", line 374, in <module>
    from nipype.pipeline.plugins.callback_log import log_nodes_cb
ImportError: No module named callback_log

Thanks again, your help is much appreciated.

That error has to do with out-of-sync versions of Node.js and bids-validator in the C-PAC v1.3.0 container.

Thanks for bringing this issue to our attention. In looking into the issue, I discovered that we inadvertently removed the bids-validator CLI, so trying to run with the BIDS validator in versions 1.8.6 and 1.8.7 will fail as well.

I think the best way forward for this use case is to add the optional argument --skip_bids_validator to your run command, optionally using the BIDS validator outside of C-PAC first.

1 Like

Thanks again for your help. I added that argument, it seems that another part of the error still persists.

Namespace(analysis_level='participant', aws_input_creds=None, aws_output_creds=None, bids_dir='/bids_dataset', bids_validator_config=None, data_config_file=None, disable_file_logging=False, mem_gb=None, mem_mb=None, n_cpus='1', output_dir='/outputs', participant_label=None, participant_ndx=None, pipeline_file='/cpac_resources/default_pipeline.yaml', save_working_dir=False, skip_bids_validator=True)
skipping bids-validator...
#### Running C-PAC
Number of participants to run in parallel: 1
Input directory: /bids_dataset
Output directory: /outputs/output
Working directory: /scratch/working
Crash directory: /outputs/crash
Log directory: /outputs/log
Remove working directory: True
Available memory: 6.0 (GB)
Available threads: 1
Number of threads for ANTs: 1
/usr/local/miniconda/lib/python2.7/site-packages/matplotlib/__init__.py:1405: UserWarning: 
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

  warnings.warn(_use_error_msg)
Traceback (most recent call last):
  File "/code/run.py", line 374, in <module>
    from nipype.pipeline.plugins.callback_log import log_nodes_cb
ImportError: No module named callback_log

It looks like there was a bug in the 1.3.0 release that was fixed in the then-latest release but a persistent tag wasn’t added until 1.4.0.

I rebuilt 1.3.0 with the fix for FCP-INDI/C-PAC#844 and resynced the Node.js/bids-validator versions while was at it.

Can you give fcpindi/c-pac:release-v1.3.0.post2 a try and see if it works for you?