PyMVPA BIDS-App problem

Software Support
1

Summary of what happened:

Hi, there! I’m trying to run PyMVPA on my bids dataset but something went wrong. Could someone help me solve this problem?

Command used (and if a helper script was used, a link to the helper script or the command generated):

docker run -i --rm \
	-v /media/menglab/work2_4T/hy/face_imagery:/bids_dataset:ro \
	-v /media/menglab/work2_4T/hy/face_imagery/derivatives/PyMVPA:/outputs \
	bids/pymvpa \
	/bids_dataset /outputs participant_prep -t imagery

Version:

PyMVPA BIDS-App version #4

Environment (Docker, Singularity, custom installation):

Docker installed by Python

Data formatted according to a validatable standard? Please provide the output of the validator:

File Path: Participants and phenotype .tsv files must have a 'participant_id' column.

	Type:		Error
	File:		participants.tsv
	Location:		face_imagery/participants.tsv
	Reason:		Participants and phenotype .tsv files must have a 'participant_id' column.
	Line:		1
	Evidence:	Column headers: "participant_id", "age", "sex", "group"

It says my .tsv file is missing the ‘participant_id’ column, however I’ve already included it.

participant_id age sex group
Sub-1 23 M control
Sub-2 22 F control

Relevant log outputs (up to 20 lines):

/usr/lib/python2.7/dist-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
/usr/lib/python2.7/dist-packages/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
  DeprecationWarning)
/usr/lib/python2.7/dist-packages/sklearn/learning_curve.py:22: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the functions are moved. This module will be removed in 0.20
  DeprecationWarning)
/usr/lib/python2.7/dist-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/tmp/tmp5pt_Qe: line 1: bids-validator: command not found
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
mkdir: cannot create directory '/outputs': File exists
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Screenshots / relevant information:

2

Hi @Yan_Huang and welcome to neurostars!

Looks like it is trying to create a file called /outputs when a file or directory like that already exists in the container. Can you make sure the derivatives folder exists but remove or rename derivatives/PyMVPA if it exists and try:

docker run -it --rm \
	-v /media/menglab/work2_4T/hy/face_imagery:/bids \
	bids/pymvpa \
	/bids/ /bids/derivatives/PyMVPA/ participant_prep -t imagery

Best,
Steven

3

I removed the directory and ran your codes accordingly. Here’s what comes out:

docker run -it --rm \
 -v /media/menglab/work2_4T/hy/face_imagery:/bids \
 bids/pymvpa \
 /bids/ /bids/derivatives/PyMVPA/ participant_prep -t imagery

/usr/lib/python2.7/dist-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
/usr/lib/python2.7/dist-packages/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
  DeprecationWarning)
/usr/lib/python2.7/dist-packages/sklearn/learning_curve.py:22: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the functions are moved. This module will be removed in 0.20
  DeprecationWarning)
/usr/lib/python2.7/dist-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/tmp/tmp0MuRoI: line 1: bids-validator: command not found
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
4

Well, at least this seems to fix the /outputs error! Unfortunately, the /tmp/tmp0MuRoI: line 1: bids-validator: command not found is going to be the main limiting factor. This is something that @sajjad should address, as it appears the bids-validator was not included in the Dockerfile. Can you add the --skip_bids_validator flag and run again?

5

Also, I am adding a PR to the repo using a better mkdir command that will not throw errors when the output already exists, in the meantime, please continue to remove/rename the output directory. Use `mkdir -p` instead by smeisler · Pull Request #19 · bids-apps/PyMVPA · GitHub

6

I added the --skip_bids_validator, here’s the results:

docker run -it --rm -v /media/menglab/work2_4T/hy/face_imagery:/bids bids/pymvpa /bids/ /bids/derivatives/PyMVPA/ participant_prep -t imagery --skip_bids_validator

/usr/lib/python2.7/dist-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
/usr/lib/python2.7/dist-packages/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
  DeprecationWarning)
/usr/lib/python2.7/dist-packages/sklearn/learning_curve.py:22: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the functions are moved. This module will be removed in 0.20
  DeprecationWarning)
/usr/lib/python2.7/dist-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
7

Please format your code output using tickmarks, or using the </> button in the text editor.

All of those looks like warnings, did it stop your code?

8

Yes, the code stopped after the last two warnings.

9

Unfortunately, I’ve done all that I can do to help as someone who does not manage this repository. Hopefully @sajjad will be able to help. I will note that this problem appears to not be new as it was reported back in October on the GitHub: [BUG] PyMVPA BIDS-App in Singularity Container · Issue #18 · bids-apps/PyMVPA · GitHub. I also opened a PR to add the BIDS validator to the Dockerfile. Add validator to Dockerfile by smeisler · Pull Request #20 · bids-apps/PyMVPA · GitHub

Best,
Steven

10

Thank you for your help, Steven. I’ll go on looking for solutions.

11

Looks like a workaround is explained here: PyMVPA BIDS-App in Singularity Container - #5 by sajjad

12

Hi @Yan_Huang I saw this late, sorry. Is your problem solved? As Steven mentioned, I’ve explained how to solve this in the other post. If you still have issues running the app, please first send your data’s tree structure here and we’ll move from there.