Error when trying to datalad create a new subdataset

Hi DataLad team,

Contrary to my previous version of DataLad, with the latest version (datalad 0.15.3+363.g6b4d5567e) installed via

pip install git+git://github.com/datalad/datalad.git@master,

I now unpredictably get the following error when simply trying to create a new subdataset in my superdataset via

datalad create --description "code subdataset for WP 4 of the DiSCOVERIE H2020 project" -c text2git -d . code, which worked fine before?

[ERROR ] ImportError(cannot import name ‘AnnotatePaths’ from ‘datalad.interface.annotate_paths’ (/opt/anaconda3/lib/python3.8/site-packages/datalad/interface/annotate_paths.py)) (ImportError)

Any help solving this annoying problem would be appreciated!

Thanks a lot in advance!

Cheers,

Lukas

Important note: I uninstalled the latest version, and reinstalled pip install datalad, which installs 0.15.3, which solves the problem - may be a bug in latest version?

and the odd thing is that I cannot reproduce in a clean (non-conda) installation and it should not happen since there is no such import I can find in master version – may be some installation/upgrade went “odd”

detailed attemp
lena:/tmp
$> py=3; d=venvs/dev$py; python$py -m venv -- $d && source $d/bin/activate && python3 -m pip install git+git://github.com/datalad/datalad.git@master
Collecting git+git://github.com/datalad/datalad.git@master
  Cloning git://github.com/datalad/datalad.git (to revision master) to /home/yoh/.tmp/pip-req-build-0r11nrpc
  Running command git clone -q git://github.com/datalad/datalad.git /home/yoh/.tmp/pip-req-build-0r11nrpc
  Running command git submodule update --init --recursive -q
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting tqdm
  Using cached tqdm-4.62.3-py2.py3-none-any.whl (76 kB)
Collecting simplejson
  Using cached simplejson-3.17.6-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (136 kB)
Collecting annexremote
  Using cached annexremote-1.5.0-py3-none-any.whl (25 kB)
Collecting python-gitlab
  Using cached python_gitlab-2.10.1-py3-none-any.whl (197 kB)
Collecting boto
  Using cached boto-2.49.0-py2.py3-none-any.whl (1.4 MB)
Collecting humanize
  Using cached humanize-3.12.0-py3-none-any.whl (90 kB)
Collecting whoosh
  Using cached Whoosh-2.7.4-py2.py3-none-any.whl (468 kB)
Collecting distro
  Using cached distro-1.6.0-py2.py3-none-any.whl (19 kB)
Collecting patool>=1.7
  Using cached patool-1.12-py2.py3-none-any.whl (77 kB)
Collecting platformdirs
  Using cached platformdirs-2.4.0-py3-none-any.whl (14 kB)
Collecting requests>=1.2
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting iso8601
  Downloading iso8601-1.0.2-py3-none-any.whl (9.7 kB)
Collecting keyring>=8.0
  Using cached keyring-23.2.1-py3-none-any.whl (33 kB)
Collecting keyrings.alt
  Using cached keyrings.alt-4.1.0-py3-none-any.whl (20 kB)
Collecting chardet>=3.0.4
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting fasteners>=0.14
  Using cached fasteners-0.16.3-py2.py3-none-any.whl (28 kB)
Collecting msgpack
  Downloading msgpack-1.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB)
     |████████████████████████████████| 322 kB 4.2 MB/s 
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting SecretStorage>=3.2
  Using cached SecretStorage-3.3.1-py3-none-any.whl (15 kB)
Collecting jeepney>=0.4.2
  Using cached jeepney-0.7.1-py3-none-any.whl (54 kB)
Collecting importlib-metadata>=3.6
  Using cached importlib_metadata-4.8.2-py3-none-any.whl (17 kB)
Collecting zipp>=0.5
  Using cached zipp-3.6.0-py3-none-any.whl (5.3 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.7-py3-none-any.whl (38 kB)
Collecting cryptography>=2.0
  Downloading cryptography-36.0.0-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
     |████████████████████████████████| 3.6 MB 11.5 MB/s 
Collecting cffi>=1.12
  Using cached cffi-1.15.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (444 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting future
  Using cached future-0.18.2-py3-none-any.whl
Requirement already satisfied: setuptools in ./venvs/dev3/lib/python3.9/site-packages (from humanize->datalad==0.15.3+363.g6b4d5567e) (44.1.1)
Collecting requests-toolbelt>=0.9.1
  Using cached requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
Building wheels for collected packages: datalad
  Building wheel for datalad (PEP 517) ... done
  Created wheel for datalad: filename=datalad-0.15.3+363.g6b4d5567e-py3-none-any.whl size=1272513 sha256=0f2877a04d0f54bbdcb3e5847331068ed236b2fb689a42cefe4f3283f8e667f0
  Stored in directory: /home/yoh/.tmp/pip-ephem-wheel-cache-s26et2e1/wheels/36/ce/4c/e861c137319d2a01ddaeb5c4bd42e243df029bb272ba625f1d
Successfully built datalad
Installing collected packages: pycparser, urllib3, idna, charset-normalizer, cffi, certifi, zipp, requests, jeepney, cryptography, six, SecretStorage, requests-toolbelt, importlib-metadata, future, whoosh, tqdm, simplejson, python-gitlab, platformdirs, patool, msgpack, keyrings.alt, keyring, iso8601, humanize, fasteners, distro, chardet, boto, annexremote, datalad
Successfully installed SecretStorage-3.3.1 annexremote-1.5.0 boto-2.49.0 certifi-2021.10.8 cffi-1.15.0 chardet-4.0.0 charset-normalizer-2.0.7 cryptography-36.0.0 datalad-0.15.3+363.g6b4d5567e distro-1.6.0 fasteners-0.16.3 future-0.18.2 humanize-3.12.0 idna-3.3 importlib-metadata-4.8.2 iso8601-1.0.2 jeepney-0.7.1 keyring-23.2.1 keyrings.alt-4.1.0 msgpack-1.0.3 patool-1.12 platformdirs-2.4.0 pycparser-2.21 python-gitlab-2.10.1 requests-2.26.0 requests-toolbelt-0.9.1 simplejson-3.17.6 six-1.16.0 tqdm-4.62.3 urllib3-1.26.7 whoosh-2.7.4 zipp-3.6.0
python3 -m pip install git+git://github.com/datalad/datalad.git@master  12.49s user 1.09s system 95% cpu 14.181 total
(dev3) 1 25958 [1].....................................:Wed 24 Nov 2021 09:36:10 AM EST:.
lena:/tmp
$> datalad --version
datalad 0.15.0
(dev3) 1 25959 [1].....................................:Wed 24 Nov 2021 09:36:25 AM EST:.
lena:/tmp
$> rehash
(dev3) 1 25960 [1].....................................:Wed 24 Nov 2021 09:36:27 AM EST:.
lena:/tmp
$> datalad --version
datalad 0.15.3+363.g6b4d5567e
(dev3) 1 25961 [1].....................................:Wed 24 Nov 2021 09:36:28 AM EST:.
lena:/tmp
$> datalad create --description "code subdataset for WP 4 of the DiSCOVERIE H2020 project" -c text2git -d .
create(error): . (dataset) [will not create a dataset in a non-empty directory, use `--force` option to ignore]
(dev3) 1 25962 ->1 [1].....................................:Wed 24 Nov 2021 09:36:36 AM EST:.
lena:/tmp
$> mkdir dir                                                                                               
(dev3) 1 25963 [1].....................................:Wed 24 Nov 2021 09:36:42 AM EST:.
lena:/tmp
$> cd dir
(dev3) 1 25964 [1].....................................:Wed 24 Nov 2021 09:36:44 AM EST:.
lena:/tmp/dir
$> datalad create --description "code subdataset for WP 4 of the DiSCOVERIE H2020 project" -c text2git -d .
[INFO   ] Creating a new annex repo at /tmp/dir 
[INFO   ] Running procedure cfg_text2git 
[INFO   ] == Command start (output follows) ===== 
[INFO   ] == Command exit (modification check follows) =====                                                                                                                                                         
create(ok): . (dataset)  

note: I would recommend to not specify “–description”. It is a description for this particular instance/clone/sibling (whatever name you like to choose ;)) of this dataset, e.g. should be (if decide to specify to overload default host:path) like “on my laptop somewhere”.

Thanks a lot for the prompt response and advice Yarik!

Lukas

MANAGED BY INCF