Datalad create errors due to permissions on network drive

datalad

#1

I’d like to use datalad to manage a large neuroimaging dataset stored on a NAS device, but the shared filesystem doesn’t offer fine-grained permissions which causes an error in datalad (below). Is there any way to address this? Thanks a bunch!

$ datalad create -f dl_test
[INFO   ] Creating a new annex repo at /some/network/path/dl_test 
[ERROR  ] Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)
|   cmdline: /usr/lib/git-annex.linux/git init
|   stderr: 'error: chmod on /mnt/warren_lab/user_docs/dewarren/scratch/dl_test/.git/config.lock failed: Permission denied
| fatal: could not set 'core.filemode' to 'false'' [cmd.py:execute:821] 
[ERROR  ] Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)
|   cmdline: /usr/lib/git-annex.linux/git init
|   stderr: 'error: chmod on /mnt/warren_lab/user_docs/dewarren/scratch/dl_test/.git/config.lock failed: Permission denied
| fatal: could not set 'core.filemode' to 'false'' [cmd.py:execute:821] (GitCommandError)

#2

Not sure if datalad could work on such type of a mount - it seems that even git can’t operate there for a basic “git init”.
Can you “git init” a new repository there without an error?


#3

Good question. Git throws some error messages related to chmod ops, but it does create an empty repo with a populated .git directory:

[ me@somehost ] path/to/share/scratch
$ git init stuff
error: chmod on path/to/share/scratch/stuff/.git/config.lock failed: Permission denied
error: chmod on path/to/share/scratch/stuff/.git/config.lock failed: Permission denied
error: chmod on path/to/share/scratch/stuff/.git/config.lock failed: Permission denied
Initialized empty Git repository in path/to/share/scratch/stuff/.git/
[ me@somehost ] path/to/share/scratch
$ cd stuff
[ me@somehost ] path/to/share/scratch/stuff
$ tree -a . -L 2
.
└── .git
    ├── branches
    ├── config
    ├── description
    ├── HEAD
    ├── hooks
    ├── info
    ├── objects
    └── refs

6 directories, 3 files

#4

what is the exit code of “git init” command? (you could discover it by “echo $?” right after). If non-0, it means that git errored out. I wouldn’t rely then on it operating correctly (even if seemingly full tree is under .git there)


#5

Interesting — despite the chmod errors, git is returning an exit code of 0.

[ dave@dew-linux ] path/to/share/scratch
$ git init stuff; echo $?
error: chmod on path/to/share/scratch/stuff/.git/config.lock failed: Permission denied
error: chmod on path/to/share/scratch/stuff/.git/config.lock failed: Permission denied
error: chmod on path/to/share/scratch/stuff/.git/config.lock failed: Permission denied
Initialized empty Git repository in path/to/share/scratch/stuff/.git/
0

#6

well – in the original case it was 128 :frowning: so “go figure”

my point is that such NAS mount isn’t “fully supported” to say the least. Could you provide more information - for that path - what is the file system, mount options. What is the OS, datalad version, output of datalad plugin wtf could be of help


#7

I am having the same issue as here, with git commands erroring out with exit code 128, while attempting to create-sibling on a remote.

Ubuntu 14.04
Datalad 0.11.0

$ datalad wtf
/usr/local/lib/python2.7/dist-packages/dicom/__init__.py:53: UserWarning: 
This code is using an older version of pydicom, which is no longer 
maintained as of Jan 2017.  You can access the new pydicom features and API 
by installing `pydicom` from PyPI.
See 'Transitioning to pydicom 1.x' section at pydicom.readthedocs.org 
for more information.

  warnings.warn(msg)
/usr/local/lib/python2.7/dist-packages/bids/grabbids/__init__.py:6: FutureWarning: grabbids has been renamed to layout in version 0.6.5, and will be removed in version 0.8
  warnings.warn("grabbids has been renamed to layout in version 0.6.5, and will be removed in version 0.8", FutureWarning)
# WTF
- path: /scratch/box/webdav/box/MRI1
- type: dataset
## configuration <SENSITIVE, report disabled by configuration>
## datalad 
  - full_version: 0.11.0
  - version: 0.11.0
## dataset 
  - metadata: <SENSITIVE, report disabled by configuration>
  - path: /scratch/box/webdav/box/MRI1
  - repo: AnnexRepo
## dependencies 
  - appdirs: 1.4.3
  - boto: 2.49.0
  - cmd:annex: 6.20181011+git124-g94aa0e2f6-1~ndall+1
  - cmd:git: 2.19.1
  - cmd:system-git: 1.9.1
  - cmd:system-ssh: 6.6.1p1
  - exifread: 2.1.2
  - git: 2.1.11
  - gitdb: 2.0.5
  - humanize: 0.5.1
  - iso8601: 0.1.12
  - msgpack: 0.5.6
  - mutagen: 1.41.1
  - requests: 2.20.0
  - six: 1.11.0
  - tqdm: 4.28.1
  - wrapt: 1.10.11
## environment 
  - GIT_PYTHON_GIT_EXECUTABLE: /usr/lib/git-annex.linux/git
  - LANG: en_CA.ISO-8859-1
  - LC_ALL: en_CA.ISO-8859-1
  - PATH: /opt/sge/bin:/opt/sge/bin/lx-amd64:/opt/sge/bin:/opt/sge/bin/lx-amd64:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/bin:/usr/local/bic/bin:/usr/local/mni/bin
## extentions 
  - neuroimaging: 
    - description: Neuroimaging tools
    - entrypoints: 
      - datalad_neuroimaging.bids2scidata.BIDS2Scidata: 
        - class: BIDS2Scidata
        - load_error: None
        - module: datalad_neuroimaging.bids2scidata
        - names: 
          - bids2scidata
    - load_error: None
    - module: datalad_neuroimaging
    - version: 0.1.5
## git-annex 
  - build flags: 
    - Assistant
    - Webapp
    - Pairing
    - S3(multipartupload)(storageclasses)
    - WebDAV
    - Inotify
    - DBus
    - DesktopNotify
    - TorrentParser
    - MagicMime
    - Feeds
    - Testsuite
  - dependency versions: 
    - aws-0.19
    - bloomfilter-2.0.1.0
    - cryptonite-0.25
    - DAV-1.3.2
    - feed-1.0.0.0
    - ghc-8.2.2
    - http-client-0.5.12
    - persistent-sqlite-2.8.1.2
    - torrent-10000.1.1
    - uuid-1.3.13
    - yesod-1.6.0
  - key/value backends: 
    - SHA256E
    - SHA256
    - SHA512E
    - SHA512
    - SHA224E
    - SHA224
    - SHA384E
    - SHA384
    - SHA3_256E
    - SHA3_256
    - SHA3_512E
    - SHA3_512
    - SHA3_224E
    - SHA3_224
    - SHA3_384E
    - SHA3_384
    - SKEIN256E
    - SKEIN256
    - SKEIN512E
    - SKEIN512
    - BLAKE2B256E
    - BLAKE2B256
    - BLAKE2B512E
    - BLAKE2B512
    - BLAKE2B160E
    - BLAKE2B160
    - BLAKE2B224E
    - BLAKE2B224
    - BLAKE2B384E
    - BLAKE2B384
    - BLAKE2S256E
    - BLAKE2S256
    - BLAKE2S160E
    - BLAKE2S160
    - BLAKE2S224E
    - BLAKE2S224
    - BLAKE2SP256E
    - BLAKE2SP256
    - BLAKE2SP224E
    - BLAKE2SP224
    - SHA1E
    - SHA1
    - MD5E
    - MD5
    - WORM
    - URL
  - local repository version: 5
  - operating system: linux x86_64
  - remote types: 
    - git
    - gcrypt
    - p2p
    - S3
    - bup
    - directory
    - rsync
    - web
    - bittorrent
    - webdav
    - adb
    - tahoe
    - glacier
    - ddar
    - hook
    - external
  - supported repository versions: 
    - 3
    - 5
    - 6
  - upgrade supported from repository versions: 
    - 0
    - 1
    - 2
    - 3
    - 4
    - 5
  - version: 6.20181011+git124-g94aa0e2f6-1~ndall+1
## metadata_extractors 
  - annex: 
    - load_error: None
    - module: datalad.metadata.extractors.annex
    - version: None
  - audio: 
    - load_error: None
    - module: datalad.metadata.extractors.audio
    - version: None
  - bids: 
    - load_error: None
    - module: datalad_neuroimaging.extractors.bids
    - version: None
  - datacite: 
    - load_error: None
    - module: datalad.metadata.extractors.datacite
    - version: None
  - datalad_core: 
    - load_error: None
    - module: datalad.metadata.extractors.datalad_core
    - version: None
  - datalad_rfc822: 
    - load_error: None
    - module: datalad.metadata.extractors.datalad_rfc822
    - version: None
  - dicom: 
    - load_error: None
    - module: datalad_neuroimaging.extractors.dicom
    - version: None
  - exif: 
    - load_error: None
    - module: datalad.metadata.extractors.exif
    - version: None
  - frictionless_datapackage: 
    - load_error: None
    - module: datalad.metadata.extractors.frictionless_datapackage
    - version: None
  - image: 
    - load_error: None
    - module: datalad.metadata.extractors.image
    - version: None
  - nidm: 
    - load_error: None
    - module: datalad_neuroimaging.extractors.nidm
    - version: None
  - nifti1: 
    - load_error: None
    - module: datalad_neuroimaging.extractors.nifti1
    - version: None
  - xmp: 
    - load_error: None
    - module: datalad.metadata.extractors.xmp
    - version: None
## system 
  - distribution: Ubuntu/14.04/trusty
  - encoding: 
    - default: ascii
    - filesystem: ISO-8859-1
    - locale.prefered: ISO-8859-1
  - max_path_length: None
  - name: Linux
  - release: 3.13.0-153-generic
  - type: posix
  - version: #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018

#8

I guess details about the remote location is what would be interesting. But again - if even git fails to init there cleanly, I am not sure if we could do anything about it, and we shouldn’t just ignore… so on remote location, where you are trying to create-sibling on

  • what is the version of git?
  • does plain git init work and exits with 0 status code?
    if that remote location also has standalone build of annex installed, could you check what is git version shipped with it, and either it fails the same way?
  • details about that remote file system would be of interest as well

#9

If I’m allowed back into this thread after abandoning it (sorry), here’s my info.

Local machine (Ubuntu 16.04):

$ uname -a
Linux hostname 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.4 LTS
Release:	16.04
Codename:	xenial
$ git --version
git version 2.7.4
$ datalad --version
datalad 0.11.0

Remote machine (FreeNAS 10.3 serving a Windows-type network share):

FreeBSD freenas.local 10.3-STABLE FreeBSD 10.3-STABLE #0 455d13d(9.10-STABLE): Sun Jun 26 22:47:03 PDT 2016     etc.

git init on my local machine performs as originally reported above.


#10

It would be informative to see the output of the mount command for the shared drive on the local system. In order to get a handle on this issue, we need to figure out why git init seemingly fails for datalad (code 128), but not when run manually. Was the directory confirmed empty (no hidden content) in both cases? It would be good to have all tests above include the actual creation of the repo directory. Likewise informative would be to see the output of ls -la on the parent directory.

That being said a local cifs could look all kinds of ways on the local system and (based on server-side) configuration behave in seemingly unexpected way. The more we get to know about both ends, the easier it will be to get a handle on this.


#11

Here are a few more pointers. All done under a cifs mount with the following parameters (minus banking passwords that I redacted, etc…):

//remote*** on /mnt/smb/fritz type cifs (rw,relatime,vers=1.0,cache=strict,username=***,domain=,uid=1000,forceuid,gid=1000,forcegid,addr=***,soft,unix,posixpaths,serverino,mapposix,acl,noperm,rsize=61440,wsize=65536,echo_interval=60,actimeo=1)

And here is what git/datalad have to say:

mih@meiner /mnt/smb/fritz % mkdir democifs
mih@meiner /mnt/smb/fritz % cd democifs
mih@meiner /mnt/smb/fritz/democifs % git init
Initialized empty Git repository in /mnt/smb/fritz/democifs/.git/
mih@meiner /mnt/smb/fritz/democifs (git)-[master] % cd ..
mih@meiner /mnt/smb/fritz % rm -rf democifs
mih@meiner /mnt/smb/fritz % . ~/env/datalad3-dev/bin/activate
(datalad3-dev) mih@meiner /mnt/smb/fritz % datalad create democifs
[INFO   ] Creating a new annex repo at /mnt/smb/fritz/democifs 
[INFO   ]   Detected a crippled filesystem. 
[INFO   ]   Disabling core.symlinks. 
[INFO   ]   Enabling direct mode. 
create(ok): /mnt/smb/fritz/democifs (dataset)
(datalad3-dev) mih@meiner /mnt/smb/fritz % rm -rf democifs
(datalad3-dev) mih@meiner /mnt/smb/fritz % datalad create democifs --annex-version 6
[INFO   ] Creating a new annex repo at /mnt/smb/fritz/democifs 
[INFO   ]   Detected a crippled filesystem. 
[INFO   ]   Disabling core.symlinks. 
[INFO   ]   Entering an adjusted branch where files are unlocked as this filesystem does not support locked files. 
[INFO   ] Switched to branch 'adjusted/master(unlocked)' 
create(ok): /mnt/smb/fritz/democifs (dataset)

#12

It looks like my problem was with permissions in the local folder.
I had created a new unix user specifically for datalad to ssh-key into the remote (during create-sibling) and had incorrectly set the local folder permissions.
Thanks for the help!


#13

Michael, thanks for the details of your mount.cifs call. I’ll give this a try when I’m back at the office.


#14

Trying this with a new, UNIX-flavored CIFS share on our FreeNAS system helped a bit — git seems to be working without issue — but datalad still throws an error (see below). Of note, my system’s mount.cifs command will take most of the parameters @eknahm mentioned, but not unix or relatime. If anyone has suggestions, I’d be happy to try again after Thanksgiving.

[ dewarren@hostname ] /mnt/dataladtest
$ mkdir democifs
[ dewarren@hostname ] /mnt/dataladtest
$ cd democifs
[ dewarren@hostname ] /mnt/dataladtest/democifs
$ git init
Initialized empty Git repository in /mnt/dataladtest/democifs/.git/
[ dewarren@hostname ] /mnt/dataladtest/democifs
$ cd ..
[ dewarren@hostname ] /mnt/dataladtest
$ rm -rf democifs/
[ dewarren@hostname ] /mnt/dataladtest
$ datalad create democifs
[INFO   ] Creating a new annex repo at /mnt/dataladtest/democifs 
[INFO   ] warning: unable to unlink '.git/objects/4b/tmp_obj_BwSAEc': Permission denied 
[INFO   ] warning: unable to unlink '.git/objects/fc/tmp_obj_ET5qUd': Permission denied 
[INFO   ] git-annex: .git/annex/misctmp/gaprobe: removeLink: invalid argument (Invalid argument) 
[INFO   ] git-annex: init: 1 failed 
Failed to run ['git', '-c', 'receive.autogc=0', '-c', 'gc.auto=0', 'annex', 'init'] under '/mnt/dataladtest/democifs'. Exit code=1.
init  
failed