I’m moving to a new lab and have the opportunity to determine a general databasing solution for the lab. I’ve found relatively little to guide me in how to do this online, so I’d thought I’d ask for any leads. The broad remit is:
manage data from 500+ subjects across multiple studies, sessions
store demographic, physical, physiologic measurements, plus store/link/(view?) inputs, QC and outputs of imaging analysis pipelines (MRI and EEG)
generation of forms to be filled out e.g. on tablets/paper by non-academic staff
easy interface with languages such as Python,R to automate calculation derivatives of raw data
ease of use such that PhD students could create new data forms for new studies, and compile new datasets via a query engine
high level of security, flexible access settings, etc
established software, open source etc.
flexible and robust data storage and server setup
At the same time, we’ll be reviewing our imaging data analysis flows, e.g. storing raw data BIDS compliantly, standardising analysis outputs and simplifying the storage of results/pipelines and submission to online repositories.
It is challenging to decide on things like backend database, and whether to focus on general database solutions or the various newer projects focused on science/neuroimaging. The preference would be not to have to spend extensive time with the initial setup or maintenance - i.e. use contractors if feasible - but I’m wary of ending up with complex all-in-one solutions from this. I’d be very interested to hear if people have pointers, recommendations, or similar issues in their lab.
You might want to take a look at DataLad (http://datalad.org). It would give you all you need re general data management features, has cmdline and Python API, and extensible metadata support. There is a paper coming up that describes the system. It does not do anything re form generation, though.
@eudu - regarding the form generation we are working on a project for general purpose form generation, project management, and collecting data. it’s early stages but can be followed here: https://github.com/ReproNim/brainverse
we just finished the form editor, and presently working on the project management piece. our goal is to get that piece finished by end of the month.
we have some specific needs to align things with the NIMH data archive and create a common repository of neuropsych, clinical, and cognitive test forms. the goal of the current version is to help community curation of the information.
however, if you may want to look into redcap or open clinica for user surveys and form based data collection. redcap also has an api that you can use programmatically.
Here’s an overview of what we are doing, built on XNAT and REDCap. I’m not sure it hits all your points but we are finding it pretty handy so far.
Harrigan RL, Yvernault BC, Boyd BD, et al. Vanderbilt University Institute of Imaging Science Center for Computational Imaging XNAT: A multimodal data archive and processing environment. NeuroImage. 2016;124(0 0):1097-1101. doi:10.1016/j.neuroimage.2015.05.021.