GSoC 2021 project idea 4.1: Unit tests for brains

Description: Unit tests are ubiquitous in software development (Google alone has 4.7 million running all the time!), but surprisingly sparse in science. The SciUnit (http://sciun.it) framework was developed to help researchers create unit tests for scientific models. SciUnit tests ask how well a scientific model does what it claims to do, by testing its predictions against specific experimental results. Performance on these tests is thus one measure of whether the model is a good (or at least useful) description of reality. SciUnit is being used in large projects in neuroscience, including in the Human Brain Project and OpenWorm, as well as in individual labs.

Aims: These unit tests can be used not only to assess developed models, but also to guide model development, including the tuning of model parameters to reproduce experimental data. Several challenges arise in multi-objective optimization of models against data, including parallelization, visualization, and reproducibility. We aim to solve some of these challenges in the optimization of models of neurons and neural circuits. In particular, we want to refine and deploy reproducible optimization workflows for models against diverse sets of neurophysiology data, and then share and compare these optimized models for subsequent use in larger research questions.

Mentor: Rick Gerkin @rgerkin (rgerkin@asu.edu), ICON Lab, Arizona State University, USA

Tags: Python, Git, Jupyter, Numpy

1 Like

Hello,
I am Aditi Medhane, an Undergrad at the College of Engineering, Pune pursuing B.Tech in Computer Engineering. I am interested in this project and would like to contribute to it.
@rgerkin I am looking for a good starting point so that I can improve my understanding of this project. I would like to explore more about Neuroscience as I’m new to this domain, so can I get some reading recommendations from your side?

For the same, currently, I’m reading the research paper “Collaborative Infrastructure for Test-Driven Scientific Model Validation” and following a Documentation tutorial to understand the usage of SciUnit.

@AditiM
You should take a look at the GitHub repository (http://github.com/scidash/sciunit) for SciUnit, which is in pretty good shape. The current projects that rely upon it are in various levels of development. NeuronUnit (from my lab, http://github.com/scidash/neuronunit) has many examples of usage in neuroscience, but also needs a lot of additional work. There are other examples of neuroscience projects using SciUnit, including FoooFUnit, ASSRUnit, HippoUnit, MorphoUnit, NetworkUnit, and a dozen or so others. Unfortunately the GitHub “dependents” view is broken so I can’t see any of them right now.

As for understanding the relevant neuroscience, I would recommend reading about computational neuroscience (of the flavor that I do) in Encyclopedia:Computational neuroscience - Scholarpedia for example, where there are many short Wikipedia-style articles. The first few entries under “Experimental Neuroscience” and all of the entries under “Theoretical Neuroscience” are relevant. The kinds of models described there are the kinds that we aim to build tests for.

There are many technologies for building these models, but one I would like to advance during this period is http://netpyne.org/, and in particular I would like to be able to easily build and run unit tests aimed at models constructed using NetPyne. This will eventually be part of a larger integration with the new NetPyne GUI (linked in the URL above).

MANAGED BY INCF