Description: Unit tests are ubiquitous in software development (Google alone has 4.7 million running all the time!), but surprisingly sparse in science. The SciUnit (http://sciun.it) framework was developed to help researchers create unit tests for scientific models. SciUnit tests ask how well a scientific model does what it claims to do, by testing its predictions against specific experimental results. Performance on these tests is thus one measure of whether the model is a good (or at least useful) description of reality. SciUnit is being used in large projects in neuroscience, including in the Human Brain Project and OpenWorm, as well as in individual labs.
Aims: These unit tests can be used not only to assess developed models, but also to guide model development, including the tuning of model parameters to reproduce experimental data. Several challenges arise in multi-objective optimization of models against data, including parallelization, visualization, and reproducibility. We aim to solve some of these challenges in the optimization of models of neurons and neural circuits. In particular, we want to refine and deploy reproducible optimization workflows for models against diverse sets of neurophysiology data, and then share and compare these optimized models for subsequent use in larger research questions.
Hello,
I am Aditi Medhane, an Undergrad at the College of Engineering, Pune pursuing B.Tech in Computer Engineering. I am interested in this project and would like to contribute to it. @rgerkin I am looking for a good starting point so that I can improve my understanding of this project. I would like to explore more about Neuroscience as Iām new to this domain, so can I get some reading recommendations from your side?
For the same, currently, Iām reading the research paper āCollaborative Infrastructure for Test-Driven Scientific Model Validationā and following a Documentation tutorial to understand the usage of SciUnit.
@AditiM
You should take a look at the GitHub repository (http://github.com/scidash/sciunit) for SciUnit, which is in pretty good shape. The current projects that rely upon it are in various levels of development. NeuronUnit (from my lab, http://github.com/scidash/neuronunit) has many examples of usage in neuroscience, but also needs a lot of additional work. There are other examples of neuroscience projects using SciUnit, including FoooFUnit, ASSRUnit, HippoUnit, MorphoUnit, NetworkUnit, and a dozen or so others. Unfortunately the GitHub ādependentsā view is broken so I canāt see any of them right now.
As for understanding the relevant neuroscience, I would recommend reading about computational neuroscience (of the flavor that I do) in Encyclopedia:Computational neuroscience - Scholarpedia for example, where there are many short Wikipedia-style articles. The first few entries under āExperimental Neuroscienceā and all of the entries under āTheoretical Neuroscienceā are relevant. The kinds of models described there are the kinds that we aim to build tests for.
There are many technologies for building these models, but one I would like to advance during this period is http://netpyne.org/, and in particular I would like to be able to easily build and run unit tests aimed at models constructed using NetPyne. This will eventually be part of a larger integration with the new NetPyne GUI (linked in the URL above).
Hello @rgerkin, I am Ankita Priya. I am a Computer Science Engineering undergrad at B.I.T Mesra, India. Currently, I am involved in Python programming which involves the use of Jupyter Notebook and I also push my works on GitHub so I am pretty familiar with Git as well.
@rgerkin I am very new to this domain of Neuroscience but Iād love to explore more and contribute to this project. Please guide me for the same. I hope I am not late for making a project proposal for GSoC 2021.
Good day everyone!
I plan to send in the GSOC application to the SciUnit project, and Iād like to briefly introduce myself here.
My name is Evgenia Karunus, and Iām a last-year Creative Computing student at Goldsmiths (UK).
Iām an active contributor to open-source, and Iāve worked as a developer for ~7 years now, spending a few years with Python in the process.
As my masters are nearing, Iām getting my feet wet in the world of science, and the analogy between the process of the industry-standard PR reviews and scientific peer reviews put forward in the SciUnit paper [1] struck a chord with me, along with the corresponding lack of unit testing & lack of coordination concerns (not mentioning the concerns of the ābut whereās the code thoughā nature [2]).
I agree with the paper much, I enjoy the proposed solution from a brief overview, and I think I might be handy in this project (I contributed to one of the first Node testing frameworks, and to one of the e2e frameworks, but Iāll expand on that in the application).
Contributing with the theoretical neuroscience models sounds all the more amazing, Iām long interested in the physiology of the brain behind the simplified model youād see in machine learning. I havenāt studied neuroscience officially, but I got a brief introduction in the AI course & read a few hopefully relevant books on my own (mostly about sensation & perception, but they touched in some detail on the neural interactions behind it).
(1)
āCollaborative Infrastructure for Test-Driven Scientific Model Validationā by Cyrus Omar, Jonathan Aldrich, and Richard C. Gerkin
@rgerkin/@malin, as far as I see thereās no application template in INCF, and last year tutors were sending out application templates via emails.
Should our proposals be free-form, or do you have a template in mind?
(In either case, my email is lakesare@gmail.com).
A successful proposal can take many forms. I reviewed grants for the US National Science Foundation this week and they all following a slightly different form. Here are some of the common elements:
Introduction: Briefly explain the background to the project you want to work on, including anything that I would need to learn/read about in order to understand your plans.
Specific Aims: Say what it is you would like to achieve. Include deliverables. Each Aim (1-4 of them) should be about a paragraph.
Overall approach: Explain in more detail what you will do and how you will do it. But do not go into extreme details about technical methods. Describe what success would look like. Who might benefit from it?
Methods: How will you do it, with technical details? A few paragraphs. Code samples could go here.
This probably sounds very difficult, given that you (at least right now) probably donāt know too much about either SciUnit or neuroscience. Thatās OK! I just want to see what you are thinking about, what you foresee the project being like, and how you organize your thoughts.
What do you think the work on this project will largely consist of - integration of netpyne.org models with NeuronUnit? Creation of computational neuroscience models in netpyne.org, and submission of these models to NeuronUnit? Work on SciUnit?
@Evgenia I am open to many things and you may think of a great idea or something you are excited about that I havenāt thought of. Many research projects end up very different than they started, and ultimately they are fueled by the excitement of the person doing most of the work (you) and I want to support and help shape things that excite people that I also think are cool.
Donāt get too attached to NeuronUnit: it is just one of many possible ways to use SciUnit to test neuroscience models and I think it is a bit too bloated right now and full of technical debt to be the ideal starting point. Iād rather refactor/cannabalize the important pieces of it into good solid code that does a limited number of things extremely well.
One (possible) thing Iād like to see down extremely well is to develop SciUnit tests around the kinds of models that people can build in NetPyne (and especially are building in NetPyne even if this is only a subset of functionality), ensure that those tests can cover a number of flavors of such models, including different approaches to constructing them (within NetPyne), and then help people discover and run those tests on their models. This could mean a companion package (e.g. NetPyneUnit) of tests that people can be confident run on their NetPyne models, or it could mean thinking about the NetPyne UI frontend and giving people options for designing/scheduling/viewing tests and test results in the NetPyne UI web application.
I really want to develop something that the NetPyne community will use, that will help them evaluate their models, and that may in turn drawn people into that community as they see how cool test-driven development can be.
The thing I am currently most uncertain about is how āgenerallyā the problem can be solved. One can build a lot of models in NetPyne and it is unclear how many models can be scoped into a small number of tests. But I think there will be good opportunities to identify the common patterns that people use and develop tests around those.
Hi. This is Zhanyuan Ye. am a senior student double major in mathematics and statistics and a minor in computer science. I am proficient with python, jupyter, and various python libraries. I have great interest in this project and discover more about biology through coding I want to ask how many seats would be available to this project and if there is seat left for this project?
Thank you.
@Davidyzy I would likely ask for 2 spots but it doesnāt mean I will get 2 as there are a fixed number of total spots being shared across many projects and investigators. But the spot(s) are open until the application deadline ā no one will be chosen until all of the applications have been received and Iāve read them all.
@AditiM@davidyzy@Evgenia@AnkitaxPriya@bickrombishsass Tomorrow (Tuesday) is the deadline for proposals to be finalized. Iām not sure if it is 11:59pm or some other time, nor do I know the time zone, but if you are still interested you should make submit your proposals very soon.