Thanks @malin, @pgleeson. That sounds great. I’ll fill in the info at the INCF portal in the next few days.
I’d like to start some activities while all of this is set up. Would anyone like to do a “dev session”? We could start in early November. That gives us time to schedule sessions and send out the announcements.
For reviews, I’ve setup a GitHub team and a repository. If you can please tell me your GitHub usernames (privately if you prefer), I’ll add everyone to the these too.
What we run into often is the difference between nominal file format support and undocumented interpretations of the format (often exacerbated by a lack of a formal spec). Large databases even may have tooling around this, working around such an undocumented interpretation, which in turn is undocumented, making this all quite tricky.
So we underline this fundamental problem as published here [1] recently in the strongest possible terms. It is difficult, or impossible, to reproduce work, and that may be the largest problem in computational neuroscience at this time. E.g. requiring a formal spec, requiring successful tests before compatibility may be claimed, those are the types of things we think will help fix these reproducibility challenges and general interop, as outlined in the paper, and of course is known throughout the software development world. It is something we’d like to push for, through this community.
Maybe these things are in better shape outside of morphologically detailed simulators; would you (or anyone) say so? (My experience is very limited there.)
I’ve not visited this thread as faithfully as I should, so I’m not certain if or where exactly this part of the discussion has moved. Of course, for certain formats or tools there already exist platforms or venues for discussion, but we see added value in a place for overarching discussion: what parts of the ecosystem still have issues with standardization, what infrastructure could be copied from areas where this happens successfully?
(The private message button isn’t showing up for me.) My Github handle is @brenthuisman
I could talk about Arbor development and how one might contribute, but I’ll not be able to fill an hour with that It’s a CMake/C++ project, and we use Github for just about everything.
Thanks @brent. I’m afraid I’m not an expert on standardisation yet so I can’t answer all your queries here. Standardisation, however, is certainly within the scope of the SIG. So we could work with existing standardisation initiatives on this. Examples would be:
The COMBINE meeting is on now, and there’s a workshop on standardisation on Friday. (I think some folks from the arbor dev team are attending the workshop too)
That sounds great. I think there’ll be quite a bit to talk about—I expect folks will start with a short user focused overview of the tool to set the context of the talk. Even when using GitHub, different development teams implement different pipelines, for pull requests and CI for instance. So the idea is to go into these in some detail so that a non dev audience can learn standard development workflows (and gain confidence about the correctness of tools by getting an idea of the work that goes into developing them), and a dev audience can perhaps compare and contrast the different pipelines to improve their own systems. (If dev teams can talk about contributor paths to help newcomers participate in the development, that’ll be even better!)
Hi - we from the GeNN team can certainly run a session, @kernfel & @jamie are involved. I think there is enough to talk about, workflow, including CI with jenkins (oh the joy, @jamie!) but also issues where we would welcome contributions etc.
Finding some time might be tricky - but let’s cross that bridge when we get there.
Hi and sorry for the very late reply. Indeed, the JOSS is not essentially about the software development side but rather of the “user experience”/reproducibility, i.e. installation instructions, documentation and so on. There’s certainly room for more specific recommendations, in particular programming-language specific ones. E.g. best practices for packaging a tool for Python. My personal opinion is that relative generic guidelines are often more useful than overly specific ones. E.g. I’d find it more important that a code is well-structured than that it has a maximum line length of 79 characters. But I’ll guess the best is to take this kind of discussion to the SoftwareReview repository you created.
I hadn’t come across that yet, and it looks like a great resource. I don’t know if it’s meant to be for individual tools or for the whole pipeline that’s used in research work, but I certainly see bits there that can go into our guidelines.
I’ve opened an issue on GitHub so we don’t forget:
The process that we want to follow needs some thinking but can wait until the first version of the guidelines are ready. We don’t want it to be a review that may feel like a stick to people. Perhaps we use a carrot instead: encourage compliance and reward compliant tools (with a banner or a star rating etc.)?
Hi, I didn’t get around to posting when this started but I’m Ethan and I’m a PhD student at the University of Pennsylvania in Philadelpha. Unlike several of the impressive commenters so far I haven’t built my own neuro software package yet, but in my previous role as a research tech at the University of Minnesota I used Open Ephys for in vivo electrophysiology and was a big contributor to its main code as well as several plugins.
Now in graduate school, I’m doing more comp neuro stuff but I’m feeling a hole in my heart for C++ development/contributing to scientific software in general. I’m excited to see all these great projects that are looking for contributors! Looking forward to the presentations that you’re planning!
Could you please see when you can fit a 1–2 hour session in your calendars in the coming months so we can start scheduling them? We could do 1 or 2 a month perhaps, to space the sessions out. If November is too soon, we could just start in January 2021. So, the slots would be:
January: slot 1
January: slot 2
February: slot 1
February: slot 2
That gives us 10–12 slots before CNS*2021 in the first week of July, where it’ll be good to have a SIG meeting (apart from other meetings that we can have whenever members think they’re needed). I can speak to the Board about fitting this in to the conference programme agenda, of course.
Hi Sanjay, I am Anubhab. I will be beginning my undergrad this year and wish to study programming and data science. I want to pursue computational neuroscience research in the future and have a strong interest for the same. However, I am just a begineer with very basic knowledge in the computational field and neuroscience and psychiatry. But I am always interested in learning more as computational neuroscience intrigues me.
I was wondering whether I am eligible to join this group, does it make sense?
We need some information to set up the INCF group. I’ve filled in what I know, could people please add the missing bits? I think a week should be enough for this, so I’ll submit the info next Wednesday, 11th November.
Your official decided title (“INCF SIG on …”): INCF/OCNS Software SIG (or SIG on Software)
Named chair(s) and deputy chair(s) (at least one of each, or two chairs is also fine): I (@sanjayankur31) can be one as the current liaison with the OCNS Board. I suggest @Shailesh_Appukuttan be the second, as they’re going to take over as OCNS Webmaster when my term finishes at the end of 2021. Thoughts?
A short and clear statement of the groups’ aim (~150-400 chars): How does this sound? "The Software SIG will focus on aiding the neuroscience research community in the development and maintenance of software in the field.
Contact details for at least one and preferably several ways of contacting the group, such as email, mailing list, forum, webpage, github…): This needs some thought—Neurostars is good for our discussion but doesn’t serve well as a way to contact the SIG. Most SIGs set up a Google group/mailing list. What do people think of setting that up?
A slightly longer (~500-1000) section “About this SIG”: From the initial statement: "Computational neuroscience cannot exist without the tools that we all rely on. So, this SIG will focus on these. We’ll find and discuss tools, we will learn how to use them, we will test and review them, we will file bugs to inform the developers of issues, and finally we will try to learn how they work and try to get involved in their development—to ensure that these tools that we rely on remain in good shape by having communities looking after them. Since many members of the SIG are themselves tool developers, we will also learn from each other and will work towards improving interoperability between related tools.
While we do all of this, we will improve our knowledge of transferable skills: anything involved in modern software development (which are not limited to only writing tools, but are extremely important/useful when developing models using these tools)."
Members list: everyone that has interacted with us here. @malin: since this is meant to be a community SIG where people can join whenever they wish, I’d expect the members list to change (hopefully increase) regularly. Does it make sense to have a static page for the member list, or should we link to the GitHub group or another page for membership?
A section stating the SIG’s goals and planned outcomes (no limit, as long as you need): The SIG is designed to not be linked to any one particular goal to allow it remain flexible enough so that it can take on projects that its members are interested in working on. Currently, the goals of the SIG are a) write up a set of best practices in software development and encourage software developers in the research community to check their tools against there; b) host regular “developer sessions” where developer teams of various tools discuss their development pipelines—to disseminate various development practices, and help potential contributors get started.
A short statement on your SIG’s planned way of working (e.g. biweekly or monthly virtual meetings): The primary mode of communication for the SIG will remain on Neurostars. However, the regular developer sessions will be supplemented with virtual SIG meetings to allow our global membership to communicate with each other.
A list of currently planned and recent meetings, if any: Not one yet, WIP.
if possible, a group photo - or a zoom group screenshot: Will be able to provide one after the first one.
It’s a CMake/C++ project, and we use Github for just about everything.