GSoC 2020 project idea 9: Realtime simulations

Brian 2 can generate code in a so-called “standalone mode”, where it generates a full set of code files that encapsulate the full simulation without references to external libraries (apart from the standard library). Furthermore, it allows to combine high-level simulation code (differential equations, etc.) with user-specified low-level functions written in the target language (e.g. C++). This can be used to connect a neural simulation to external input (e.g. from a camera or a microphone) or to actuators (e.g. a motor). These features make Brian 2 potentially highly suited to write neural simulation code that runs on an external device such as a robot. However, currently the simulation time advances in fixed time step, completely independent from the “real time”. To make Brian more useful in such contexts, we would therefore like two support two modes of “realtime” simulation:

  1. a mechanism that slows down a simulation so that its progress matches the realtime

  2. a mechanism that adapts the simulation time step so that it matches the real time that has passed since the last update

Skills: C++ and Python programming, experience with Brian or other neural simulators helpful

Mentors: Marcel Stimberg (marcel.stimberg@inserm.fr) and Dan Goodman (d.goodman@imperial.ac.uk )

2 Likes

Hi @mstimberg ,
I’m Stefano Manunza, a 2nd-year undergrad Computer Science student from Padova, Italy.
I’m interested in the field of computational neuroscience, and I’m slowly approaching the subject, but I have had no experience so far and I want to improve that.
I believe that this project idea may be a good starting point for me since I have some experience with C++ and Python, and I think it would be kinda fun.
However, I have never used a neural simulator, and I’m not totally sure what Brian2 is capable of, but I’m working on it: so far I’ve managed to install it and to run the tests, and I’ve started reading the documentation.
Is there anything I can do to better understand how Brian works other than that?
How can I start contributing the project?

Hi. Thanks for your interest in our project. I’d say that this project (and our other proposed project) do only require very limited knowledge of computational neuroscience, so this is not a major obstacle. As a starting point, I’d have a look at our 2014 and 2019 papers about Brian – the first is a bit more about the technical details and the second more about the usefulness from a user’s point of view. For a more practical introduction, have a go with our tutorials. Getting a bit of background in computational neuroscience certainly wouldn’t hurt, you can have a look at Dan Goodman’s website for some freely available material. But again, no need to become an expert for this project.

2 Likes

Hey ,this is Yamini Jhawar,a 3rd year Computer Science student from Bits Pilani,Goa .I am currently working on solving the neuronal dynamics exercises running on the brian platform under my professor .So I am interested to contribute .Can you please guide me through the process what should i start with?

Hi, thank you for your interest in the project. The exercises from the book are a great way to learn about computational neuroscience, but be aware that their code hides away a lot of how Brian works. To get more familiar with Brian itself I’d recommend having a look at the tutorials: https://brian2.readthedocs.io/en/stable/resources/tutorials/index.html

For the project, it will be important to understand how Brian’s code generation mechanism works. Have a look at section 3 in our 2014 paper or e.g. at section 2.1 in the code generation review paper that we co-authored. Finally, to really get a detailed understanding of what is happening behind the scenes you can run a simple example with Brian’s standalone mode and have a look at the generated code.

I have gone through the above mentioned sites.Now can you please give me a concrete example of this: “currently the simulation time advances in fixed time step, completely independent from the real time”.i.e., in what format is the real time input ,say from a sensor, is fed into the simulation model and in what way mechanism slowing the simulation is to be applied i.e, a c++ function is to be written or changes in source code of Brian is to be made?

I want to start analysing the solution for the problem,can someone please help me with the query?

Hi. There is no general mechanism to feed real time input into Brian, but you can use its function extension system for that task. The only actual “real time” example we have is an example from our recent 2019 paper where we feed audio input from a microphone into a model. You can find a notebook with the example here: https://github.com/brian-team/brian2_paper_examples/blob/master/example_4_microphone.ipynb (the C++ file responsible for getting the microphone input is here: https://github.com/brian-team/brian2_paper_examples/blob/master/example_4_microphone.ipynb). We have a similar example for video input, although in the example it comes from a file (but it could use the same framework to get the input from a camera): https://brian2.readthedocs.io/en/stable/examples/advanced.opencv_movie.html These examples use quite naive blocking operations and might actually not be real time after all.

The “slow down” mechanism should be quite straightforward to implement. Inside the C++ code we’d need to keep track of the real time in addition to the simulated time, and use a sleep operation if necessary. On the Brian side, it would only need some minor changes to expose some syntax triggering the mechanism. The second mechanism (adaptive time step) would be more involved on the Brian side, since many parts of the code assume that the time step cannot change during a simulation and therefore do some pre-calculations with a fixed time step.

Now I get a much clear idea .I will work on it and keep you updated
.

Dear students interested in this project. Please see my comment on project 8 for a task (https://github.com/brian-team/brian2/issues/1162) that I’d like all prospective applicants to do as part of the application process.

Dear students. As you all know, the application period for this year’s GSoC has started. I hope you are all doing well and are able to cope with the issues around the ongoing pandemic. Since GSoC is an online-only program it is obviously much less affected than other activities; at the moment no changes to the schedule (e.g. deadline extension) are planned. For some of you it might be difficult to get the necessary proof of enrolment from your university at a time of lockdowns and office closures. Google has updated their guidelines, hopefully you can find a solution that works for you. From the GSoC program lead: “We know some of you may need to get creative to show us forms and that’s okay. Submit what you have before the March 31 deadline and if there is a problem with the form you will be emailed the first week of April and can submit a different proof at that point.”

Regarding your applications for this specific project:

  • please include a link to any Brian contribution you made (e.g. to the Brian issue I linked earlier)
  • the most important thing about the application is not so much the detailed timeline (it’s always hard to plan these things in advance), but rather that you show that you understand the task and its potential problems. You can refer to the existing examples I mentioned earlier and discuss what would change in a realtime setting. For example, where in the generated code would be the best place to slow down a simulation if it advances too fast? In the opposite case, what solutions are possible if a complex simulation is running too slow to cope with incoming video/audio input? Whenever you can, try to show concrete examples. These can be something like an “imagined” generated code snippet. I.e., try show the outcome of what your code should do in the end.

Hopefully this makes things a bit clearer, let me know if you have any further questions.

Hello Everyone!
My name is Osama and I am in my final year, pursuing a bachelor’s degree in Computer Science from University of Karachi, Karachi, Pakistan. I have been working on web related technologies for the past one year (Node, React, Django). I like to use C++ in my course projects as I think it’s a great language to learn core programming concepts. I have been studying different research papers related to BCI, EEG and neuroscience but I haven’t worked with a brain simulator before and have started watching tutorials on TVB. I would be interested in this project. I hope I’m not too late. Thanks!

Hi Osama. Thank you for your interest in the project. Please have a look at the linked documents in the comments above, and at the Brian simulator in general (https://briansimulator.org) to see whether this is something you’d be interested to work on. There are also a number of projects for TVB, and several related to web technologies – maybe these would be a better fit? However, it will be difficult to discuss any type of project in detail in the short remaining time, so this certainly puts you at a disadvantage with respect to students that had back-and-forth discussion with their mentors for some time.

Thank you for the suggestions. I have already gone through most of the project ideas here. Most of them have requirements that don’t match with my skills set. Those which match don’t perfectly align with my interests. I have gone through some of the links and documentation for the project. I am known to be a fast learner. I hope that I can catch up.

I have also gone through the code. Can you clarify what variant of cpp is used in the standalone module? I have very basic knowledge of cython but it seems a bit different.

Hi Osama. The standalone mode uses standard C++ (C99). Cython is used for the runtime mode. Is this what you were asking? I’m not 100% sure I understood your question correctly.

Sorry for the confusing question.
I was referring to the following syntax which closely resembles django ( python based web framework). I haven’t seen this syntax used in C++ before.Capture

Ah ok, sorry now I get it. This is not the final C++ code but the template that is used for code generation. This will be filled with the information from the actual model defined by the user to get the final C++ code. The templating language we use is Jinja2, which is indeed quite often used in web frameworks.

Thank you for the clarification!

Hello @mstimberg. I have made some changes in the project to solve the issue you posted earlier. Can you guide how can I test the changes if they worked or not?

Hi Osama, are you referring to github issue #1162? If yes, to test it you can run one of the standalone examples, e.g. https://brian2.readthedocs.io/en/stable/examples/standalone.STDP_standalone.html . If needed, you can increase the runtime a bit so that the displayed elapsed/estimated remaining time is big enough to go into minutes/hours.