The EBRAINS Software Distribution (ESD) is a curated collection of tools for computational neuroscience, supporting education, research, and simulation across heterogeneous hardware — including HPC, GPUs, and neuromorphic computing systems. Many ESD tools are developed by the research community and leverage diverse hardware architectures for performance and scalability.
This project aims to integrate, parameterize, and automate a comprehensive benchmarking suite for key neural simulators within ESD. The focus will be on established benchmarks for widely used simulators like Arbor, NEURON, and NEST, creating a unified framework for performance validation, regression testing, and hardware profiling across EBRAINS cloud and HPC environments.
Key Objectives:
- Integrate and Unify Simulator Benchmarks
- Develop a framework to incorporate existing benchmark suites into spack package descriptions for:
- Arbor: HPC-focused simulator benchmark suite
- NEURON: Widely-used classic simulator performance tests
- NEST: Large-scale network simulation benchmarks
-
Parameterize Benchmark Configurations
-
Design a flexible configuration leveraging the related ESD spack package descriptions system allowing users and CI systems to easily adjust:
- Model size and complexity
- Hardware targets (CPU core counts, GPU usage, MPI configuration)
- Output granularity (performance summaries, detailed profiling)
-
Embed Micro-Benchmarks for System Profiling and Performance Verification
-
Integrate low-level performance tests into the ESD to complement simulator benchmarks:
- OSU Micro-Benchmarks — Network and MPI performance
- nccl-test — Multi-GPU communication scaling
Technology Stack & Required Skills:
- Primary (Must Have): Python and Shell Scripting (Bash)
- Secondary (Good to Have): SLURM, Spack Package Manager, System Benchmarking