Papers

Toward OpenEEG-Bench: A Live Community-Driven Benchmark for EEG Foundation Models

P Guetschel, B Aristimunha, D Truong, K Kokate, M Tangermann, A Delorme

Proceedings of the 34th European Signal Processing Conference …•bruaristimunha.github.io

Abstract

The rapid emergence of foundation models for electroencephalography (EEG) promises to transform braincomputer interfaces and clinical neuroscience. In many cases, however, results reported on foundation models are snapshots in time, which are hard to compare due to heterogeneous evaluation protocols, such as differing pre-processing, datasets, data-splits, finetuning methods etc

To mitigate this, a few benchmark papers have recently been proposed that try to standardize the comparison of foundation models. Compared to other domains, however, the EEG field still lacks a continuously updated, open-source benchmark which provides a live online leaderboard and allows for a fair, reproducible comparison of current foundation models, and also allows for introspecting which design decisions are impactful. In this paper, building upon our experience organizing the NeurIPS 2025 EEG Foundation Model Challenge, we describe our four initial design choices for a live EEG foundation model benchmark, that implements a continuously updated leaderboard:(1) It embraces existing open-source tools, including MNE-Python, Braindecode, and HuggingFace for easier adoption.(2) We recommend using openly accessible datasets and will include new datasets in the future.(3) We standardize the finetuning and pre-processing procedures for comparable results.(4) We implement a community-driven governance to ensure long-term sustainability..