[Eeglablist] Test-retest for Mobile EEG in Older Adults

[Cedric Cannard]

Dear Charlotte,

Sorry for the slow response. What is the task? And what do you mean by test-retest reliability of the source estimates here?

Since you’re using mobile EEG, my main concern would be, how are you controlling for movement artifacts?  Do you have IMU data or other modalities to help control for movements? Movement inconsistency between sessions will dominantly affect test-retest reliability.

You mention AMICA then source localization, what method are you using for source localization? Or did you mean source separation?

I’d recommend only rejecting components where your classifier has very high confidence (>90%) to avoid removing good brain signals by accidents.

K-means clustering may overcomplicate things, clustering can vary between runs, which confounds the reliability you’re measuring.

I would consider simpler approaches:
	∙	ICCs or Cronbach Alpha for spectral power in matched sources between days
	∙	Bland-Altman plots to visualize agreement and systematic biases
	∙	Bootstrap-based testing (e.g., using LIMO-EEG with TFCE correction) for robust mass univariate comparisons without parametric assumptions
	∙	Bayesian credible intervals to quantify evidence for stability

For within-subject matching (your Option 2), you could match sources between days using dipole location and spatial correlation, then calculate reliability on matched pairs?

Cedric Cannard

Sent from Proton Mail for iOS.

-------- Original Message --------
On Tuesday, 12/09/25 at 13:25 DeVol, Charlotte via eeglablist <eeglablist at sccn.ucsd.edu> wrote:
Hello all! My name is Charlotte DeVol, and I am a relatively new EEG researcher working with Dan Ferris. We are planning a study on the stability of source-level mobile EEG recorded 6 months apart in ~30 older adults, and we would appreciate input from the community on the best analysis strategies for a test–retest comparison. Below I outline our current pipeline, possible approaches we are considering for how to integrate a test-retest comparison, and the limitations we see so far.

High-level overview of our typical analysis pipeline run in EEGLAB:
|EEG preprocessing| --> |AMICA| --> |Source localization| -->
|Reject non-brain components| --> |K-means clustering to group sources across subjects| --> |Spectral analysis|

Possible methods to compare source-level EEG between two visits:

  1.  Combine data from Day 1 and Day 2 before running AMICA. Do all subsequent steps with a combined dataset until the spectral analysis step.
     *   Potential limitation: Electrode positions differ slightly between days. In other approaches we can account for this using custom head models during source localization, but that may not be possible if AMICA is run on combined data.
  2.  Run AMICA, source localization, and rejection of non-brain components separately for Day 1 and Day 2, but do not do k-means clustering across subjects. Instead, look for nearest sources from Day 1 to Day 2 within each subject and compare those pairs.
     *   Potential limitation: K-means clustering typically ensures that results reflect source regions common across many subjects (e.g., >50%). Without clustering, additional methods would be needed to aggregate subject-level results.
  3.  Perform the full pipeline separately for Day 1 and Day 2, including k-means clustering. Then compare group-level clusters between Day 1 and Day 2.
     *   Potential limitation: The same cluster from Day 1 and Day 2 may include different subjects, making it challenging to determine if true instability is the  cause for differences or if it is the subjects.


We would be grateful for any feedback or suggestions as to which method would be the best approach or if there are alternatives we have not considered. Thank you all!



Charlotte R DeVol, PhD (she/her)

T32 Translational Research in Aging and Mobility Postdoctoral Fellow

Department of Health Outcomes & Biomedical Informatics

Human Neuromechanics Laboratory<https://urldefense.com/v3/__https://faculty.eng.ufl.edu/human-neuromechanics-laboratory/__;!!Mih3wA!CfX8fvAQHg53LaYm9X9EOABM-z0lkSmey6MZrN_-UX-vI0p4MIjpOZ6IkuPxLucQU2Cr9ZFMd-JEj__Zn1SLNuaSQlNDYVsSYg$ >

J. Crayton Pruitt Family Department of Biomedical Engineering

University of Florida


_______________________________________________
To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu or visit https://sccn.ucsd.edu/mailman/listinfo/eeglablist .