The EEGLAB News #8
Fiorenzo Artoni, Ph.D.
Maître Assistant,
Functional Brain Mapping Lab
University of Genève, Genève,
Switzerland
It sounds like a Sci-Fi movie: A man loses his hand in an accident. Scientists fit him with a state-of-the-art bionic hand, using artificial intelligence and robotics. They use algorithms to decode the patient's brain signals, which can predict how he plans to move his fingers, and robotics to help him grasp objects. The patient learns to move his new artificial hand with ease, returning to his normal activities instead of facing a once-debilitating condition.
This seemingly fictitious scenario is quickly becoming reality, thanks to researchers like Dr. Fiorenzo Artoni, Biomedical and Automation Engineer, currently Maitre Assistant at the Functional Brain Mapping Lab in Switzerland. Dr. Artoni recently completed a 2-year H2020 Marie Curie Project, BIREHAB, aimed at creating the tools needed to build a more 'life-like' bidirectional prosthetic hand that can be naturally controlled and that can deliver touch sensations to improve the lives of amputees.
"Progress in the field of prosthetics is essential," Dr. Artoni emphasizes, "as in the United States alone, about 1.6 million people live with an amputation, 541,000 of which affects the upper limbs.” Advances in the field can have dramatic positive effects, as hand loss is a very disabling condition that decreases quality of life, mobility and independence. But developing effective prosthetics can be challenging. Many available prosthetic hands offer limited feeling and a restricted range of motion. “Prostheses are only as useful as the richness of the sensation (i.e., “natural” - or as close to natural as possible - sensory feedback), and dexterity (ease of control and capabilities close to the intact human hand) that they provide.” In other words, Dr. Artoni explains, in order to feel a limb as really our own, we need to be able to control it but also simultaneously receive and process all the information from the outside world. (Photo: Event-related potentials elicited by somatotopic median and nerve transcutaneous electrical nerve stimulation (TENS); Credit: Artoni).
He continues, "What we perceive in our exploration of the world around us by touching is essential to our close interactions with our physical environment and particularly, with objects within it. Touch involves a variety of different receptors in the skin (which are of course missing in an amputee with a prosthesis). Typically we take touch for granted, but we realize its importance quickly when it is missing. However, physical touch sensations are only one part of the story. From the viewpoint of cognitive embodiment, our hand is also a cognitive entity embodying our sense of active touch while also, as our primary physical touch effector, being the organ itself comprising actuators and sensors that ultimately maintain the link between our tactile perception and effective action. Importantly, touch sensation is combined with continuous feedback from hand and arm muscles and tendons as we actively move and explore the world. This proprioceptive information tells us about the position of our tactual surfaces, both relative to the rest of our body and to the external world. These features of touch are unique and raise many interesting philosophical issues. In particular, touch is a central topic of discussion in debates about the multisensory nature of perception, relationships between perception and action, and connections between touch and bodily awareness."
"The aim of the project," Dr. Artoni maintains, "was to develop both software (i.e., how best to disentangle real myoelectric information from noise) and hardware tools (i.e., defining and developing the optimal recording equipment for high density and medium density surface electromyography - EMG) to characterize ‘referred’ sensations that an amputee can actually feel on their phantom hand."
The BIREHAB Project has provided important insights for future research in the field. In fact, it was so successful that it was recently featured as a EU Success Story: Building a better prosthetic hand. Dr. Artoni explains further, “I was able to study how the different number of EMG channels (up to 400) influenced the performance of a EMG-based grasp classification decoder and developed a novel medium-density EMG dry 64-channel sleeve that drastically reducesEMG recording setup times as well as the need to locate each muscle individually. I was also able to characterize the neural correlates of different referred touch sensations elicited by electrical stimulation of the median and ulnar nerves."
In parallel, the collaboration with the TNE Lab at Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland led to successfully using artificial intelligence to test new technology that combines robotic control with users' voluntary control. This approach -- shared control between machine and user intention -- is an advancement in neuroprosthetic technology (see video in End Notes). Dr. Artoni explains, "We proposed a novel shared control scheme in which robotic automation aids in object grasping by maximizing the contact area between the hand and the object, thus greatly increasing grasp success and object hold times in both a virtual and a physical environment. These results presented a viable method of prosthesis control implemented in real time, for reliable articulation of multiple simultaneous degrees of freedom." (Photo above: Researcher Fiorenzo Artoni demonstrates the shared control system, using a robotic arm. Credit EPFL/Alain Herzog).
Research. “What inspires my research,” Dr. Artoni shares candidly, “is my wish to provide patients who have neurological impairments effective and personalized therapy with an optimized path of recovery, and to help them regain the possibility of leading a life worth living.” In order to achieve this goal, Dr. Artoni focuses his research on three main topics: Novel Artificial Intelligence algorithms to decode brain signals and improve prostheses control, novel Data Science tools (e.g., RELICA, discussed below and Microstates analysis) to better capture brain activity and disentangle it from noise, and the development of novel Mobile Brain/Body Imaging (MoBI) tools to image the brain and assess the neuro-biomechanical state of the patient while moving in his/her natural environment. (Image at left: Mobile Brain/Body Imaging setup with EEG, EMG and gait phases synchronized and recorded during treadmill walking. Artoni et al., 2017).
On this last topic, for instance, Dr. Artoni is looking for funding to further develop the tools to assemble a "modular, high-resolution, real-time, non-invasive, whole-body neuroimaging MoBI platform." With minimum effort from the user, the system will be able to synchronize, collect, integrate, store and analyze physiological measures including EEG, electromyographic (EMG), kinematics and ground-reaction-force (GRF) data. "This will provide clinicians with reliable real-time cortico-cortical and cortico-muscular connectivity measures and other useful information for the assessment of the neuro-biomechanical state of the patient."
Beginnings. Born and raised in Casalmaggiore, a charming small town near Milan in the north of Italy, Dr. Artoni quickly showed a passion for mathematics and science. With encouragement from his teachers, he began participating in competitions that expanded his knowledge – the International Mathematics Olympic Games (bronze medal) and the European Union Contest for Young Scientists (2006, winner). There were no limits to his exploration: “I was very lucky to live in such a culturally rich environment that allowed me to explore my passions for classical guitar (which I played since the age of 6), singing (in the Gospel “Joy Voices Youth Choir”), sports (orienteering and basketball) and baking (pastry).”
Dr. Artoni threw himself into his studies, earning a competitive position as Allievo Ordinario at Sant’Anna School of Advanced Studies (SSSA) – being one of only 11 students chosen for engineering, nationwide, that year. He was mentored by Prof. Maria Chiara Carrozza and Prof. Alberto Landi, who successfully guided him to obtaining a double BSc degree in Biomedical Engineering and Industrial Engineering, and a double MSc degree in Automation Engineering and Industrial Engineering. From there, he obtained his Ph.D. in Biorobotics and was a post-doc at The Biorobotics Institute at SSSA under the supervision of Prof. Silvestro Micera.
Dr. Artoni's curiosity about the brain rapidly transformed into interest and then passion. “I have always been fascinated by the complexities and mysteries of the incredible machine that is our body,” Dr. Artoni admits. “Every action we do, no matter how trivial, requires such a complex control and structural integration. For example, think about something as simple as grabbing a paper glass half-full with water: you need to plan ahead the grabbing action, coordinate all the muscles so that their contraction brings the joints and the hand exactly in the right position, time the opening and closing of the hand perfectly and finally adapt in real time the pressure to apply to the glass so as not to squeeze it so much that the water would spill but also so little that the glass would slip from the hand. And at the origin of it all, there is always the brain.” (Image at right: Representation of a touch experiment in which tactile stimuli with different levels of roughness were slid beneath the subject's finger. Meanwhile, the EEG was recorded and analyzed to explore the neural correlates of touch. Genna et al., 2018).
Swartz Center for Computational Neuroscience (SCCN). During his studies, Dr. Artoni visited SCCN twice – once during his MSc (2011) and then during his Ph.D. (2014). “I immediately fell in love with the lab, the people, the impeccable organization, the passion for research that you could touch everywhere, the lasting friendships and of course the fun, the tea times, the concerts, the table tennis matches," he admits readily. "I feel extremely lucky, as the research of the SCCN lab is of inspiration to the whole neuroscientific community and I believe that collaborating with Prof. Scott Makeig and Prof. Arnaud Delorme is the dream of all neuroscientists. Certainly, it was mine. I had the honor of working with them on very interesting projects, and we have been collaborating ever since to explore algorithms and methods to extract reliable information from EEG.”
EEGLAB and RELICA (RELiable ICA). One of the prominent results of Dr. Artoni’s collaboration with Drs. Makeig and Delorme is the development of RELICA, a toolbox now offered within the EEGLAB distribution.
Dr. Artoni prefaces, “EEGLAB is the most popular software to process EEG data by a wide margin and with good reason. It is open source and allows users with even little background to develop complex analysis pipelines and extract valuable information from their data. Also, EEGLAB comes with an extensive support structure formed by tutorials, a very active EEGLAB support list and of course the EEGLAB worldwide workshops in which I often participate as mentor/staff. I currently use EEGLAB in all my projects involving EEG and it has proven to be, time and again, an essential asset.”
He goes on to explain RELICA (Artoni et al., 2014) and its value in the field: “RELICA is a novel method to characterize Independent Component reliability within subjects. Independent Component Analysis (ICA) is a widely applied data-driven method for parsing brain and non-brain EEG source signals, mixed by volume conduction to the scalp electrodes, into a set of maximally temporally and functionally independent component (IC) processes. Many ICs may be identified with a precise brain physiological (or non-brain) origin." But, he explains, this process is hindered by partial instability in ICA results that can arise from noisy, nonstationary, or insufficient data. "RELICA," he continues, "first computes IC ‘dipolarity,’ a measure of physiological plausibility (Delorme et al., 2012), plus a measure of IC consistency across multiple decompositions of bootstrap versions of the input data. RELICA then uses these two measures to visualize and cluster the separated ICs, providing a within-subject measure of IC reliability that does not involve checking for its occurrence across subjects. RELICA might be used with any linear blind source separation algorithm to reduce the risk of basing conclusions on unstable or physiologically un-interpretable component processes. RELICA is available from the EEGLAB Extension Manager, and from its GitHub repository (https://github.com/sccn/relica - For more information, read "What is RELICA?" midway down the page).
To enable RELICA decomposition of large datasets, we implemented two easy routes, from the EEGLAB GUI or MATLAB command line, to optionally run the RELICA processing on the high-performance computing (HPC) resources of the U.S. XCEDE network via The Neuroscience Gateway. (See Martínez-Cancino et al., 2020 for more details). Results of RELICA can be used to judge confidence in conclusions drawn about individual ICs or about the stability of between-subject IC clusters." Recently, Zeynep Akalin Acar and Dr. Makeig showed that using IC map covariance across a RELICA IC cluster during Sparse-Compact-Smooth (SCS) IC source localization (Cao Cao et al., 2012) can contribute to its accuracy (Akalin Acar & Makeig, 2021).
Figure at right: Scalp maps of select clusters of reliable ICs calculated by RELICA. Each cluster refers to one Independent Component (IC), whose centroid is represented with a yellow dot. The picture also shows the scalp topography and localized source of a few select ICs (e.g., representing saccades, mu-rhythm etc. Artoni et al., 2014)
Dr. Artoni continues, “An interesting application of RELICA also led us to demonstrate that, while dimension reduction by Principal Component Analysis (PCA) has often been recommended before ICA decomposition of EEG data, both to minimize the amount of required data and computation time, PCA rank reduction reduced the numbers of near-equivalent ICs across subjects and increased uncertainty in the equivalent dipole positions and spectra of the IC brain effective sources. These results fundamentally demonstrate that, when applying ICA decomposition to EEG data, PCA rank reduction should best be avoided.”
The Future. In 2020, Dr. Artoni moved to the Functional Brain Mapping Lab (Prof. Michel Christoph) as Maître Assistant, where he is involved with a challenging project “aiming to describe, with brain microstates, the path to unconsciousness of anesthetized patients.” He continues, “I am convinced that microstates syntax is an excellent descriptor of brain non-stationary processes and pathologies and I am eager to unify microstates and Independent Component Analysis to reach a holistic description of the human brain.”
In another recent project, Dr. Artoni recently validated a Long-Short-Term Memory (LSTM) deep learning model to decode gait phases from Electroencephalography (EEG) and demonstrated how the use of hybrid EEG and EMG interfaces can increase the stability decoding performance of Brain Computer Interfaces (Tortora et al., 2020a; Tortora et al., 2020b).
Dr. Artoni plans to continue developing novel technology and exploring the mysteries of the brain, especially as it relates to helping improve peoples' lives. “I am not only interested in how the brain works in itself,” he shares, “but rather in how our increasing technical knowledge can be successfully used to help people who suffer from brain damage such as stroke, Parkinson’s disease, etc. Or to help people who have lost a limb and need devices that can help them regain their independence and the ability to lead a normal life.” He pauses, concluding, “For an engineer like me, studying the brain cannot be anything but the most fascinating research topic of all, the ultimate engineering challenge, as we study the essence of life itself.”
R. Weistrop, April 2021
_______________________
Video above: Researcher Fiorenzo Artoni demonstrates grasping wth a robotic arm, using the shared control system. Credit EPFL.