Measuring two brains simultaneously.
What is hyperscanning?
Exchanging educational conversations between teacher and student? Competing with fellow athletes in a triathlon race? Recognizing cooperation between violinists during a classical concert? These are a few examples of social interactions that are a vital part of human life.
When individuals interact with each other, perceived information is shared between their brains. In the last decades, new technologies have allowed neuroscientists to investigate the brain activity of multiple subjects simultaneously. This novel technique, called hyperscanning, facilitates the exploration of interbrain neural relations, helping researchers understand the neural foundations of social interactions.1 Hyperscanning can be done using multiple modalities, either through recording brain activity directly (EEG) or indirectly (fMRI and fNIRS). This blog explores different recording modalities, discusses how to record and analyze hyperscanning data, and highlights several applications of hyperscanning.
What are the different types of hyperscanning methods?
Hyperscanning with EEG
Electroencephalography (EEG) is the most-used neuroimaging method for hyperscanning studies.2 With EEG, brain activity is measured directly by electrodes on the scalp. These measurements are recorded electrically, rather than indirectly from changes associated with blood flow, as with fMRI and fNIRS. This method using EEG is very portable and has the best temporal resolution compared to the other methods. However, it has the lowest spatial resolution.2 Researchers have shown that this modality can be extended to more than two people at the same time.4
Hyperscanning with fMRI
Functional magnetic resonance imaging (fMRI) hyperscanning studies require each participant to lie completely still in the scanner. In this scanner, brain activity is measured indirectly, by detecting changes associated with blood flow: the blood-oxygen-level dependent (BOLD) contrast.5 As the participants are forced to lay still, usually in different scanner rooms, the interaction between participants is very limited. These interactions are often performed through a computer interface, which has questionable real-world validity.6 Due to this limitation, this method is mostly chosen in hyperscanning studies when great spatial resolution is required, as this technique allows the mapping of deeper brain structures.1
Hyperscanning with fNIRS
Functional near-infrared spectroscopy (fNIRS) is another method that indirectly measures brain activity. It does this by detecting the ratio between oxygenated and de-oxygenated hemoglobin. fNIRS is a portable measurement modality, allowing for multiple different, naturalistic research settings and free mobility of the participants. Additionally, the temporal resolution is higher and the spatial resolution is lower compared to fMRI.7 As the spatial resolution is higher than EEG, this modality can be used in hyperscanning studies when good spatial resolution is desired in a portable setting. As of recently, EEG and fNIRS modalities can be combined for hyperscanning studies.
In the rest of this blog, we will be focusing on hyperscanning measurements using EEG, as it is the most used method.
How to record and analyze hyperscanning data?
How can you record hyperscanning data synchronously?
Synchronization is a key aspect of a hyperscanning experiment, as meaningful comparisons between subjects can only be performed when collected data streams are synchronous over time. For the analysis of hyperscanning data, synchronization of the data needs to be done first. At TMSi, we provide amplifiers and headcaps which can measure participants during hyperscanning situations synchronously. An example of one data synchronization option is shown using TMSi's high-density SAGA amplifier below (Figure 1). This figure shows the use of the sync in and sync out ports of SAGA used for the synchronization of data. Alternatively, the lab streaming layer found in our Python library can be used as a supporting tool for the synchronization of data.
Figure 1: Illustrated example of data synchronization from two participants during a hyperscanning experiment.
How can you analyze hyperscanning data?
Two types of measures are used most commonly for the analysis of hyperscanning data: connectivity and correlation.1 Connectivity measures the similarity between two neural signals coming from different brains. Alternatively, the synchrony between two brains can be estimated by calculating the correlation between the two signals.1
What does an experiment set-up look like?
Hyperscanning studies are often done in a natural environment with data collected simultaneously from two or more subjects.
An example of an experiment is described in a music study by Greco et al.3, which investigated the neural synchrony between musicians. In this study, four professional saxophonists (soprano, alto, tenor, and baritone) observed their own concert performance while EEG signals were being simultaneously recorded. Before the experiment, the participants were fitted into a pre-wired EEG cap prepared by researchers. The music performance was composed in a way that each saxophonist took turns being the musical leader of the quartet. This study found that there was significantly lower hyperconnectivity in the left Brodmann area 44 (around F3 and F7 in the 10-20 system), which is attributed to music generation, empathy processes, and communication, for the soprano compared to the other members.
Figure 2: An example of a hyperscanning EEG experiment of interacting musicians.
What are other applications for hyperscanning?
Hyperscanning application categories include joint action, social neuroscience, social interaction, and interpersonal coordination. Within these application categories, successful applications have been established in many fields, such as speech studies, common vision studies, temporal motor synchronization, shared attention, decision-making, classroom learning, and music production.2 Hyperscanning studies have also looked at coordination and synchronization, emotion and affect, cooperation and competition, or games and decision making.1
Conclusion
In summary, hyperscanning is a method used to evaluate social interaction, by observing and analyzing simultaneously collected neuronal signals from two or more subjects. With these different modalities, methods, and applications, there are vast and interesting possibilities for future research studies on the social brain. Are you a researcher who would like to perform hyperscanning studies? TMSi provides headcaps and amplifiers, which can be used synchronously, to assist you with your research. These products, along with more information, can be found here.
References
1. Czeszumski A, Eustergerling S, Lang A, et al. Hyperscanning: A Valid Method to Study Neural Inter-brain Underpinnings of Social Interaction. Front Hum Neurosci. 2020;14:39. Published 2020 Feb 28. doi:10.3389/fnhum.2020.00039
2. Nam CS, Choo S, Huang J, Park J. Brain-to-Brain Neural Synchrony During Social Interactions: A Systematic Review on Hyperscanning Studies. Applied Sciences. 2020; 10(19):6669.
3. Greco A, Spada D, Rossi S, Perani D, Valenza G, Scilingo EP. EEG Hyperconnectivity Study on Saxophone Quartet Playing in Ensemble. Annu Int Conf IEEE Eng Med Biol Soc. 2018;2018:1015-1018. doi:10.1109/EMBC.2018.8512409
4. Dikker S, Wan L, Davidesco I et al. Brain-to-Brain Synchrony Tracks Real-World Dynamic Group Interactions in the Classroom. Current Biology. 2017;27(9):1375-1380. doi:10.1016/j.cub.2017.04.002
5. Glover G. Overview of Functional Magnetic Resonance Imaging. Neurosurg Clin N Am. 2011;22(2):133-139. doi:10.1016/j.nec.2010.11.001
6. Konvalinka I, Roepstorff A. The two-brain approach: how can mutually interacting brains teach us something about social interaction?. Front Hum Neurosci. 2012;6:1-10. doi:10.3389/fnhum.2012.00215