multimodal Archives - BIOPAC Blog

Capturing More Than Physiology: Multi-Camera and Multimodal Research

In the early 20th century, Cubist painters like Pablo Picasso and Georges Braque argued that traditional linear perspective on a subject was an illusion. Something was always hidden from the viewer, distorted, or lost altogether. Just as a Cubist painter might fracture their subject to reveal its true shape, a physiology lab uses multiple synchronized camera angles to bypass a single lens’ limitations.

The same principle applies to modern physiological and psychophysiological research. Physiological signals such as electrocardiogram (ECG), electroencephalogram (EEG), electrodermal activity (EDA), respiration, and blood flow provide valuable insight into how the body responds to stimuli, environments, and social interactions. Yet physiological data alone often leaves important questions unanswered. What was the participant doing? What was happening in the environment? Did social interaction, movement, or behavioral events coincide with a change in the recorded signal?

To answer these questions, many studies now combine physiological recordings with multiple video streams, eye tracking, audio, environmental monitoring, and other forms of media capture. This multimodal approach lets investigators examine physiological measurements alongside the events occurring around them, revealing a richer, more complete picture of human and animal behavior. As these data streams multiply, so does the need to capture, synchronize, and review them together.

Researchers at Macquarie University in Australia and collaborating institutions recently described a multimodal laboratory designed to study face-to-face communication and social interaction. The platform centered on a BIOPAC data acquisition system, amplifiers, and amplifier input module that acquired ECG, EDA, photoplethysmography (PPG), respiration, and skin temperature. The researchers combined physiological measurements with a Tobii eye tracker, wearable sensors, audio recording, motion analysis systems, and six video cameras positioned throughout the laboratory. The project showed that adding more recording sources often complicates data collection coordination, and that integrating multiple acquisition systems requires careful synchronization and workflow design. The camera system played a particularly important role. They used multiple viewing angles for behavioral coding, gesture analysis, participant observation, and synchronization verification. Rather than serving only as a record of the experiment, the video data became an important part of the analysis itself. For researchers studying communication, teamwork, education, or human-computer interaction, the work illustrates how behavioral observations can enhance interpretation of physiological measurements.

A team from the Swedish University of Agricultural Sciences (SLU) explored similar ideas outside the traditional laboratory by developing the SLU Multisensory Outdoor Laboratory (SENSOLA). The researchers used BIOPAC data acquisition and analysis systems with AcqKnowledge software and BioNomadix wireless devices to collect respiration and PPG measurements as participants moved through natural and built environments. Their platform combined those recordings with eye tracking, stimulation, video, audio, GPS data, participant feedback, and environmental measurements. One key observation was that physiological measurements become much more insightful when paired with information about a participant’s surroundings and experiences. Video proved especially valuable because it documented environmental conditions, participant behavior, and unexpected events that might otherwise be difficult to identify during later analysis. As physiological research moves beyond controlled laboratory settings, projects such as SENSOLA demonstrate the value of capturing physiological and observational data together.

Preclinical research is increasingly applying the same principles. In a recent toxicology study from Japan, investigators developed the ToxQb platform around a BIOPAC data acquisition system and AcqKnowledge software with NDT to collect EEG and ECG data from laboratory rats. The physiological measurements were integrated with overhead, front, and side-view cameras, along with thermal imaging systems. The goal was to monitor behavioral and physiological changes simultaneously over extended periods. By reviewing camera footage alongside EEG and ECG recordings, researchers could relate changes in movement, posture, and activity to physiological events. The authors concluded that combining these diverse data streams provided deeper insight than any single measurement could. The study demonstrates that multimodal acquisition extends beyond human research, across a wide range of life-science applications.

Although these studies span communication research, environmental psychology, and toxicology, they point to a common trend: researchers are capturing much more than physiology. Like the artists who challenged the limitations of a single viewpoint, today’s researchers increasingly rely on multiple synchronized perspectives to reveal a more complete picture of physiology, behavior, and environment. Multiple cameras, video recordings, eye tracking, environmental sensors, and behavioral observations can provide valuable context for interpreting physiological measurements. Learn how using BIOPAC’s MultiCapture feature in AcqKnowledge 6 to integrates multi-camera video and other media into research via our webinar on the topic.

Are you planning to integrate multiple camera perspectives in your next study? Our sales team can help find the right equipment and show you how to leverage AcqKnowledge‘s MultiCapture feature to achieve the best possible results.

Neuroergonomics: Measuring Cognitive Workload in Real-World Tasks

In some high-stakes jobs, the pressure can feel overwhelming. From air traffic controllers tracking dozens of aircraft simultaneously to surgeons guiding robotic instruments through delicate procedures, many modern occupations require people to process extraordinary amounts of information while maintaining accuracy and focus. As workplaces become more automated and technologically sophisticated, understanding how the brain and body respond to these demands has become a central challenge for ergonomics researchers. A growing area of study, known as neuroergonomics, combines neuroscience, physiology, and human factors research to better understand how people perform in complex real-world environments.

In human factors research, neuroergonomics conintues to gain traction by offering a more comprehensive picture of cognitive workload than traditional measures. Two individuals may perform a task equally well while experiencing dramatically different levels of mental effort. Excessive cognitive workload can contribute to fatigue, stress, errors, and reduced situational awareness, while insufficient workload can lead to disengagement and decreased vigilance. Researchers are therefore concentrating attention on measuring not only what people do but also how their brains and bodies respond as they do it.

Among the most valuable tools for studying cognitive workload are physiological signals that provide objective measurements of mental state. Electroencephalography (EEG) can reveal changes in neural activity associated with attention and cognitive effort, while functional near-infrared spectroscopy (fNIRS) measures changes in cortical blood oxygenation linked to mental workload. Electrocardiography (ECG) and heart rate variability (HRV) offer insight into autonomic nervous system activity, and eye-tracking measures such as pupil diameter and gaze behavior can indicate shifts in attention and cognitive demand. More often, researchers are combining multiple synchronized signals to create a more complete view of how people interact with demanding tasks and environments.

A team of researchers from Drexel University and Lockheed Martin investigated how multiple physiological signals can be combined to measure cognitive workload across a range of mental activities. Participants completed tasks involving working memory, vigilance, risk assessment, attention shifting, situation awareness, and inhibitory control while researchers simultaneously monitored EEG, fNIRS, ECG, photoplethysmography (PPG), electrooculography (EOG), and eye tracking. Brain activity was measured using an fNIR Devices system alongside other physiological monitoring technologies. The researchers found that different physiological measures varied in their sensitivity to specific task demands, with neuroimaging measures providing particularly valuable insight into changes in workload. The findings demonstrate the value of combining signals from the brain, heart, and eyes to create a broader perspective of cognitive effort in complex environments.

Researchers at Loughborough University in the United Kingdom examined how rising task complexity influences cognitive workload in modern manufacturing environments. Participants performed a series of assembly tasks while researchers recorded pupil diameter with Tobii eye-tracking and measured HRV with wearable ECG sensors. By comparing physiological responses across low-, medium-, and high-complexity tasks, the team identified significant changes in both pupillometry and cardiac indicators of cognitive load. Their findings suggest that physiological monitoring can provide an objective means of evaluating cognitive ergonomics in highly digitalized manufacturing environments and may help support the design of more sustainable, human-centered workplaces.

Another important challenge in neuroergonomics is determining which physiological measurements provide the most reliable indicators of mental workload. A collaborative team of researchers from France investigated how physiological measurements can be used to establish standardized levels of mental workload during aviation-inspired tasks. Using the Multi-Attribute Task Battery (MATB-II), participants performed simulated flight-related tasks under varying levels of demand while researchers measured EEG, eye-tracking, electrodermal activity, and cardiac activity. ECG and EDA signals were acquired using a BIOPAC data acquisition and analysis system running AcqKnowledge software. The study found that several cardiac, ocular, and electroencephalographic measures successfully distinguished high-workload conditions and correlated with subjective workload ratings. The researchers found that several physiological measures, particularly EEG, eye tracking, and cardiovascular metrics, reliably distinguished between different workload levels and closely tracked participants’ subjective assessments of task difficulty. Their findings suggest that multimodal physiological monitoring could support the development of adaptive systems that detect cognitive overload in real time, improving both performance and safety in demanding operational environments.

These studies highlight the increasingly multimodal nature of neuroergonomics research. Eye tracking provides insight into attention and visual processing, cardiovascular measures reveal how the autonomic nervous system responds to mental demands, and brain-imaging techniques such as EEG and fNIRS offer direct windows into neural workload. By combining these complementary perspectives, researchers are moving beyond simple measures of task performance toward a deeper understanding of how people think, adapt, and perform in complex environments. As workplaces continue to evolve alongside automation and intelligent technologies, neuroergonomics is helping researchers design systems that better support human capabilities, improve safety, and reduce the hidden costs of cognitive overload.

Are you planning a multimodal neuroergonomics study? Our team of experts can help you plan your research and identify the tools that will provide the best possible data.

Under the Skin: Digging Deeper into the Physiology of Fear

From the thrill of cinematic jump scares to debilitating phobias that can turn daily life for some into a waking nightmare, our sense of fear can feel like both a blessing and a curse. Fear is one of the most fundamental human emotions, serving as a necessary safeguard for recognizing and responding to danger. Despite its primal origins, the mechanisms of fear remain complex. For decades, psychophysiologists have studied fear using measures such as the startle reflex and skin conductance, building a deep understanding of how the body reacts to threats. Today, however, researchers are beginning to look beyond these traditional markers. Technologies such as virtual reality (VR) expand how researchers can stimulate the fear response. At the same time, a broad range of signals and techniques can measure not only the body’s autonomic responses, but also how people allocate attention to threats and how the brain processes fear-related information.

One reason fear remains such a compelling topic is that it sits at the intersection of cognition, emotion, memory, and physiology. Fear responses can be observed through changes in sweating, heart activity, breathing patterns, pupil size, neural activity, and even social behavior. As researchers seek to gain a better understanding of anxiety disorders, phobias, post-traumatic stress disorder (PTSD), and related conditions, they increasingly rely on multiple synchronized signals to capture a more complete picture of how fear is learned, remembered, and generalized.

Among the most widely used measures is electrodermal activity (EDA), often recorded as skin conductance response (SCR), which reflects changes in sympathetic nervous system arousal. More recently, eye tracking and pupillometry have emerged as valuable tools for measuring vigilance and attention, while functional near-infrared spectroscopy (fNIRS) has enabled monitoring of changes in cortical activity during fear-related tasks in more naturalistic settings. Together, these approaches help researchers move beyond viewing fear as a single physiological reaction and toward understanding it as a coordinated brain-body process.

A team of researchers at South China Normal University in China explored how the duration of memory retrieval influences the reconsolidation and extinction of fear memories. Using a four-day fear-conditioning paradigm, participants were conditioned to associate visual cues with mild electric shocks. Simultaneously, skin conductance responses were recorded using an MP36 running Biopac Student Lab System. The experiment was designed and implemented with E-Prime stimulus presentation software. The team found that briefly reactivating a fear memory before training participants to view the stimulus as safe was most effective at preventing the fear from returning. Longer periods of memory reactivation produced weaker effects. The study highlights how electrodermal activity can reveal subtle changes in fear-memory processing and may ultimately contribute to improved behavioral interventions for anxiety-related disorders.

A study conducted at the University of Würzburg in Germany examined how anxiety sensitivity influences attention and memory during threat and safety learning. Participants learned associations between neutral faces and aversive outcomes while researchers recorded both electroencephalography (EEG) and pupil diameter. Pupillometry data were acquired using a Tobii eye tracker, allowing researchers to measure moment-to-moment changes in attention. Individuals with higher anxiety sensitivity showed larger pupil responses and enhanced neural markers of attention when viewing both threatening and safe cues, reflecting a generalized pattern of hypervigilance. Interestingly, this heightened attentional engagement did not translate into better memory performance. The findings suggest that pupillometry provides a valuable window into how anxious individuals process potential threats and may help researchers better understand mechanisms underlying pathological anxiety.

Another perspective on fear comes from research at Sichuan Normal University in China, where investigators look at how trait anxiety influences observational fear learning. Unlike traditional conditioning studies, participants learned about danger by watching another person experience threat-related events. Researchers measured skin conductance using a BioNomadix wireless EDA plus data acquisition and analysis system and monitored prefrontal brain activity using fNIRS. The study found that individuals with high trait anxiety exhibited stronger physiological fear responses and increased medial prefrontal cortex activation, even when observing threats indirectly. The work demonstrates that fear can be acquired socially and that neural measures such as fNIRS can reveal aspects of fear processing that are not readily captured by autonomic measures alone. These findings may have important implications for understanding how fear and anxiety spread through social environments and how vulnerability to anxiety disorders develops.

These studies illustrate how fear research leverages multimodal approaches to incorporate multiple physiological perspectives. Skin conductance continues to provide valuable information about autonomic arousal, while pupillometry reveals how attention is divided between threat and safety cues, and fNIRS offers insight into the neural processes involved in fear learning and regulation. As multimodal research platforms become increasingly accessible, researchers are gaining unprecedented insight into how fear is acquired, maintained, and modified. By combining signals from the body, the eyes, and the brain, modern psychophysiology is shining a light on the shadowy labyrinth that is one of humanity’s most powerful and important emotions. To learn more about how to construct a fear stimulation lab for your next fear-conditioning research project, check out our webinar on the subject.

If you are planning a multimodal fear conditioning study, our staff can help identify the right equipment and software for stimulus presentation and data recording and analysis.

A System of Systems: Network Physiology Connects the Dots

In the field of physiology, it is tempting to see the human body as a collection of independent systems performing a range of functions—converting food to energy, protecting us from harm and disease, and helping us react to and interact with our environments, to name a few. It is easy to overlook that these systems often work together and are interconnected. The field of network physiology does just that—look at how these systems (nervous, cardiovascular, muscular, etc.) work together, seeing them as part of a dynamic network rather than a collection of independent parts.

The author of the article linked above, Boston University research professor of physics Plamen Ivanov, first coined the term “network physiology” in 2006. Through his work as director of the Keck Laboratory of Network Physiology, Ivanov and his team pioneered research in this new and rapidly growing field. The latest research methods, breakthroughs, and ideas in network physiology will take center stage this summer at the Fourth International Summer Institute on Network Physiology (ININP) hosted at the Lake Como School of Advanced Studies in Italy.

Researchers are exploring various ways to apply the concepts of network physiology, including how stress affects both the brain and the body. Medical researchers are looking into how to identify early warning signs of future health problems in how various physiological systems interact when developing treatments that address multiple bodily systems. While this may sound like the multimodal approach to physiology—collecting data simultaneously and synchronously from multiple systems—the two terms are not synonymous. Network physiology uses multimodal methods to collect data, but goes further by connecting the dots between the various systems.

A study by a team of researchers from the University of New York and the University of Houston demonstrated how combining data from different body systems can give a more complete picture of how the brain and body work together, specifically in response to music and mental effort. The study explored how different types of music—either calming or exciting—affect a person’s working memory (the ability to retain and use information for short periods) and overall brain and body arousal. During the experiment, participants completed memory tasks while listening to either calming or exciting music. Researchers collected detailed data from the body using a set of BIOPAC sensors and electrodes, including general-purpose electrodes for gathering electrocardiogram (ECG) signals, a respiratory effort transducer, a wireless skin temperature sensor, a wireless pulse transducer for photoplethysmogram (PPG) data, and electrodes to measure electrodermal activity (EDA), blood flow in the finger, and muscle activity in the shoulders. These multimodal signals were gathered using a BIOPAC data acquisition and analysis system. The data was collected at 2 kHz to capture even small changes in the body’s responses. The research team found that while the difficulty of the memory tasks didn’t significantly impact arousal levels, the type of music did. Exciting music helped participants feel more alert and even improved their performance on some tasks.

A 2024 South Korean study investigated how the autonomic nervous system (ANS) responds differently to social pain (such as emotional distress from losing a loved one) versus physical pain (like discomfort from a pressure cuff). Researchers used a BIOPAC 16-channel data acquisition and analysis system running AcqKnowledge software to collect a range of signals from 73 healthy young adult participants. The following signals were gathered synchronously using a variety of sensors and transducers: ECG electrodes placed on both wrists and the left ankle to monitor heart activity, EDA sensors attached to the index and middle fingers to assess skin conductance (indicating stress levels), PPG sensor on the thumb to measure blood volume changes, chest-wrapped sensor to track breathing patterns, and a temperature sensor on the ring finger to monitor peripheral temperature changes. Participants experienced social pain via a 60-second grief-themed film clip and physical pain through a blood pressure cuff inflated to 250 mmHg. Social pain increased heart rate and skin conductance, while lowering blood volume pulse, pulse transit time, breathing rate, and finger temperature, reflecting a mix of sympathetic and parasympathetic nervous system activity. Physical pain, on the other hand, raised heart rate variability and skin conductance, reduced blood volume pulse and pulse transit time, but left finger temperature unchanged, suggesting activation of the body’s stress response system. These distinct patterns show how different types of pain trigger unique responses across interconnected body systems.

A multimodal approach was employed in a study by researchers from several institutions including the University of Southern California, University of California, Irvine, Rutgers University, Vanderbilt University, and Wayne State University School of Medicine to examine how daily heart rate variability (HRV) biofeedback training influences brain function, emotional regulation, and physiological responses in both younger and older adults. In the study, 193 participants were randomly assigned to either increase or decrease their heart rate oscillations through daily biofeedback sessions over five weeks. Before and after the intervention, participants underwent various assessments, including MRI scans, psychological evaluations, and sleep monitoring using wristband sensors. During the MRI sessions, physiological data such as blood pressure, pulse, respiration (using a BIOPAC MRI-safe respiration transducer and module), and end-tidal CO₂ were continuously recorded. During stress tasks, blood pressure and hemodynamic measurements were taken using a noninvasive blood pressure monitor and transferred to a BIOPAC data acquisition and analysis system. The study’s findings suggest that HRV biofeedback can positively impact both brain function and physiological health, highlighting the interconnected nature of bodily systems, a central concept in network physiology.

These are just a few examples of how network physiology is being applied in research today. For more information on the topic, including links to related webinars, citations, and events, see our Network Physiology applications page.

Are you planning to integrate a range of signals for your next network physiology study? Our customer service specialists can help identify the best equipment and strategies for the synchronous collection of high-quality multimodal data.