The activity of the IPS (a sulcal structure between the superior and inferior parietal lobules in the parietal lobe, as shown in Fig. 2) was used as a key evaluation measure. According to the automatic anatomical labeling (AAL) system, the IPS is approximately in areas 63 to 66, while Brodmann’s map places it between areas 5 and 7 [17]. The IPS is a part of the parietal association cortex, posterior to the primary somatosensory cortex in the central sulcus and anterior to the occipital lobe, which contains the visual cortex. As a multimodal processing region, the IPS integrates sensory and visual information, particularly by relying on visual cues for controlling hand movements. In both hemispheres, the IPS is bilateral, with the left and right hemispheres predominantly controlling the right and left hands, respectively.

Positions of the IPS and measurement channels

To measure IPS activity, we used functional near-infrared spectroscopy (fNIRS; OEG-17APD system, Spectratech Inc., Tokyo, Japan), a noninvasive optical imaging technique using near-infrared light for measuring changes in regional cerebral blood flow. The fNIRS system operates by emitting near-infrared light into the scalp and detecting the amount of light absorbed by the hemoglobin in the blood. As oxygenated hemoglobin levels fluctuate in response to neural activation, these light absorption changes allow for the indirect measurement of the brain activity. The fNIRS system comprises light emitters and detectors arranged in a grid pattern, with each emitter–detector pair forming a measurement channel. These channels are also shown in Fig. 2. The fNIRS system specifically quantifies changes in oxygenated and deoxygenated hemoglobin levels in the brain region directly beneath the channels. Compared with other neuroimaging methods, fNIRS is relatively simple and cost effective while possessing minimal physical constraints, as it only requires attachment to the head without interfering with limb movements.

For brain activity measurements using fNIRS, the participants first wore a dedicated cap designed to securely attach the fNIRS sensors. The placement of the fNIRS probes was determined using a 3D digitizer (PATRIOT, Polhemus, Colchester, VT) and the PoTATo toolbox in MATLAB (MathWorks, Natick, USA), allowing for precise spatial mapping between the measurement positions and of the participants’ brain regions. The spatial coordinates were further analyzed using the 10–20 international system to estimate the correspondence between the sensor positions and underlying cortical structures.

We specifically targeted the IPS for measurements. The fNIRS cap was adjusted to ensure the optimal alignment with the right- and left-sided IPS regions. However, if the measurement positions and IPS did not precisely overlap because of individual anatomical variations, adjustments were made to capture the activity from the adjacent regions. Specifically, the measurement regions were shifted to encompass both the superior and inferior parietal lobules, which are functionally associated with the IPS and contribute to visuomotor control.

We also used the Unity game engine (Unity Technologies, San Francisco, USA) to develop a VR-based surgical simulator to investigate the effects of the delay on surgical operability. The simulator enables participants to perform surgical tasks while recording their IPS activities. The block diagram of the simulator is shown in Fig. 3. The operator controls the simulator using two hand controllers simultaneously. We employed one touch device (3D Systems, Rock Hill, USA) for each hand, allowing for six-degree-of-freedom input for the stylus position and orientation while also providing three-dimensional force feedback. When data were output from the personal computer (PC) to the monitor, simulated delay was implemented by introducing a fixed waiting time. The operator received visual feedback, including on the effects of their manipulations, through a 2D monitor. In VR, the robotic arm’s movements were updated based on the operator’s input. Additionally, force feedback was applied based on the motion of the input device and the interaction with virtual objects in the simulator. While haptic feedback is not a standard feature in current clinical robotic systems, force feedback is implemented to represent geometric constraints of the virtual robotic arms, such as touch with an organ. Without force feedback, the trajectory of movement depends on only the input controller, ignoring influence by touch and interference of surgical robot with organs in the working space. Therefore, force feedback was implemented to assist in conveying the movement limitations of the robotic arms within the VR simulator.

Block diagram of the VR simulator

Figure 4 shows the entire VR-simulated surgical environment and the correspondence between the hand controllers’ and robotic arms’ movements in the simulator, where the robotic arms, including two instrument arms and one camera arm each and the essential associated tools, were modeled based on the dimensions previously measured by Loi et al. [18] to replicate the da Vinci surgical robot. The positions and orientations of the instrument arms’ end effectors were associated with the positions and orientations of the styli on the touch hand controllers, respectively.

The simulator overview and the relationship between the touch controller and forceps position. The simulator includes two instrument arms and one camera arm, which dimensions were previously measured by Loi et al. [18]

We simulated a laparoscopic surgical environment as a representative case of RAMIS and replicated the suturing environment for colorectal resection surgery. The da Vinci surgical robot has been increasingly utilized in gastrointestinal surgery, particularly in procedures requiring intracorporeal suturing within a confined surgical space [19]. Among the various tasks involved in colorectal resection, such as suturing, tissue excision, and tissue manipulation, this study focused specifically on suturing because inadequate suturing can lead to complications, such as anastomotic leakage and bowel obstruction, potentially necessitating reoperation [20]. Furthermore, suturing requires fine bimanual coordination and is sensitive to delay-induced disruptions in timing and precision, making it a suitable task for evaluating the cognitive and operational impacts of communication delays. The reaction forces, both when the robot contacted the colon and when a needle was inserted into the colon during suturing, were calculated based on literature values and provided as three-dimensional force feedback to the 3D touch device.

Figure 5 shows the task procedure, involving two suturing operations. The target positions for needle insertion and exit were marked with visual indicators. Before starting the task, the operator set the 3D touch device at its initial position and kept it stationary during the preparation phase. During the task, the completion time and suturing accuracy, defined as the total positional deviation between the actual suturing and target points marked at four insertion and exit locations, were measured. The operator was instructed to complete the suturing as quickly and accurately as possible.

The experimental setting is shown in Fig. 6. We recruited eight right-handed participants (seven men and one woman) between the ages of 22 and 25, all of whom were non-medical students with no prior surgical experience. Before the main experiment, the participants practiced using the VR simulator and suturing tasks once or twice without delay to familiarize themselves with the system while minimizing learning effects. The participants then performed the VR simulation task while wearing an fNIRS measurement device on their heads. During the experiment, we randomly introduced seven delays (0, 50, 100, 150, 200, 250, and 300 ms) between the controller input and output video to prevent order effects. Each participant repeated the task five times per delay condition, and left- and right-sided IPS activities were separately recorded from task preparation to completion. Additionally, task performance metrics, such as the task completion time and suturing error rate, were measured. To mitigate fatigue effects, participants took a 5-min break after every five trials. After each break, they performed one additional practice round without delay before resuming the main task.

The brain activity data were analyzed using generalized linear modeling (GLM), which has been widely adopted in fNIRS research for evaluating cognitive loads. Suzuki et al. demonstrated that beta values derived from the GLM of block-designed fNIRS data reliably indicate cognitive-processing demand [21]. GLM-based fNIRS analysis has also been utilized in several other studies [22, 23], further supporting its applicability in assessing cognitive workloads during motor tasks.

Three GLM models were constructed, as shown in Fig. 7. Model 1 involved the convolution of the hemodynamic response function (HRF)—which models the temporal dynamics of the stimulated brain activation, allowing for the extraction of task-related neural responses—with the oxygenated hemoglobin level changes recorded during both the task preparation and execution phases. The data were analyzed using the spm12 toolbox in MATLAB. To account for potential baseline shifts in the fNIRS signal, Models 2 and 3 incorporated constant baseline offset and linearly increasing baseline drift functions, respectively. These additional models helped to control for physiological noise and other slow fluctuations unrelated to task-induced brain activity. The mathematical formula for the GLM analysis is as follows:

$$ \beginc} x_ \left( t \right) + \beta_ x_ \left( t \right) + \beta_ x_ \left( t \right) + e\left( t \right)} \\ \end $$

(1)

Models used in GLM analysis

Because of the GLM framework, the fNIRS signal (\(y(t)\)) was modeled as a linear combination of the explanatory variables (\(_\left(t\right), _\left(t\right)\), and \(_\left(t\right)\)), corresponding to Models 1, 2, and 3, respectively. Coefficients \(_, _\), and \(_\) were estimated to minimize the squared error (e(t)) and optimize the models’ fit to the measured data.

Among the coefficients, \(_\) is a key brain activity index, indicating the contribution of HRF (Model 1) to the overall response. A higher \(_\) value indicates stronger task-related neural activation, while \(_\) and \(_\) primarily account for baseline fluctuations and drift, respectively. To determine the IPS-specific brain activity, \(_\) values were assigned based on the overlap degree between the fNIRS measurement regions and the IPS’s anatomical location. If a measurement site directly coincided with the IPS, the corresponding \(_\) value was used. Where a measurement covered both the superior and inferior parietal lobules, the IPS-associated \(_\) value was computed as the average \(_\) value of both lobules, accounting for potential signal dispersion. This approach allowed for a quantitative evaluation of IPS activity across different experimental conditions, enabling an objective assessment of task-related cognitive processing at various delays.