We demonstrated the feasibility of real-time robotic control using finger-level MI (Fig. 1, Supplementary Fig. S1, Supplementary Movies S1, S2). Across 21 participants, the results of a two-way repeated measures Analysis of Variance (ANOVA) model indicated a significant improvement in MI performance across two online sessions for both binary (F = 14.455, p = 0.001, Fig. 1A) and ternary (F = 24.590, p < 0.001, Fig. 1B) paradigms. Performance enhancements were evident in the precision and recall metrics for individual classes, showing consistent improvements across all categories between sessions (Fig. 1C, Supplementary Fig. S1A). Furthermore, the feature representations learned by the deep learning decoder exhibited enhanced class discriminability with increased training size across sessions, as demonstrated by visualizations of intermediate-layer activations (Supplementary Fig. S2).Fig. 1

Group-level online performance for 2-finger (A) and 3-finger (B) MI across sessions (n = 21 subjects). Comparisons between the base and fine-tuned models for Sessions 1 and 2 were conducted using a two-tailed Wilcoxon signed-rank test with Bonferroni correction for multiple comparisons (*** if p < 0.001, * if p < 0.05). Two-way repeated-measures ANOVAs were applied with main effects of session and model, and comparisons between Sessions 1 and 2 were indicated by the main effect of session (### if p < 0.001, ## if p < 0.01). The red dashed line indicates the chance level, while the gray dashed line represents the median performance under each condition. The black dots represent every instance. P-values: 0.0002 (A, Session 1), 0.0122 (A, Session 2), 0.0011 (A, Session 1 vs. 2); 0.0271 (B, Session 1), 3.62e−05 (B, Session 2), 7.55e−05 (B, Session 1 vs. 2). Group-level precision of 2-finger and 3-finger MI decoding across sessions (C) and group-level recall of 2-finger and 3-finger MI decoding results for different classifiers (D) (n = 21 subjects). The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. Diamonds indicate outliers that are more than 1.5 times the interquartile range above the third quartile or below the first quartile. Comparisons between the precision across sessions and recall across models were conducted using a two-tailed Wilcoxon signed-rank test with Bonferroni correction for multiple comparisons (*** if p < 0.001, ** if p < 0.01, * if p < 0.05). P-values: 4.62e−08 (C, Thumb, 2-class), 0.0009 (C, Thumb, 3-class), 0.0395 (C, Index, 3-class), 0.0012 (C, Pinky, 3-class), 0.0412 (D, Thumb, 3-class), 0.0007 (D, Index, 3-class). E Group averaged alpha band (8−13 Hz) ERD topology for MI online control tasks. F Group-averaged saliency topological maps of the EEGNet models for online MI sessions. Channels with the highest 25% saliency values are highlighted by white circles on the electrodes.

The ANOVA also identified a significant main effect of model type for binary (F = 31.380, p < 0.001) and ternary (F = 22.443, p < 0.001) tasks. Post-hoc pairwise comparisons were performed to assess the efficacy of fine-tuning during each session on the performance outcomes. Bonferroni-corrected two-tailed Wilcoxon signed-rank test results indicated that fine-tuned models significantly outperformed base models in all online sessions and paradigms (p < 0.001 for Session 1 binary tasks and Session 2 ternary tasks, p < 0.05 for Session 2 binary tasks and Session 1 ternary tasks; Fig. 1A, B). Comparisons of intermediate-layer feature maps further indicated that fine-tuned models exhibited enhanced decoding capabilities relative to base models (Supplementary Fig. S2). This observation aligns with previous studies demonstrating that fine-tuning effectively enhances model capacity by adapting it to the target dataset’s distribution^51,52. Despite relatively balanced precision values across all categories using the base models (Supplementary Fig. S1B), imbalances were noted in the models’ ability to identify relevant instances of different classes (Supplementary Fig. S1C, D). Specifically, the thumb and the index finger in the 3-class paradigm had lower recall values when using the base models (Fig. 1D). Fine-tuning contributed to addressing these class imbalances by significantly enhancing the detection capability of the categories with low recall values, as confirmed by multiple-comparison corrected two-tailed Wilcoxon signed-rank test results (Fig. 1D). After one session of online training and model fine-tuning, 2-finger online MI robotic control can be achieved with an average accuracy of 80.56% across 21 subjects, and 3-finger online control can achieve the average accuracy of 60.61%.

The involvement of contralateral motor and premotor cortices during MI has been documented in many studies^53,54. To further elucidate this phenomenon, alpha band (8–13 Hz) event-related desynchronization (ERD) patterns were extracted from EEG data to analyze MI-induced electrophysiological activation. A significant power decrease was observed in the left sensorimotor region during MI tasks (Fig. 1E). Complementing these findings, saliency maps generated by EEGNet models, using the gradient of the output class probability relative to the input EEG signals, highlighted the hand knob area in the left primary motor cortex (Fig. 1F). These findings align with prior research identifying this region as critical for motor intention. The spatial concordance between the saliency topologies of the decoders and the ERD patterns reinforced the sensorimotor region’s role in the model’s predictions. To validate this observation, we conducted offline decoding experiments comparing performance across sensorimotor-specific channels, non-sensorimotor channels, and whole-scalp inputs. To control for the potential confounding effect of input dimensionality, the number of channels in the sensorimotor and non-sensorimotor regions was matched. Sensorimotor channels achieved decoding performance comparable to whole-scalp input, whereas non-sensorimotor channels showed significantly reduced accuracy (Supplementary Fig. S3A, C), reinforcing the critical role of sensorimotor regions in finger MI decoding. Further dissection of cortical contributions was achieved by isolating four subregions, each comprising 11 channels, including contralateral and ipsilateral hand knob areas, which are responsible for controlling the fine movements of fingers, occipital cortex, and frontal regions (Supplementary Fig. S4A). Among the subregions, the left and right hand knob areas yielded the highest decoding accuracies (Supplementary Fig. S4E), aligning with our electrophysiological analysis results and confirming the decoder’s dominant reliance on these regions. This convergence confirms the decoder’s primary reliance on the sensorimotor region during MI. However, the accuracy using only the contralateral hand knob as input was found to be significantly lower than 128-channel whole-scalp input (Supplementary Fig. S4C). This finding aligns with the saliency maps of the EEGNet models, which revealed a predominant focus on the hand knob area in the left primary motor cortex with additional emphasis in the right hand knob region and parietal channels (Fig. 1F). Together, the saliency maps and subregion analyses underscore the key role of the contralateral hand knob region in decoding finger movements, while also suggesting that distributed electrode configurations capture supplementary task-relevant signals.

To delineate the factors contributing to the observed performance improvements, offline simulations were conducted using EEG data collected during MI robotic control sessions. Data from fine-tuned runs in Session 2 were evaluated using three models: the Session 1 Base model (S1-B), the Session 1 Fine-tuned model (S1-F), and the Session 2 Base model (S2-B). Across sessions and models, significant or near-significant improvements were evident in binary tasks (S1-B vs. S1-F: p = 0.052; S2-B vs. S2-F: p < 0.001; S1-B vs. S2-B: p = 0.019; S1-F vs. S2-F: p < 0.001, Fig. 2A) and ternary tasks (S1-B vs. S1-F: p = 0.044; S2-B vs. S2-F: p = 0.004; S1-B vs. S2-B: p < 0.001; S1-F vs. S2-F: p < 0.001, Fig. 2A). The observed improvements were achieved using the same input data across compared models, ruling out data quality as confounding factors, and therefore highlighting the machine learning effects. The enhanced performance of fine-tuned models (S1-F, S2-F) over their base counterparts (S1-B, S2-B) demonstrates the effectiveness of machine learning-driven adaptation in refining feature extraction for session-specific data. Furthermore, the superior performance of Session 2 models (S2-B, S2-F) over Session 1 models (S1-B, S1-F) suggests enhanced decoding capacities facilitated by increased training sizes. To investigate the effects of human learning, offline decoding was performed using the Filter Bank Common Spatial Pattern (FBCSP) method paired with a Linear Discriminant Analysis (LDA) classifier⁵⁰. A two-way repeated-measures ANOVA revealed a significant main effect of the model in 3-class offline simulations (Fig. 2B). Significant performance improvement was observed during the first 3-class session when comparing the last eight fine-tuned runs with the first eight base runs (p < 0.01, Fig. 2B). A numerical enhancement in decoding performance was also noted in Session 2 compared to Session 1 for ternary tasks (Cohen’s d = 0.222, F = 3.065, p = 0.095). However, no clear improvement trend was observed in online decoding accuracy over eight continuous runs (Fig. 2C). These findings suggest an initial within-session human learning effect and cross-session human training for the 3-finger MI tasks.Fig. 2

A Offline simulation results (n = 21 subjects) for Session 2 fine-tuned runs using Session 1 base models (S1-B), Session 1 fine-tuned models (S1-F), and Session 2 base models (S2-B), as well as the online decoding results for Session 2 fine-tuned runs (S2-F). The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. Diamonds indicate outliers that are more than 1.5 times the interquartile range above the third quartile or below the first quartile. Comparisons between the sessions and the models were made using a two-tailed Wilcoxon signed-rank test with Bonferroni multi-comparison correction (*** if p < 0.001, ** if p < 0.01, * if p < 0.05, n.s. if no statistical significance is found). P-values: 0.0003 (S2-B vs. S2-F, 2-class), 0.0254 (S1-B vs. S2-B, 2-class), 0.0003 (S1-F vs. S2-F, 2-class), 0.0449 (S1-B vs. S1-F, 3-class), 0.0074 (S2-B vs. S2-F, 3-class), 0.0009 (S1-B vs. S2-B, 3-class), 0.0002 (S1-F vs. S2-F, 3-class). B Offline FBCSP decoding results on MI robotic finger control data (n = 21 subjects). S1-B, S1-F, S2-B, and S2-F denote decoding results for Session 1 base runs, Session 1 fine-tuned runs, Session 2 base runs, and Session 2 fine-tuned runs respectively. The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. Diamonds indicate outliers that are more than 1.5 times the interquartile range above the third quartile or below the first quartile. Statistical analysis was conducted using a two-way repeated-measures ANOVA with main effects of session and model. Main effect of the model: ### if p < 0.001, n.s. if no statistical significance is found. Post hoc comparisons between models within the same session were made using a two-tailed Wilcoxon signed-rank test with Bonferroni correction (*** if p < 0.001, ** if p < 0.01, * if p < 0.05, n.s. if no statistical significance is found). P-values: 0.0005 (model, 3-class), 0.0114 (S1, 3-class). C Group-averaged performance trends across eight consecutive online runs within the same paradigm. The center lines indicate the mean values. The shaded areas represent the standard errors.

The same group of participants (n = 21) also participated in one offline session and two online sessions of finger ME tasks, following the same experimental design as the MI tasks (Supplementary Movies S3, S4). Significant performance improvements were observed across sessions and between the fine-tuned and base models, as indicated by a two-way repeated-measures ANOVA (2-class session effect: F = 8.826, p = 0.007; 2-class model effect: F = 35.034, p < 0.001, 3-class session effect: F = 12.914, p = 0.001; 3-class model effect: F = 35.869, p < 0.001, Fig. 3A, B, Supplementary Fig. S5). Enhanced class separability was observed in the learned feature space across sessions, with fine-tuned models exhibiting clearer class distinction than their base model counterparts (Supplementary Fig. S6). Online training substantially increased precision and recall for thumb and pinky across sessions (Fig. 3C, Supplementary Fig. S5A). Offline simulation results comparing different models on the same dataset demonstrated enhanced decoding performance using Session 2 models compared to Session 1 models (2-class decoding: p(S1-B vs. S2-B) = 0.140; p(S1-F vs. S2-F) < 0.001; 3-class decoding: p(S1-B vs. S2-B) = 0.009; p(S1-F vs. S2-F) < 0.001, Supplementary Fig. S7A). This improvement highlighted the impact of increased training size on machine learning performance across sessions. Significant decoding improvements were observed when comparing fine-tuned models to base models within Session 2 (2-class decoding: p(S2-B vs. S2-F) < 0.001; 3-class decoding: p(S2-B vs. S2-F) = 0.001, Supplementary Fig. S7A), confirming the efficacy of fine-tuning in enhancing performance within the same session.Fig. 3

Group-level online performance for 2-finger (A) and 3-finger (B) ME across sessions (n = 21 subjects). Comparisons between the base and fine-tuned models for Sessions 1 and 2 were conducted using a two-tailed Wilcoxon signed-rank test with Bonferroni correction for multiple comparisons (*** if p < 0.001, ** if p < 0.01). Two-way repeated-measures ANOVAs were applied with main effects of session and model, and comparisons between Sessions 1 and 2 were indicated by the main effect of session (## if p < 0.01). The red dashed line indicates the chance level, while the gray dashed line represents the median performance under each condition. The black dots represent every instance. P-values: 0.0001 (A, Session 1), 0.0055 (A, Session 2), 0.0075 (A, Session 1 vs. 2); 0.0003 (B, Session 1), 0.0026 (B, Session 2), 0.0018 (B, Session 1 vs. 2). Group-level precision of 2-finger and 3-finger ME decoding across sessions (C) and group-level recall of 2-finger and 3-finger ME decoding results for different classifiers (D) (n = 21 subjects). The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. Diamonds indicate outliers that are more than 1.5 times the interquartile range above the third quartile or below the first quartile. Comparisons between the precision across sessions and recall across models were conducted using a two-tailed Wilcoxon signed-rank test with Bonferroni correction for multiple comparisons (*** if p < 0.001, * if p < 0.05). P-values: 0.0003 (C, Thumb, 2-class), 1.79e−05 (C, Pinky, 3-class), 0.0310 (D, Thumb, 2-class), 3.35e−05 (D, Index, 3-class). E Group averaged alpha band (8−13 Hz) ERD topology for ME online control tasks. F Group-averaged saliency topological maps of the EEGNet models for online ME sessions. Channels with the highest 25% saliency values are highlighted by white circles on the electrodes.

As with the MI paradigm, offline simulation results using the FBCSP-LDA model to decode EEG data from online sessions showed within-session performance improvements during the first 3-class session, but not under other conditions (Supplementary Fig. S7B). A significant session effect in the 3-class simulation results further suggested human learning involvement in the 3-class paradigm (Supplementary Fig. S7B). However, no consistent improvement in online decoding accuracy was observed across eight continuous runs within the same paradigm (Supplementary Fig. S7C).

Fine-tuning effectively mitigated the imbalances in the classification of different fingers by improving recall and precision for the most challenging classes (Fig. 3D, Supplementary Fig. S5B–D). With the fine-tuned model, 2-finger online ME robotic control was achieved with an average accuracy of 81.10% across 21 subjects, while the average accuracy for 3-finger control was 60.11%. Electrophysiological analysis showed a pronounced desynchronization in the alpha band power within the left sensorimotor region during right-hand finger movements, consistent with established neurophysiological patterns of motor planning and execution (Fig. 3E). Deep learning decoders similarly focused predominantly on the motor cortex (Fig. 3F, Supplementary Fig. S3B, S4D), aligning with known neurophysiological mechanisms.

Whereas significant improvement in online control performance across sessions has been observed in arm-level MI-based BCIs^11,55,56, to thoroughly investigate the learning effects of online MI robotic control at the individual finger level, 16 out of the 21 subjects completed a total of five MI online sessions. The base model was updated on a session-by-session base by training on all the available subject-specific data from previous MI sessions. The same metric, i.e. majority voting accuracy, was used for performance evaluation.

Two-way repeated-measures ANOVAs were conducted to evaluate the effects of session, model, and session-model interaction on performance. The results revealed significant learning effects across sessions for both 2-class and 3-class tasks (2-class: F = 7.127, p < 0.001; 3class: F = 14.406, p < 0.001, Fig. 4A, B), demonstrating noticeable performance improvements during online training. Additionally, a significant model effect indicated that fine-tuning consistently enhanced decoding accuracy as training sessions progressed (2-class: F = 35.606, p < 0.001; 3class: F = 39.138, p < 0.001, Fig. 4A, B). However, the session-model interaction effect was not significant, suggesting comparable learning speeds between the base and fine-tuned models (2-class: F = 1.375, p = 0.253; 3class: F = 1.686, p = 0.164).Fig. 4

Group-level trends in 2-finger (A) and 3-finger (B) MI accuracy over five online sessions (n = 16 subjects). The center lines indicate the mean values. The gray dashed line represents the chance level, and error bars indicate the standard errors. The performance distribution for the Base and Fine-tuned models across all sessions is shown on the right. The center lines indicate the median value. The boxes extend from the lower quartile to the upper quartile, and the lines indicate 1.5 times the interquartile range. Statistical analysis was conducted using a two-way repeated-measures ANOVA with main effects of session and model. Main effect of the model: ### if p < 0.001. P-values: 2.6e−05 (A), 1.53e−05 (B). Pairwise comparisons of 2-finger (C) and 3-finger (D) MI accuracy between consecutive sessions (n = 16 subjects). The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. The whiskers span up to 1.5 times the interquartile range. One-sided Wilcoxon signed-rank tests were performed for each session pair and model type. Statistical significance after Bonferroni correction is denoted as ***p < 0.001, **p < 0.01, *p < 0.05. P-values: 0.0260 (C, 2-1, Base), 0.0024 (D, 2-1, Base), 0.0012 (D, 2-1, Fine-tuned), 0.0103 (D, 4-3, Base). Linear regression analysis of the improvement in 2-finger (E) and 3-finger (F) MI accuracy achieved by the fine-tuned model compared to the Base model across five sessions. The center line indicates the linear regression model fit of the data. The shaded region indicates the 95% confidence interval for the regression.

To further investigate performance dynamics, comparisons between consecutive sessions were conducted using one-sided Wilcoxon signed-rank tests. These analyses revealed a noticeable boost in performance during the early stages of training, specifically between Session 2 and Session 1 (Fig. 4C, D). However, subsequent sessions showed no significant performance enhancements, despite numerical improvements (Fig. 4C, D). Specifically, one session of online training was sufficient for the EEGNet models to extract distinguishable features from scalp EEG signals, performing comparably to later training sessions. This highlights the efficiency of the training process. To evaluate whether the compact architecture of EEGNet-8,2, with only a few thousand trainable parameters, limited its capacity to leverage larger datasets, we performed offline analyses using a wider and deeper variant (deepEEGNet). This model increased the number of filters in each layer and incorporated two additional separable convolutional layers before the flatten layer (Supplementary Table S1). Despite modest numerical improvements in decoding accuracy (2-class: +1.21%; 3-class: +1.52%) compared to the simulated results using EEGNet, the overall learning trend remained consistent using deepEEGNet (Supplementary Fig. S8). For this comparison, deepEEGNet results were benchmarked against EEGNet’s simulated outcomes rather than online performance metrics to mitigate potential confounding effects of real-time feedback. This persistent plateau suggests that the observed trend may stem from inherent limitations in EEG-derived features for sophisticated finger movement decoding, rather than model capacity alone.

To quantify the effects of fine-tuning, linear regression analyses were performed on the accuracy differences between fine-tuned models and base models across training sessions. The results demonstrated a significant superiority of fine-tuned models as training progressed, reflected by significant positive offsets (2-class: t = 4.251, p < 0.001; 3class: t = 2.992, p = 0.004, Fig. 4E, F). However, the session effect was not significant (2-class: t = −1.558, p = 0.311; 3class: t = 0.286, p = 0.776, Fig. 4E, F), indicating that the decoding capability of the fine-tuned models remained consistently superior to that of the base models throughout training. These findings confirm the effectiveness of fine-tuning in mitigating the cross-session variability and improving the performance of deep-learning models across multiple sessions.

In addition to online behavioral results, offline decoding and electrophysiological analysis were conducted to examine the training effects on the human side. Offline decoding of single-session EEG data using an FBCSP-LDA model revealed no improvement in signal discriminability for binary tasks (Supplementary Fig. S9A). However, for ternary tasks, significant improvements were observed in fine-tuned runs compared to base runs, as indicated by a two-way repeated-measures ANOVA (F = 15.028, p = 0.001, Supplementary Fig. S9B), and the enhancement in the data discriminability mainly occurs in the first session (Cohen’s d = 0.502 for Session 1, Supplementary Fig. S9B). This indicates a plateau in human MI learning at the early stages of training. One-way ANOVA analysis of the EEG data collected during the online training sessions showed that ERD levels at channel C3, which is believed to produce the strongest neural signals related to right-hand MI, remained consistent throughout five training sessions (2-class: F = 0.432, p = 0.784; 3-class: F = 0.284, p = 0.887, Supplementary Fig. S10). These findings suggest that the proposed paradigm requires minimal training to achieve satisfactory performance.

Despite the strong performances achieved by deep-learning-based decoders in finger-level robotic control tasks, this sophisticated control paradigm remains challenging. The primary difficulties stem from the low signal-to-noise ratio and limited spatial resolution of scalp EEG, particularly in ternary classification tasks with an increased number of categories⁵⁷. To further enhance robotic control performance, we developed an online smoothing approach aimed at stabilizing robotic movements by incorporating previous decoding outputs into the decision-making process (Supplementary Movies S3, S4). Out of the 21 subjects studied, 16 participated in one ME online session and one MI online session, with each session comprising four runs using the original control mechanism and four runs using the smoothing mechanism. In addition to accuracy, we evaluated the efficacy of the smoothing approach using two additional metrics: Label Shift, defined as the number of changes in the continuously predicted results within one trial, and All-Hit Ratio, calculated as the percentage of trials in which all predictions matched the true label.

Separate linear mixed-effect models were constructed for each metric across the 2-finger ME paradigm (Fig. 5A), 3-finger ME paradigm (Fig. 5B), 2-finger MI paradigm (Fig. 5C), and 3-finger MI paradigm (Fig. 5D). No significant differences were observed between the smoothed outputs and the original ones in terms of accuracy, which suggests that the smoothing approach maintained task performance across these paradigms. Furthermore, in all four paradigms, we observed a significant reduction in the number of label shifts, reflecting enhanced stability in the output control commands. Comparing the percentage of All-Hit trials before and after applying the smoothing approach, we found that it effectively filtered out small perturbations during the tasks, leading to better overall control (Fig. 5E, F). These findings underscore the potential of the smoothing approach to improve the robustness and reliability of EEG-based robotic control systems.Fig. 5

Comparison between the original outputs from the fine-tuned EEGNet model and the smoothed outputs for 2-finger ME (A), 3-finger ME (B), 2-finger MI (C), and 3-finger MI (D) online tasks (n = 16 subjects). Accuracy is calculated using the majority vote method; Label Shifts represents the average number of shifts in predicted labels within one single trial, which quantifies how stable the decoder is; All-Hit Ratio is defined as the percentage of the trials that are classified correctly throughout the whole trial. The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. Diamonds indicate outliers that are more than 1.5 times the interquartile range above the third quartile or below the first quartile. Comparisons between the two conditions were made using a linear mixed effects model (*** if p < 0.001, ** if p < 0.01, * if p < 0.05). No p-value adjustments were made. P-values: 7.66e−13 (A, Label Shifts), 7.97e−06 (A, All-Hit Ratio), 2.16e−28 (B, Label Shifts), 2.78e−10 (B, All-Hit Ratio), 3.43e−12 (C, Label Shifts), 9.04e−07 (C, All-Hit Ratio), 3.49e−33 (D, Label Shifts), 9.77e−16 (D, All-Hit Ratio). Comparison between the original and the smoothed probabilities for the target class of one example 2-finger trial (top) and one example 3-finger trial (bottom) for ME (E) and MI (F) tasks. The black dashed line represents the probability at the chance level.

The decoding results from offline sessions provided valuable insights into the discriminability of different finger groups, guiding the design of online experimental paradigms. Subjects were instructed to perform either ME or MI of the thumb, index finger, middle finger, and pinky of their dominant hand (right hand). Offline decoding was performed on all possible 2-finger, 3-finger, and 4-finger combinations using EEGNet and FBCSP.

Among all the 2-finger groups, the thumb–pinky pair demonstrated the highest decoding performance across paradigms and classifiers, achieving an average accuracy of 77.58% for the MI task and 75.65% for the ME task using EEGNet. Conversely, the index finger and the middle finger combination exhibited the least discriminability (Fig. 6A, B). To explore the electrophysiological basis for the variation in discriminability across finger pairs, the distances between EEG channels showing maximum ERD activation for each pair were computed and compared. The thumb–pinky and index–pinky pairs exhibited the greatest distance in MI-induced ERD activation within the alpha band, aligning with the offline decoding results and somatotopic mapping (mean distance: 46.316 mm for thumb–pinky and 42.262 mm for index–pinky, Supplementary Fig. S11). For 3-finger combinations, the thumb-index-pinky and thumb-middle-pinky groups showed the highest accuracy, as reflected in the averaged results. In the 4-finger classification, an average accuracy of 43.61% for the MI task and 42.17% for the ME task was obtained. These offline decoding results demonstrate the enhanced performance of EEGNet in finger-level EEG-BCI decoding compared to non-deep-learning decoders like FBCSP, underscoring the potential of deep-learning-based methods in accurately distinguishing between various finger movements or motor intentions.Fig. 6

Group-level offline decoding performance of finger ME (A) and MI (B) using EEGNet and FBCSP (n = 21 subjects), where 1 represents thumb, 2 represents index finger, 3 represents middle finger, and 4 refers to pinky. The red dashed lines indicate the chance levels. The center lines indicate the median values. The boxes extend from the lower quartile to the upper quartile. The whiskers span up to 1.5 times the interquartile range. Comparisons between two decoders were conducted using a two-tailed Wilcoxon signed-rank test with Bonferroni correction for multiple comparisons. P-values: 0.0011 (A, 1-2), 0.0003 (A, 1-3), 0.0006 (A, 1-4), 0.0132 (A, 2-3), 0.0308 (A, 2-4), 0.0009 (A, 3-4), 3.81e−06 (A, 1-2-3), 0.0011 (A, 1-2-4), 0.0002 (A, 1-3-4), 2.67e-05 (A, 2-3-4), 3.81e-06 (A, 1-2-3-4), 0.0054 (B, 1-2), 0.0001 (B, 1-3), 5.72e−06 (B, 1-4), 0.0009 (B, 2-3), 0.0011 (B, 2-4), 1.14e−05 (B, 3-4), 0.0003 (B, 1-2-3), 0.0002 (B, 1-2-4), 1.14e-05 (B, 1-3-4), 0.0005 (B, 2-3-4), 0.0002 (B, 1-2-3-4). Group-level task-specific alpha band (8−13 Hz) (C) and beta band (13–30 Hz) (D) ERD topographies (n = 21 subjects). From left to right, ERD topographies corresponding to thumb, index, middle, and pinky movements are displayed. The top row presents results from ME data, while the bottom row presents results from MI data. Comparison of offline EEGNet decoding performance (n = 21 subjects) for ME (E) and MI (F) tasks using EEG signals filtered with different bandpass settings (4–40 Hz, alpha band, beta band, delta band, theta band). The x-axis shows classification results for different frequency bands. Offline classifications were performed on thumb vs. pinky (1-4), thumb vs. index finger vs. pinky (1-2-4), and all four fingers (1-2-3-4). Two-way ANOVA was conducted across different frequency bands and finger pairs, and statistical significance was observed in all task conditions. Significance stars indicate post hoc pairwise comparison results using an FDR-corrected two-tailed Wilcoxon signed-rank test (*** if p < 0.001, ** if p < 0.01, * if p < 0.05). P-values: 0.0181 (E, 4–40 Hz vs. Alpha), 9.53e−06 (E, 4–40 Hz vs. Beta), 4.45e−05 (E, 4–40 Hz vs. Delta), 4.45e−05 (E, 4–40 Hz vs. Theta), 0.0018 (E, Alpha vs. Beta), 0.0020 (E, Alpha vs. Delta), 0.0023 (E, Alpha vs. Theta), 0.0197 (E, Beta vs. Delta), 0.0329 (F, 4–40 Hz vs. Alpha), 3.17e−06 (F, 4–40 Hz vs. Beta), 3.17e−06 (F, 4–40 Hz vs. Delta), 1.19e−05 (F, 4–40 Hz vs. Theta), 0.0002 (F, Alpha vs. Beta), 3.17e−06 (F, Alpha vs. Delta), 4.76e−05 (F, Alpha vs. Theta), 8.74e−05 (F, Beta vs. Delta).

The suppression of oscillatory activities within the alpha (8–13 Hz) and beta (13–30 Hz) frequency bands in the contralateral sensorimotor region during both ME and MI tasks has been widely discussed^58,59. Previous studies have shown that alpha and beta frequency components offer effective discriminations between limb-level MI tasks^10,11,18,60. To explore the frequency-specific information encoded in EEG during finger ME and MI tasks, we extracted alpha and beta frequency components from offline EEG recordings. Task-specific ERD topologies revealed highly similar spatial patterns in both alpha and beta bands when subjects performed or imagined movements of different fingers (Fig. 6C, D). During ME tasks, a bilateral power decrease in both frequency bands was observed over the sensorimotor region, with the strongest suppression on the contralateral side. In contrast, MI tasks exhibited a more localized ERD in the left sensorimotor region across both frequency bands. When comparing the modulation of EEG oscillations induced by ME and MI tasks, we observed stronger bilateral desynchronization during ME, consistent with previous findings^59,61–63. Additionally, power modulations in the alpha and beta bands were also observed in parietal regions.

Since the analysis focused on the BCI responders after two rounds of screening based on their ME and MI performance, which might influence the group-level alpha and beta band ERD trends, we conducted analyses comparing ERD activations between qualified participants and non-responders excluded for unsatisfactory performance. Among participants excluded for poor ME performance (n = 13), alpha and beta ERD in the contralateral sensorimotor region remained detectable but markedly weaker than in responders (Supplementary Fig. S12A, B, F). For individuals excluded due to inadequate MI performance despite intact ME ability (n = 5), contralateral ERD was evident during ME tasks but absent during MI (Supplementary Fig. S12C, D). Across both responders and non-responders, MI tasks consistently elicited weaker alpha and beta ERD than ME tasks (Supplementary Fig. S12E), supported by medium-to-large effect sizes across groups. Non-responders also exhibited reduced ERD magnitudes compared to responders in both ME and MI tasks (Supplementary Fig. S12F), indicating their inability to generate robust sensorimotor cortical activation. These findings demonstrate that the attenuated ERD during MI relative to ME in alpha and beta bands is a consistent feature across both groups.

Beyond the alpha and beta rhythms, movement-related cortical potentials (MRCPs) in the low-frequency band (0.3–3 Hz) have been frequently used to differentiate hand and finger movements^39,64,65. To assess the contribution of these slow components, we visualized time-resolved amplitude topoplots and MRCP waveforms at the selected channel (C3) (Supplementary Fig. S13). All four fingers elicited significant changes in low-frequency EEG amplitudes within the first 1.5 seconds following trial onset (Supplementary Fig. S13A, B). The amplitude modulation began in the contralateral fronto-central regions and spread to the contralateral parietal areas over time, in both ME and MI conditions. No significant changes were observed prior to the trial onset or after 1.5 seconds. Compared to MI, ME elicited stronger amplitude changes. MRCPs were present in both finger ME and MI, characterized by a negative deflection in the electrode shortly after trial onset (Supplementary Fig. S13C, D). Although finger-specific differences were observed during a brief window around the MRCP deflection, low-frequency activity across individual fingers showed considerable overlap throughout the trial.

To assess the discriminability of individual finger movements across frequency bands, we filtered the EEG data into delta (0.5–4 Hz), theta (4–8 Hz), alpha, and beta bands. Each was separately input into EEGNet, and the resulting decoding accuracies were compared with those obtained from broadband (4–40 Hz) data. Decoding performance significantly declined when using individual frequency bands compared to the 4–40 Hz input, highlighting the contribution of broadband information in decoding finger movements (Fig. 6E, F). Among the single-band conditions, the alpha band yielded significantly higher accuracy than the delta, theta, and beta bands, while the beta band significantly outperformed the delta band. To further examine the role of low-frequency components, we expanded the broadband range to 0.3–40 Hz to include delta activity. The decoding performance of this extended broadband input remained superior to that of individual bands, and was comparable to that of the alpha band (Supplementary Fig. S14). These findings highlight the pivotal role of broadband EEG signals and specifically underscore the critical role of alpha and beta oscillations in distinguishing individual finger movements.

Online processing and classification were performed using custom Python (3.8.4) scripts made for BCPy2000 (2021.1.0), a part of the BCI2000 program⁹². The 128-channel EEG signals were re-referenced to the common average, downsampled to 100 Hz, and bandpass filtered between 4 and 40 Hz using a fourth-order Butterworth filter. The most recent 1-second segment of data was then z-scored and input to the online decoder. The decoding outputs were received by a custom C++ script for robotic finger movement control, with updates occurring every 125 ms. The finger with the highest current probability flexed its four joints by 0.1 rad within the 125 ms duration, providing visual feedback. At the end of each trial, the finger that flexed the most was considered the predicted class for that trial. All the fingers were reset to their initial position before the next trial started.

EEGNet-8,2, known for its effectiveness in EEG-BCI decoding, was employed in this study for online decoding⁴⁶. The base model used for online decoding during the first half of each session was pre-trained for 300 epochs on data collected in previous sessions and updated for each session. Early stopping and a learning rate scheduler were employed during the training process. The fine-tuned model used in the latter half of each session was further trained on data from the first eight runs collected on the same day. During the fine-tuning process, the parameters of the first four layers, including the temporal convolution layer and the spatial depthwise convolution layer, were fixed, while the parameters in the remaining layers were optimized based on the same-day data.

Evaluation metrics for the online decoding tasks include accuracy, precision, and recall. Accuracy is defined as the proportion of correct predictions compared to the total number of trials, where for each trial the final prediction is the one that receives the most votes from the continuous classifier outputs over multiple segments of the trial⁴⁹. In multi-class decoding, the precision and recall are calculated for each class. Precision for a given class is the ratio of correctly classified instances of that class to the total number of instances predicted as that class by the classifier. Recall is calculated as the proportion of correct predictions to the total number of instances that actually belong to the class in the dataset.

A saliency map, which visualizes the most prominent input data that influence the model’s predictions, is used to understand and interpret the inner characteristics of neural network in the finger ME and MI decoding tasks. The saliency topological map for each model is computed by summing up the gradient of the predictions with respect to the input data along the time dimension, as shown in Eq. (2).2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{array}{c}{{{\boldsymbol{X}}}}=\left[{{{{\boldsymbol{x}}}}}_{{{{\bf{1}}}}},{{{{\boldsymbol{x}}}}}_{{{{\bf{2}}}}},\ldots,{{{{\boldsymbol{x}}}}}_{{{{\boldsymbol{n}}}}}\right]\\ {{{\boldsymbol{G}}}}=\frac{d{{{\boldsymbol{P}}}}}{d{{{\boldsymbol{X}}}}}\\ \begin{array}{c}{{{{\boldsymbol{S}}}}}_{{{{\boldsymbol{t}}}}}=|{{{\rm{mean}}}}\left({{{\boldsymbol{G}}}}\right)|\\ {{{\boldsymbol{S}}}}={\sum }_{t=0}^{T}{{{{\boldsymbol{S}}}}}_{{{{\boldsymbol{t}}}}}\end{array}\end{array}$$\end{document}X=x1,x2,…,xnG=dPdXSt=∣meanG∣S=∑t=0TStwhere \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\boldsymbol{x}}}}}_{{{{\boldsymbol{i}}}}}$$\end{document}xi denotes a (C x T) matrix representing a single input, with C being the number of EEG channels and T being the number of time samples in an input segment. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\boldsymbol{X}}}}$$\end{document}X is a (1 x N) vector representing an input batch, where N indicates the number of instances in the batch. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\boldsymbol{P}}}}$$\end{document}P is a (N x Z) vector representing the predictions where Z is the number of classes. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\boldsymbol{S}}}}}_{{{{\boldsymbol{t}}}}}$$\end{document}St with a dimension of (C x T) is the averaged saliency map by taking the mean of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\boldsymbol{G}}}}$$\end{document}G, the gradiant of the predictions with respect to the input data, across the batch dimension. The absolute value is taken to represent the magnitude of the gradient. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\boldsymbol{S}}}}$$\end{document}S, the spatial saliency map with a dimension of (C x 1), is obtained by summing up \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{{\boldsymbol{S}}}}}_{{{{\boldsymbol{t}}}}}$$\end{document}St along the time dimension. To show the relative importance of each channel at a group level, outliers in the saliency topological map that deviate from the mean value by greater than 2 standard deviations are removed. The Results section presents the average of the normalized saliency topological maps.

Statistical analysis was performed using custom Python (3.10.13) scripts. For the BCI behavioral results, accuracies computed by comparing the majority vote prediction and the target finger for each trial were analyzed and compared across different conditions. Two-way repeated-measures ANOVAs were applied with main effects of session and model on the online ME and MI robotic sessions. Pairwise post hoc comparisons were performed using Wilcoxon signed-rank tests with False Discovery Rate (FDR) correction if the p value of the main effect was less than 0.05. Unless otherwise stated, Wilcoxon signed-rank tests were used to compare performance metrics, including accuracy, precision, and recall, as well as electrophysiological quantifications between two different conditions, and one-way ANOVAs were implemented to test the difference in behavioral and electrophysiological metrics under multiple conditions. Bonferroni correction was used for multiple comparisons. Two-way repeated-measures ANOVAs were employed to analyze the effects of session, model, and session-model interaction on online performance and simulation results with continued MI training. Comparison between the smoothed and original results were analyzed with linear mixed-effect models. Experimental conditions (Original versus Smoothed) were modeled as fixed effects and subjects were considered random effects.

Every experiment involving human participants have been carried out following a protocol approved by the Institutional Review Board of Carnegie Mellon University. Each participant gave informed written consent.

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.