DECODING COGNITIVE LOAD: A COMPARISON OF STANDARD MACHINE LEARNING APPROACHES FOR EEG DATA ANALYSIS OF MENTALLY DEMANDING TASKS

УДК 004.852

Abstract

Reliable cognitive-load estimation is essential for adaptive human-machine systems, but EEG-based decoding remains difficult when models must generalize to unseen participants. This study benchmarks standard machine-learning approaches for subject-independent three-class cognitive-load decoding. Using the public COG-BCI database with EEG recordings from 29 participants, N-back and Multi-Attribute Task Battery II conditions were mapped onto low-, medium-, and high-load classes. Two feature-based ensemble models, XGBoost and Random Forest, were compared with two covariance-based Riemannian approaches, Tangent-space Logistic Regression and Minimum Distance to Mean. All models were evaluated using leave-one-subject-out cross-validation with nested group-aware hyperparameter selection. XGBoost obtained the highest average score among the tested baselines, with mean balanced accuracy of 0.449, macro F1-score of 0.433, and AUC of 0.636. However, this performance represents weak-to-moderate subject-independent generalization, not a practically reliable decoding solution. The medium-load class was the most difficult to classify, and strong subject-wise variability was observed. Because task conditions from N-back and MATB-II were combined into shared workload labels, the results should be interpreted as task-conditioned workload decoding. The main contribution is a realistic benchmark showing the limits of standard EEG machine-learning pipelines under strict cross-subject evaluation.

Аннотация

Надежная оценка когнитивной нагрузки важна для адаптивных человеко-машинных систем, однако декодирование когнитивной нагрузки по ЭЭГ остается сложной задачей при необходимости обобщения на новых участников. В данной работе проводится сравнение стандартных подходов машинного обучения для субъект-независимой трехклассовой классификации когнитивной нагрузки. Использован открытый набор данных COG-BCI, включающий ЭЭГ-записи 29 участников. Условия N-back и Multi-Attribute Task Battery II были сопоставлены с низкой, средней и высокой когнитивной нагрузкой. Две ансамблевые модели на основе признаков, XGBoost и Random Forest, сравнивались с двумя ковариационными римановыми подходами: Tangent-space Logistic Regression и Minimum Distance to Mean. Все модели оценивались с помощью leave-one-subject-out кросс-валидации и вложенного группового подбора гиперпараметров. Наилучший средний результат среди протестированных базовых моделей показал XGBoost: сбалансированная точность 0.449, macro F1-score 0.433 и AUC 0.636. Однако такая производительность указывает лишь на слабое или умеренное субъект-независимое обобщение, а не на практически надежное решение для декодирования. Средний класс нагрузки оказался наиболее трудным для классификации, также была выявлена выраженная вариативность между участниками. Поскольку условия N-back и MATB-II были объединены в общие метки нагрузки, результаты следует интерпретировать как задачно-обусловленное декодирование рабочей нагрузки, а не как декодирование чистой латентной когнитивной нагрузки. Основной вклад работы заключается в реалистичной оценке ограничений стандартных ЭЭГ-пайплайнов при строгой межсубъектной проверке.

Keywords: cognitive load, EEG, machine learning, LOSO cross-validation, SHAP, Riemannian geometry, passive BCI.

Ключевые слова: когнитивная нагрузка, ЭЭГ, машинное обучение, LOSO-кросс-валидация, SHAP, риманова геометрия, пассивный нейроинтерфейс.

Introduction

Cognitive load, often discussed as mental workload (MWL), is not only a measure of task difficulty, but a measure of fit between what a task demands and what a person can cognitively sustain [1], [2]. This distinction is important in modern human–machine systems, where performance often depends less on physical effort than on continuous monitoring, rapid prioritization, and decision-making under uncertainty [3], [4]. When cognitive demands exceed available resources, attention narrows, working memory becomes strained, and errors become more likely [2], [5]. This problem is especially relevant in safety-critical and high-responsibility settings, where a delayed or incorrect decision may have serious consequences [6], [7], [8]. Very low workload can also be harmful, because prolonged underload may reduce vigilance and engagement [5], [8]. Therefore, cognitive-load estimation is useful not merely as a descriptive measure, but as a basis for adaptive systems that respond to the operator’s current functional state.

Cognitive load has traditionally been assessed using subjective ratings and behavioral performance measures. Instruments such as the NASA Task Load Index (NASA-TLX), the Subjective Workload Assessment Technique (SWAT), and the Rating Scale Mental Effort (RSME) remain widely used because they are simple, inexpensive, and easy to interpret [2], [8], [9]. Behavioral indices, including reaction time and accuracy, can also reveal whether increasing task demand affects performance [2]. Nevertheless, these methods are limited when the goal is continuous monitoring. Subjective ratings are usually retrospective and may interrupt the task if collected repeatedly, while behavioral measures are indirect and can reflect motivation, fatigue, task strategy, or prior experience rather than workload alone [5], [8]. This motivates the use of physiological and neurophysiological measures, which can provide more continuous information about the user’s state during task performance.

Electroencephalography (EEG) is particularly relevant in this context because it is non-invasive, relatively portable, and has high temporal resolution [10], [11]. These properties make EEG suitable for passive brain–computer interfaces (pBCIs), where brain activity is used not to issue deliberate commands, but to infer implicit states such as workload, vigilance, fatigue, or engagement [12]. In principle, such information could support neuroadaptive systems that adjust interaction demands before the operator becomes overloaded or disengaged. In practice, EEG-based decoding remains difficult [3]. EEG signals are weak, artifact-prone, non-stationary, and strongly affected by inter-subject and inter-session variability [10], [13], [14], [15], [16]. As a result, cognitive-load decoding cannot rely on a single universal marker. It requires machine-learning pipelines that can combine multiple EEG features while still being tested under realistic generalization conditions [10], [17].

EEG-based cognitive-load research commonly focuses on spectral power changes in canonical frequency bands, especially theta and alpha activity [5], [18], [19]. Increased frontal theta power has often been associated with working-memory demand and executive control [2], [5], [6], while changes in posterior alpha power are frequently linked to attentional engagement and the reduction of cortical idling [2], [20]. Other measures, including beta activity, gamma activity, and band-power ratios such as theta/alpha, have also been investigated as potential indicators of mental effort [8], [9]. However, these EEG correlates should not be treated as universal biomarkers. Their direction and magnitude can vary across tasks, participants, recording sessions, and cognitive strategies [17], [21]. For this reason, cognitive-load decoding usually benefits from multi-domain feature representations that combine spectral information with time-domain statistics, signal-complexity measures, and Hjorth parameters, which summarize complementary aspects of EEG amplitude, frequency dynamics, and waveform complexity [10], [22].

The variability of EEG workload markers has motivated the use of machine-learning approaches that can combine multiple weak and task-dependent features [10], [17], [23]. Deep-learning models, including convolutional and recurrent architectures, can learn representations directly from raw or minimally processed EEG, but they often require larger datasets, greater computational resources, and careful control against overfitting [3], [4]. Classical machine-learning models therefore remain important baselines, especially when the dataset size is moderate and the feature representation is explicitly engineered [10], [18]. Tree-based ensemble methods such as Random Forest and XGBoost are well suited to heterogeneous feature sets containing spectral, temporal, regional, and channel-level descriptors [11], [24]. In parallel, Riemannian approaches provide a different representation by operating on covariance matrices of multichannel EEG signals, allowing classifiers such as Minimum Distance to Mean and Tangent-space Logistic Regression to use spatial covariance structure rather than handcrafted scalar features alone [18], [25], [26].

Evaluation in EEG-based cognitive-load decoding depends strongly on the task paradigm and label structure. Many studies use controlled working-memory tasks, such as N-back or Sternberg-type paradigms, whereas others use operational multitasking environments such as the Multi-Attribute Task Battery [17], [21]. Binary low-versus-high classification is common and often easier to interpret, but it provides only a coarse distinction between workload states. Multi-level classification may be more relevant for adaptive systems because it can represent intermediate workload conditions, not only the extremes [17], [27]. At the same time, three-class decoding is more difficult because the medium class may not correspond to a stable and clearly separable neurophysiological state [13], [15]. This makes task design, label construction, and validation strategy central to the interpretation of model performance.

A major obstacle for EEG-based cognitive-load decoding is generalization to unseen participants. High accuracies can be obtained when training and test sets contain epochs from the same subjects, but this evaluation can overestimate performance because EEG segments from one individual tend to share subject-specific structure [17], [28], [29]. In that case, a classifier may partly learn stable properties of the participant, such as anatomy, electrode placement, baseline activity, or recurring artifacts, rather than workload-related patterns that transfer across users [15], [24]. This is especially problematic for practical systems, where a model is expected to work for new operators with different neural and behavioral profiles [23]. Subject-independent validation is therefore necessary. Leave-one-subject-out (LOSO) cross-validation provides a stricter test by holding out all data from one participant during training and evaluating the model only on that unseen subject [29].

These issues make direct comparison across studies difficult. Reported performance can change substantially depending on the dataset, task paradigm, workload labels, preprocessing pipeline, feature representation, classifier, and validation scheme [15], [17], [23], [30], [31]. As a result, high accuracy in one setting does not necessarily imply robust cognitive-load decoding in another. There is therefore value in comparing practical model families under a common preprocessing pipeline, a shared label structure, and a subject-independent evaluation design. Such a comparison is especially useful when it includes both feature-based models, which rely on engineered EEG descriptors [11], [24], and covariance-based models, which represent multichannel EEG through spatial covariance structure [18], [25], [26].

The present study evaluates whether three levels of cognitive load can be decoded from EEG under this stricter subject-independent setting. Using the public COG-BCI database, N-back and Multi-Attribute Task Battery II (MATB-II) difficulty conditions were mapped onto low-, medium-, and high-load classes. Two feature-based ensemble models, XGBoost and Random Forest, were compared with two covariance-based Riemannian approaches, Tangent-space Logistic Regression and Minimum Distance to Mean. All models were evaluated using LOSO cross-validation, with nested group-aware model selection where applicable. In addition to epoch-level performance metrics, the analysis examined confusion patterns, subject-wise variability, task-level probability estimates, statistical differences between models, and SHAP-based feature importance for the best-performing classifier.

The goal was not to demonstrate practical deployment-ready decoding, but to compare standard EEG machine-learning baselines under a strict subject-independent setting and to assess how model performance, class ambiguity, subject variability, and feature importance constrain three-class workload decoding.

Materials and Methods

This study evaluated whether cognitive load can be decoded from EEG under a subject-independent setting. The methodological pipeline consisted of five main stages: selection of a public EEG cognitive-load dataset, EEG preprocessing, epoch-level feature extraction, classification using both conventional machine-learning and covariance-based models, and statistical evaluation under LOSO cross-validation. To reduce information leakage, fold-specific machine-learning transformations and model-selection steps, including imputation, scaling, supervised feature selection, hyperparameter tuning, and classifier fitting, were performed only on the training subjects within each cross-validation fold.

Dataset

We used the COG-BCI database, a publicly available EEG dataset developed for passive brain–computer interface applications [32]. The dataset contains EEG recordings from 29 participants collected across three sessions separated by approximately one week. Participants completed several cognitive tasks, including the N-back task, the MATB-II, the Psychomotor Vigilance Task (PVT), and the Flanker task. EEG was recorded using a 64-channel active Ag/AgCl electrode system arranged according to the extended 10–20 system at a sampling rate of 500Hz. The dataset also includes subjective measures such as the Rating Scale Mental Effort (RSME) and the Karolinska Sleepiness Scale (KSS), as well as behavioral performance measures including reaction times and accuracy [32].

The present study focused on the N-back and MATB-II tasks because both provide graded task-difficulty levels suitable for three-class cognitive-load classification. The N-back task included 0-back, 1-back, and 2-back conditions, representing increasing working-memory demands [33]. The MATB-II task included Easy, Medium, and Difficult conditions, designed to manipulate multitasking demand and mental workload in a simulated operational environment [34]. These task conditions were mapped onto three cognitive-load classes: low load, medium load, and high load. This mapping follows the structure of the dataset and enables a shared three-class comparison, but it also means that the labels may contain both workload-related and task-specific information. Therefore, the classification target should be interpreted as task-conditioned workload level rather than a pure latent cognitive-load variable.

Figure 1. Cognitive tasks used to elicit controlled variations in cognitive load: (A) N-back working-memory task, (B) Multi-Attribute Task Battery II (MATB-II)

EEG Preprocessing

Raw EEGLAB files were loaded. Channels were restricted to a canonical 62-channel set to ensure consistency across recordings. Signals were downsampled from 500Hz to 250Hz, band-pass filtered between 1 and 45Hz, and notch-filtered at 50Hz to reduce line noise. Continuous recordings were then segmented into non-overlapping 5s epochs.

Bad channels were detected using a channel-wise standard-deviation criterion. Channels whose standard deviation exceeded the recording-level mean by more than two standard deviations were flagged as bad and interpolated when valid electrode coordinates were available. An average EEG reference was then applied to the epoched data.

Artifact correction was performed using independent component analysis (ICA). ICA was fitted with the extended Infomax algorithm, retaining components explaining 99% of the variance. To reduce computational cost, ICA was fitted on a subset of epochs (90 epochs) and then applied to the full epoch set. Components were classified using ICLabel when available, and components labelled as eye blink, muscle artifact, heartbeat, line noise, or channel noise with probability  were removed. When ICLabel was unavailable or failed, an EOG-based fallback procedure was used. Cleaned epochs were saved in FIF format, and preprocessing metadata were exported for quality control and reproducibility.

Feature Extraction

Feature extraction was performed at the epoch level from the cleaned EEG data. Cognitive-load labels were constructed by mapping task conditions to three classes: 0 - low load (ZeroBack, MATB_easy), 1 - medium load (OneBack, MATB_med), and 2 - high load (TwoBack, MATB_diff).

Power spectral densities were estimated using Welch’s method over the 1–45Hz range. Absolute band power was computed for five frequency bands: delta (1–4Hz), theta (4–8Hz), alpha (8–13Hz), beta (13–30Hz), and gamma (30–45Hz). Relative band power was calculated by dividing each band-power estimate by the total power in the 1–45Hz range. In addition, spectral entropy and spectral flatness were extracted from the normalized power spectrum.

Time-domain features included Hjorth activity, mobility, and complexity, as well as variance, mean, skewness, and kurtosis. Features were first computed at the channel level and then aggregated into six regions of interest (ROIs): frontal, central, parietal, occipital, temporal, and parieto-occipital. For each ROI, feature values were averaged across available channels. Additional derived features included theta/alpha, beta/alpha, , and  ratios, as well as frontal-minus-parieto-occipital differences for theta, alpha, and beta absolute power.

A fixed set of 10 reference channels was also used for channel-level feature extraction: F3, Fz, F4, FCz, C3, C4, CPz, P3, Pz, and P4. For these channels, absolute band powers and theta/alpha ratios were extracted.

Table 1. Summary of extracted feature groups and naming conventions

The final epoch-level feature table contained 32,737 epochs and 211 columns, including metadata, labels, and extracted features (Table 1). The class distribution was approximately balanced across the three classes: 10,846 low-load epochs, 10,906 medium-load epochs, and 10,985 high-load epochs.

Feature Preparation

Before modelling, metadata and label-related columns were removed from the feature table. These included subject, subject number, session, session number, task name, epoch index, source file, binary label, RSME score, and three-class label. Subject and source-file information were retained separately for LOSO grouping and task-level aggregation, but they were not used as predictive features. After removing identification and target columns, 201 predictive features remained.

Zero-variance features were checked, but none were detected. To reduce redundancy, highly correlated features were identified from the absolute Pearson correlation matrix computed on a temporary median-imputed copy of the feature matrix. When two features had absolute correlation |r| > 0.98, one feature was removed, resulting in 193 retained tabular features for downstream classification.

Feature distributions were inspected for skewness. Features with absolute skewness greater than 3 were transformed when possible using log1p; highly skewed features containing non-positive values were winsorized at the 0.5% tails. Multivariate outliers were identified using PCA on standardized features. Distances were computed in the PC1–PC3 space, and epochs exceeding the median distance by more than six median absolute deviations were flagged as outliers. A total of 983 epochs, corresponding to approximately 3.0% of the dataset, were flagged. These epochs were retained rather than removed to avoid changing the class or subject distribution, and the outlier information was stored for auditability rather than treated as a primary EEG predictor.

Fold-specific imputation, scaling, supervised feature selection, and model fitting were performed inside the classification pipeline described below.

Covariance-Based Representation

In addition to the manually extracted feature representation, covariance-based EEG representations were computed for Riemannian models. For each epoch, covariance matrices were estimated from the 10-channel reference set after channel-wise mean centering:

where  is the channel-by-time data matrix and  is the number of time samples. The resulting covariance matrices had shape  and were stored as a memory-mapped array. Matrix symmetry and numerical consistency were verified before modelling. A log-Euclidean embedding was also evaluated as an additional covariance-based baseline.

Classification Pipeline

Model evaluation was performed using leave-one-subject-out (LOSO) cross-validation. In each outer fold, all epochs from one subject were held out as the test set, while the remaining subjects were used for training. This design evaluates cross-subject generalization and prevents epochs from the same participant appearing in both training and test sets.

Four model families were compared: XGBoost, Random Forest, Tangent-space Logistic Regression, and Minimum Distance to Mean (MDM). XGBoost and Random Forest were trained on the extracted feature table. Tangent-space Logistic Regression and MDM were trained on covariance-based EEG representations. For feature-based models, each scikit-learn pipeline included median imputation, standard scaling, univariate feature selection using ANOVA F-statistics, and classification. The number of selected features was treated as a hyperparameter.

Hyperparameter optimization was performed using nested, group-aware cross-validation. Within each outer LOSO training set, inner folds were constructed using GroupKFold with subject identity as the grouping variable. Hyperparameters were selected using HalvingGridSearchCV, with balanced accuracy as the optimization metric. The search grids included model-specific parameters such as the number of estimators, maximum tree depth, learning rate, minimum samples per leaf, and the number of selected features. All searches were run with a fixed random seed for reproducibility.

To avoid information leakage, all fold-specific transformations were fitted only on the training subjects. In particular, median-imputation values, StandardScaler mean and variance, ANOVA feature-selection scores, selected feature subsets, hyperparameters, and classifier parameters were estimated only from the training data within the relevant cross-validation fold. The held-out LOSO test subject was transformed using the fitted training-fold pipeline and was not used during imputation, scaling, feature selection, hyperparameter tuning, or model fitting. For each fold, epoch-level predictions, predicted probabilities where available, confusion matrices, selected hyperparameters, task-level summaries, and trained model artifacts were saved for subsequent analysis.

Evaluation Metrics

Model performance was evaluated at both the epoch level and task level. The primary metric was balanced accuracy, defined as the average recall across the three classes. Balanced accuracy was chosen because it provides a class-sensitive measure of performance and is more informative than overall accuracy when class-wise performance differs.

Additional metrics included overall accuracy, macro F1-score, multiclass ROC AUC, and confusion matrices. Task-level predictions were obtained by averaging epoch-level class probabilities across all epochs belonging to the same source task file. For each fold, epoch-level predictions, task-level predictions, confusion matrices, selected hyperparameters, and trained model artifacts were saved for subsequent analysis.

Statistical Analysis

For each model, LOSO performance was summarized across subjects using the mean and nonparametric bootstrap 95% confidence intervals with 1000 bootstrap iterations. Subject-level variability was visualized using boxplots, paired fold-wise plots, and per-subject performance summaries.

To test whether model performance differed systematically, a Friedman test was applied to paired LOSO fold scores. When the omnibus test was significant, pairwise comparisons were performed using Wilcoxon signed-rank tests. Pairwise -values were corrected for multiple comparisons using the Benjamini–Hochberg false discovery rate procedure with . Paired Cohen’s  was computed on fold-wise differences to estimate effect size.

For the best-performing model, confusion matrices were summed across LOSO folds to obtain an aggregated epoch-level confusion matrix. Per-class precision, recall, and F1-score were then computed from the aggregated counts. Task-level probability summaries were also analyzed by computing the mean predicted probability assigned to the true class for each task type. To account for subject dependence, a mixed-effects linear model was fitted to task-level true-class probabilities, with true class as a fixed effect and subject fold as a random intercept.

Model Interpretation

Feature importance for the best-performing model was estimated using SHapley Additive exPlanations (SHAP) values. For each LOSO fold, the fitted model was loaded, the held-out subject’s data were transformed using the corresponding preprocessing pipeline, and SHAP values were computed using TreeExplainer. Absolute SHAP values were averaged across epochs within each fold and then aggregated across folds to obtain overall and class-specific feature-importance rankings. Importance scores were reported as mean absolute SHAP values with bootstrap confidence intervals. SHAP was used as a model-inspection tool: it indicates which variables contributed to the trained model’s predictions, but it does not establish causal neurophysiological mechanisms.

Software and Computational Environment

All analyses were performed in Python. EEG preprocessing and epoch handling were implemented using MNE-Python v1.10.2 [35] and mne-icalabel v0.8.1 [36]. Numerical computation and data handling were performed using NumPy v2.3.5, pandas v2.3.3, and SciPy v1.16.3. Machine-learning models and cross-validation procedures were implemented using scikit-learn v1.7.2 [37], XGBoost v3.1.1 [38], pyRiemann v0.9 [39], and joblib v1.5.3. Statistical analyses were performed using statsmodels v0.14.6 [40]. Model interpretation was conducted using SHAP v0.50.0 [41]. Figures were generated using Matplotlib v3.10.7 and Seaborn v0.13.2.

Experiments were conducted on a personal laptop equipped with an AMD Ryzen 5 4600H with Radeon Graphics (3.00 GHz) and 8GB RAM. No dedicated GPU acceleration or high-performance computing cluster was used. A fixed random seed was used where applicable to improve reproducibility. Intermediate artifacts, including cleaned epochs, extracted feature tables, covariance matrices, trained estimators, fold-level predictions, and summary metrics, were saved to disk for auditability and later analysis.

Results and Discussion

Model Performance

Across the three-class LOSO results comparison, XGBoost ranked highest among the tested baselines (Table 2), reaching the highest mean balanced accuracy (0.449; 95% CI [0.400, 0.511]), mean F1 score (0.433), mean accuracy (0.449), and mean AUC (0.636) over 29 folds. However, all models showed weak-to-moderate subject-independent generalization rather than robust decoding performance. Random Forest reached balanced accuracy of 0.418, F1 of 0.381, and AUC of 0.622, followed closely by TangentLR (balanced accuracy 0.410, F1 0.377, AUC 0.601). MDM showed the weakest results across all reported metrics, with the lowest balanced accuracy (0.380) and F1 score (0.312). Thus, XGBoost should be interpreted as the best model within a limited baseline comparison, not as a strong practical decoder.

Table 2. LOSO Epoch-Level Classification Performance Across Models

MDM does not output class probabilities, multiclass AUC is therefore not available for this model.

The Friedman omnibus test (, ) showed a significant difference among models. Pairwise Wilcoxon tests with FDR correction (Table 3) confirmed that MDM was consistently and significantly different from all other models (RF, TangentLR, XGB), while RF vs TangentLR, RF vs XGB, and TangentLR vs XGB were not significant. The effect sizes support that pattern: the MDM contrasts have moderate negative Cohen’s  values (, , ), while the differences among the top three are small ().

Table 3. Pairwise Post Hoc Comparisons After Friedman Test

Confusion Matrix and Subject Variability

The confusion-matrix analysis for XGBoost (Figure 2) gives a more detailed picture of a class-wise performance. The aggregated epoch-level per-class metrics were: class 0 (precision 0.458, recall 0.491, F1 0.474); class 1 (precision 0.404, recall 0.380, F1 0.392); class 2 (precision 0.479, recall 0.473, F1 0.476). Among the three classes, class 1 demonstrated the lowest precision, recall, and F1 score. The confusion matrix further showed that class 1 was frequently misclassified as both class 0 and class 2.

Figure 2. Aggregated epoch-level confusion matrix (sum counts) for XGBoost across all 29 LOSO folds

The subject-wise variability shows strong subject-to-subject heterogeneity. As shown in Figure 3, for the XGBoost model, the best subjects reach very high balanced accuracy, up to 0.999 (subject 17) and 0.920 (subject 27), which are notable outliers. In contrast, the worst subjects drop to approximately 0.32, near the three-class chance level. LOSO prevents epochs from the same participant appearing in both training and test sets, so these results should not be interpreted as ordinary train-test leakage. Instead, the heterogeneity itself is a key finding: subject variability remains a dominant factor, and some folds may benefit from idiosyncratic EEG or artifact structure aligned with task blocks.

Figure 3. Model balanced accuracy across 29 LOSO folds. Each point represents one held-out subject. Subjects 17 (0.999) and 27 (0.920) are notable outliers (XGBoost)

Task-Level Probability Analysis

Task-level analysis (Figure 4) showed that XGBoost assigned the highest mean probability to the true class for the difficult condition, with somewhat lower values for ZeroBack and easy, and the lowest values for medium, OneBack, and TwoBack. Differences between task groups were modest, and variability overlapped substantially, indicating only limited task-specific separability.

Figure 4. Task-level mean predicted probability for the true class - XGBoost. Error bars represent bootstrap 95% CI.

We assembled 2088 task-level rows across all models and folds; for the mixed-effects analysis (Table 4) of the best model (XGBoost), we used 522 observations across 29 groups. The mixed model estimated an intercept of 0.424 for prob_true; the coefficient for the numeric class label was -0.008 (p=0.391), so there is no significant linear trend. The random-effect variance (0.011) again points to subject-level variability.

The analysis included the following characteristics: 522 observations; 29 groups; scale: 0.0269; mean, min, and max group sizes: 18; log-likelihood: 172.5514; converged: yes.

Table 4. Mixed-effects model for task-level true-class probability

Feature Importance via SHAP

Lastly, feature importance analysis using SHAP provided mean absolute SHAP values, which show how strongly each feature contributed to the trained model’s predictions. These values should be interpreted as model-dependent predictive importance, not as direct evidence that a feature is a causal neural marker of cognitive load.

Figure 5. Top 20 most important features for the XGBoost model ranked by mean absolute SHAP value across LOSO folds.

SHAP analysis (Figure 5) identified temporal_mobility (0.1254), occipital_activity (0.1215), occipital_gamma_abs (0.1055), frontal_complexity (0.1038), chan_P3_gamma_abs (0.1019), frontal_minus_parieto_beta (0.0958), and chan_Pz_gamma_abs (0.0915) as the strongest global predictors. After these, several parietal, occipital, and temporal gamma and theta-ratio features appeared. This pattern is useful for understanding the fitted XGBoost model, but it should remain strictly predictive: gamma-band and Hjorth/mobility features may reflect neural activity, residual muscle or ocular artifacts, visual-task structure, or subject-specific signal variability.

Discussion

This study investigated whether cognitive load can be decoded from EEG using classical machine learning and covariance-based approaches under a subject-independent evaluation setting. The main finding is not that XGBoost provides a practically strong decoder, but that standard EEG machine-learning pipelines show only limited generalization under strict LOSO validation. XGBoost achieved the highest mean balanced accuracy (0.449), which is above the theoretical chance level of 0.333 for a three-class problem, but still far from the level required for reliable real-world deployment. The subject-wise results make this limitation especially clear: for XGBoost, the best-performing fold reached almost perfect balanced accuracy for subject 17 (0.999), whereas the lowest-performing folds were close to 0.32. In other words, the model captured highly discriminative patterns for some participants, but barely generalized for others.

This subject-level variability is one of the most important findings of the study. In a random epoch-level split, such variability could easily be hidden by subject leakage, because epochs from the same participant may appear in both training and test sets [21], [28]. The LOSO design prevents this direct leakage and provides a stricter estimate of generalization to unseen participants. However, LOSO does not remove the broader problem that subject-specific EEG structure, artifacts, baseline rhythms, electrode placement, and task-related dynamics can differ strongly across individuals. Under this evaluation setting, the extracted EEG features appear to contain workload-related information, but this information is not expressed in a stable way across all subjects. Therefore, performance heterogeneity should be treated as a substantive result rather than as a secondary nuisance.

The confusion-matrix results add another layer to this problem. For XGBoost, class 1 showed the weakest performance, with lower precision, recall, and F1-score than classes 0 and 2. This is not surprising, but it matters. The experimental design assumes that low, medium, and high load are sufficiently distinct cognitive states, yet the medium-load class may not have a stable neurophysiological signature of its own. One possible explanation is that some medium-load epochs resemble low-load epochs when the participant performs efficiently, whereas others resemble high-load epochs when the task becomes more demanding. The confusion around class 1 may therefore reflect not only a limitation of the classifier, but also a limitation of the label structure itself.

The task-level probability analysis points in the same direction. Although the difficult condition received the highest mean probability for the true class, the remaining task groups showed overlapping variability. More importantly, the mixed-effects model did not show a significant linear relationship between the numeric class label and true-class probability (, ). If the model had learned a simple ordered cognitive-load continuum, clearer separation across the low, medium, and high labels would be expected. Instead, prediction confidence appeared more irregular, suggesting that the model did not simply learn a monotonic low-to-high load structure.

This interpretation is especially relevant because the three classes were constructed by combining conditions from different cognitive tasks. Although mapping N-back and MATB-II difficulty levels onto low, medium, and high workload is theoretically reasonable, these tasks differ not only in difficulty but also in cognitive processes, stimulus structure, motor demands, and temporal dynamics [21], [33], [34], [43]. Therefore, the classifier may partially exploit paradigm-specific signatures rather than only workload-related activity. The observed performance should consequently be interpreted as a mixture of workload decoding and task-dynamics decoding. A more precise formulation is that the present study evaluates task-conditioned workload decoding, not pure cognitive-load decoding as a unified latent variable.

The statistical comparison between models should also be interpreted with restraint. MDM showed the weakest performance, and the pairwise comparisons confirmed that it performed significantly worse than RF, TangentLR, and XGB after FDR correction. However, the three better-performing models did not differ significantly from one another. Thus, the safest conclusion is not that XGBoost is decisively superior, but that MDM was poorly matched to this representation and task. The moderate negative effect sizes for the MDM comparisons support this interpretation, whereas the small and non-significant differences among XGBoost, RF, and TangentLR leave the top-model ranking less certain.

One possible reason for the weaker MDM performance is that the cognitive-load classes may not form compact and transferable covariance clusters across subjects. MDM relies on class centroids in covariance space, which can be effective when class-specific covariance patterns are compact and well separated [25], [26]. If subject-specific covariance structure dominates class-specific covariance structure, then class centroids estimated from training subjects may transfer poorly to a held-out participant. Tangent-space Logistic Regression performed better than MDM, which may indicate that the covariance representation still contained useful information, but required a more flexible discriminative model. The feature-based tree models may also have benefited from their ability to use heterogeneous spectral, time-domain, regional, and channel-level features.

The SHAP analysis showed that the XGBoost model relied most strongly on temporal mobility, occipital activity, occipital gamma power, frontal complexity, parietal gamma power, and frontal-parieto-occipital beta differences. Some of this pattern is broadly compatible with EEG workload literature: frontal and parietal features are commonly associated with attention and working-memory processes [33], [44], [45], and occipital features may reflect visual processing demands [46]. However, SHAP values explain the behavior of a trained model; they do not prove causal neurophysiological relevance. The appearance of gamma-band features among the top predictors is especially important to treat with skepticism, because scalp EEG in the gamma range is vulnerable to muscle contamination and other high-frequency artifacts [48], [49], [50]. Hjorth mobility and complexity are also general descriptors of signal variability and may reflect neural dynamics, residual artifacts, or task-specific visual/motor structure. Therefore, the SHAP analysis should be interpreted as predictive model diagnostics rather than mechanistic brain explanation [51, 52].

The main limitation of this study is therefore not simply modest performance, but ambiguity about what the model learned. The approximately balanced class distribution and LOSO validation reduce two common methodological concerns, but they do not fully separate cognitive-load effects from task-specific structure, residual artifacts, or subject-specific EEG patterns [17], [20], [53]. The medium class is particularly problematic in this respect. If class 1 does not correspond to a stable intermediate neural state, then increasing model complexity alone is unlikely to solve the problem. The current findings should therefore be treated as a realistic benchmark and upper-bound estimate for this standard pipeline under the given dataset, labels, and validation design.

Future work should first test whether the same pipeline performs more reliably in a binary low-versus-high setting. This would help determine whether the main difficulty comes from the ambiguous medium class or from broader subject-transfer problems [10]. A second necessary step is cross-task validation, such as training on N-back conditions and testing on MATB-II conditions, or the reverse. Such an analysis would provide a stronger test of whether the model detects cognitive load itself rather than task identity [4], [17]. Finally, subject-adaptive approaches should be explored, because the gap between the best and worst LOSO folds is too large to treat as noise [10], [15].

The central conclusion is that the pipeline extracted predictive EEG structure, but robust subject-independent three-class workload decoding was not achieved. The results do not point to a simple failure of machine learning, nor to a simple success of XGBoost. They point to a harder problem: workload labels, subject variability, residual artifacts, and task structure may be entangled in ways that standard classifiers can exploit without fully solving the intended decoding problem.

Conclusion

This study evaluated whether three levels of workload can be decoded from EEG using standard machine-learning approaches under a subject-independent validation setting. Using the COG-BCI database, N-back and MATB-II conditions were mapped onto low-, medium-, and high-load classes, and four classifiers were compared: XGBoost, Random Forest, Tangent-space Logistic Regression, and Minimum Distance to Mean. XGBoost achieved the highest average performance among the tested baselines, with mean balanced accuracy of 0.449, macro F1-score of 0.433, and multiclass AUC of 0.636. However, this result should not be read as evidence of a strong practical decoder. Under strict LOSO validation, all models showed weak-to-moderate generalization, so reliable subject-independent three-class decoding was not achieved.

The extracted EEG features contained predictive information related to the experimental workload conditions, but did not generalize consistently across participants. Subject-wise balanced accuracy ranged from 0.32 to 0.999, and the medium-load class was the most difficult to distinguish from the adjacent classes. This shows that the central challenge is not model selection alone, but the stability of the label structure and the transferability of EEG markers across users. Combining N-back and MATB-II conditions into a shared three-class framework is useful for benchmarking, but it also introduces construct-validity limits because task-specific and workload-related structure are difficult to separate. The results should therefore be interpreted as task-conditioned workload decoding rather than pure cognitive-load decoding.

Future work should test whether the same pipeline performs more reliably in binary low-versus-high classification, evaluate cross-task generalization by training on one paradigm and testing on the other, and explore subject-adaptive or calibration-based methods. Feature-importance results should also be validated with artifact-sensitive analyses, because gamma-band and mobility-related features may reflect residual non-neural signal sources. These steps are necessary before EEG-based workload decoding can be considered reliable for practical neuroadaptive systems.

References:

1. Sweller J. Cognitive load during problem solving: Effects on learning //Cognitive Science. - 1988. - Vol. 12. - No. 2. - P. 257-285. - DOI: 10.1016/0364-0213(88)90023-7.
2. Chikhi S., Matton N., Blanchet S. EEG power spectral measures of cognitive workload: A meta-analysis //Psychophysiology. - 2022. - Vol. 59. - No. 6. - Art. no. e14009. - DOI: 10.1111/psyp.14009.
3. K N V., Gupta C. N. Systematic review of experimental paradigms and deep neural networks for electroencephalography-based cognitive workload detection //Progress in Biomedical Engineering. - 2024. - Vol. 6. - No. 4. - Art. no. 042004. - DOI: 10.1088/2516-1091/ad8530.
4. Kingphai K., Moshfeghi Y. Mental Workload Assessment Using Deep Learning Models From EEG Signals: A Systematic Review //IEEE Transactions on Cognitive and Developmental Systems. - 2025. - Vol. 17. - No. 1. - P. 40-60. - DOI: 10.1109/TCDS.2024.3460750.
5. Tao D. et al. A Systematic Review of Physiological Measures of Mental Workload //International Journal of Environmental Research and Public Health. - 2019. - Vol. 16. - No. 15. - Art. no. 2716. - DOI: 10.3390/ijerph16152716.
6. Borghini G. et al. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness //Neuroscience & Biobehavioral Reviews. - 2014. - Vol. 44. - P. 58-75. - DOI: 10.1016/j.neubiorev.2012.10.003.
7. Charles R. L., Nixon J. Measuring mental workload using physiological measures: A systematic review //Applied Ergonomics. - 2019. - Vol. 74. - P. 221-232. - DOI: 10.1016/j.apergo.2018.08.028.
8. Diarra M., Theurel J., Paty B. Systematic review of neurophysiological assessment techniques and metrics for mental workload evaluation in real-world settings //Frontiers in Neuroergonomics. - 2025. - Vol. 6. - DOI: 10.3389/fnrgo.2025.1584736.
9. Raufi B., Longo L. An Evaluation of the EEG Alpha-to-Theta and Theta-to-Alpha Band Ratios as Indexes of Mental Workload //Frontiers in Neuroinformatics. - 2022. - Vol. 16. - DOI: 10.3389/fninf.2022.861967.
10. Zhou Y. et al. Cognitive Workload Recognition Using EEG Signals and Machine Learning: A Review //IEEE Transactions on Cognitive and Developmental Systems. - 2022. - Vol. 14. - No. 3. - P. 799-818. - DOI: 10.1109/TCDS.2021.3090217.
11. Safari M. et al. Classification of mental workload using brain connectivity and machine learning on electroencephalogram data //Scientific Reports. - 2024. - Vol. 14. - No. 1. - Art. no. 9153. - DOI: 10.1038/s41598-024-59652-w.
12. Zander T. O., Kothe C. Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general //Journal of Neural Engineering. - 2011. - Vol. 8. - No. 2. - Art. no. 025005. - DOI: 10.1088/1741-2560/8/2/025005.
13. Anders C., Arnrich B. Wearable electroencephalography and multi-modal mental state classification: A systematic literature review //Computers in Biology and Medicine. - 2022. - Vol. 150. - Art. no. 106088. - DOI: 10.1016/j.compbiomed.2022.106088.
14. Kyriaki K., Koukopoulos D., Fidas C. A. A Comprehensive Survey of EEG Preprocessing Methods for Cognitive Load Assessment //IEEE Access. - 2024. - Vol. 12. - P. 23466-23489. - DOI: 10.1109/ACCESS.2024.3360328.
15. Apicella A. et al. Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods //Neurocomputing. - 2024. - Vol. 604. - Art. no. 128354. - DOI: 10.1016/j.neucom.2024.128354.
16. Saha S., Baumert M. Intra- and Inter-subject Variability in EEG-Based Sensorimotor Brain Computer Interface: A Review //Frontiers in Computational Neuroscience. - 2020. - Vol. 13. - DOI: 10.3389/fncom.2019.00087.
17. Pušica M. et al. Machine learning performance in EEG-based mental workload classification across task types: a systematic review //Frontiers in Neuroergonomics. - 2025. - Vol. 6. - DOI: 10.3389/fnrgo.2025.1621309.
18. Lotte F. et al. A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update //Journal of Neural Engineering. - 2018. - Vol. 15. - No. 3. - Art. no. 031005. - DOI: 10.1088/1741-2552/aab2f2.
19. Puma S. et al. Using theta and alpha band power to assess cognitive workload in multitasking environments //International Journal of Psychophysiology. - 2018. - Vol. 123. - P. 111-120. - DOI: 10.1016/j.ijpsycho.2017.10.004.
20. Wang Z. et al. Cross-subject workload classification with a hierarchical Bayes model //NeuroImage. - 2012. - Vol. 59. - No. 1. - P. 64-69. - DOI: 10.1016/j.neuroimage.2011.07.094.
21. Ke Y. et al. Cross-Task Consistency of Electroencephalography-Based Mental Workload Indicators: Comparisons Between Power Spectral Density and Task-Irrelevant Auditory Event-Related Potentials //Frontiers in Neuroscience. - 2021. - Vol. 15. - DOI: 10.3389/fnins.2021.703139.
22. Katmah R. et al. A Review on Mental Stress Assessment Methods Using EEG Signals //Sensors. - 2021. - Vol. 21. - No. 15. - Art. no. 5043. - DOI: 10.3390/s21155043.
23. Demirezen G., Taşkaya Temizel T., Brouwer A.-M. Reproducible machine learning research in mental workload classification using EEG //Frontiers in Neuroergonomics. - 2024. - Vol. 5. - DOI: 10.3389/fnrgo.2024.1346794.
24. Cabañero Gómez L. et al. Studying the generalisability of cognitive load measured with EEG //Biomedical Signal Processing and Control. - 2021. - Vol. 70. - Art. no. 103032. - DOI: 10.1016/j.bspc.2021.103032.
25. Barachant A. et al. Multiclass Brain–Computer Interface Classification by Riemannian Geometry //IEEE Transactions on Biomedical Engineering. - 2012. - Vol. 59. - No. 4. - P. 920-928. - DOI: 10.1109/TBME.2011.2172210.
26. Congedo M., Barachant A., Bhatia R. Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review //Brain-Computer Interfaces. - 2017. - Vol. 4. - No. 3. - P. 155-174. - DOI: 10.1080/2326263X.2017.1297192.
27. Plechawska-Wójcik M. et al. A Three-Class Classification of Cognitive Workload Based on EEG Spectral Data //Applied Sciences. - 2019. - Vol. 9. - No. 24. - Art. no. 5340. - DOI: 10.3390/app9245340.
28. Brookshire G. et al. Data leakage in deep learning studies of translational EEG //Frontiers in Neuroscience. - 2024. - Vol. 18. - DOI: 10.3389/fnins.2024.1373515.
29. Del Pup F. et al. The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: A preliminary study //Computers in Biology and Medicine. - 2025. - Vol. 196. - Art. no. 110608. - DOI: 10.1016/j.compbiomed.2025.110608.
30. Zhou Y. et al. Cross-Task Cognitive Workload Recognition Based on EEG and Domain Adaptation //IEEE Transactions on Neural Systems and Rehabilitation Engineering. - 2022. - Vol. 30. - P. 50-60. - DOI: 10.1109/TNSRE.2022.3140456.
31. Zhou Y. et al. Cross-Subject Cognitive Workload Recognition Based on EEG and Deep Domain Adaptation //IEEE Transactions on Instrumentation and Measurement. - 2023. - Vol. 72. - P. 1-12. - DOI: 10.1109/TIM.2023.3276515.
32. Hinss M. F. et al. Open multi-session and multi-task EEG cognitive Dataset for passive brain-computer Interface Applications //Scientific Data. - 2023. - Vol. 10. - No. 1. - P. 1-22. - DOI: 10.1038/s41597-022-01898-y.
33. Owen A. M. et al. N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies //Human brain mapping. - 2005. - Vol. 25. - No. 1. - P. 46-59.
34. Comstock Jr J. R., Arnegard R. J. The multi-attribute task battery for human operator workload and strategic behavior research. - NASA. - 1992. - Available: https://ntrs.nasa.gov/citations/19920007912.
35. Gramfort A. et al. MEG and EEG Data Analysis with MNE-Python //Frontiers in Neuroscience. - 2013. - Vol. 7. - No. 267. - P. 1-13. - DOI: 10.3389/fnins.2013.00267.
36. Li A. et al. MNE-ICALabel: Automatically annotating ICA components with ICLabel in Python //Journal of Open Source Software. - 2022. - Vol. 7. - No. 76. - Art. no. 4484. - DOI: 10.21105/joss.04484.
37. Pedregosa F. et al. Scikit-learn: Machine Learning in Python //Journal of Machine Learning Research. - 2011. - Vol. 12. - P. 2825-2830. - Available: http://jmlr.org/papers/v12/pedregosa11a.html.
38. Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System //Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. - New York, NY, USA, Association for Computing Machinery. - 2016. - P. 785-794. - DOI: 10.1145/2939672.2939785.
39. Barachant A. et al. pyRiemann. - 2026. - DOI: 10.5281/zenodo.593816.
40. Seabold S., Perktold J. Statsmodels: Econometric and Statistical Modeling with Python //SciPy 2010. - 2010. - DOI: 10.25080/Majora-92bf1922-011.
41. Lundberg S. M., Lee S.-I. A unified approach to interpreting model predictions //Proceedings of the 31st International Conference on Neural Information Processing Systems. - Red Hook, NY, USA, Curran Associates Inc. - 2017. - P. 4768-4777. - DOI: 10.5555/3295222.3295230.
42. Huang G. et al. Discrepancy between inter- and intra-subject variability in EEG-based motor imagery brain-computer interface: Evidence from multiple perspectives //Frontiers in Neuroscience. - 2023. - Vol. 17. - DOI: 10.3389/fnins.2023.1122661.
43. Ke Y. et al. Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression //International Journal of Psychophysiology. - 2015. - Vol. 98. - No. 2, Part 1. - P. 157-166. - DOI: 10.1016/j.ijpsycho.2015.10.004.
44. Wager T. D., Smith E. E. Neuroimaging studies of working memory: A meta-analysis //Cognitive, Affective, & Behavioral Neuroscience. - 2003. - Vol. 3. - No. 4. - P. 255-274. - DOI: 10.3758/CABN.3.4.255.
45. Ptak R. The frontoparietal attention network of the human brain: action, saliency, and a priority map of the environment //The Neuroscientist: A Review Journal Bringing Neurobiology, Neurology and Psychiatry. - 2012. - Vol. 18. - No. 5. - P. 502-515. - DOI: 10.1177/1073858411409051.
46. Vuilleumier P., Driver J. Modulation of visual processing by attention and emotion: windows on causal interactions between human brain regions //Philosophical Transactions of the Royal Society B: Biological Sciences. - 2007. - Vol. 362. - No. 1481. - P. 837-855. - DOI: 10.1098/rstb.2007.2092.
47. Hjorth B. EEG analysis based on time domain properties //Electroencephalography and Clinical Neurophysiology. - 1970. - Vol. 29. - No. 3. - P. 306-310. - DOI: 10.1016/0013-4694(70)90143-4.
48. Muthukumaraswamy S. High-frequency brain activity and muscle artifacts in MEG/EEG: A review and recommendations //Frontiers in Human Neuroscience. - 2013. - Vol. 7. - DOI: 10.3389/fnhum.2013.00138.
49. Whitham E. M. et al. Scalp electrical recording during paralysis: Quantitative evidence that EEG frequencies above 20 Hz are contaminated by EMG //Clinical Neurophysiology. - 2007. - Vol. 118. - No. 8. - P. 1877-1888. - DOI: 10.1016/j.clinph.2007.04.027.
50. Hipp J. F., Siegel M. Dissociating neuronal gamma-band activity from cranial and ocular muscle activity in EEG //Frontiers in Human Neuroscience. - 2013. - Vol. 7. - DOI: 10.3389/fnhum.2013.00338.
51. Aas K., Jullum M., Løland A. Explaining individual predictions when features are dependent: More accurate approximations to Shapley values //Artificial Intelligence. - 2021. - Vol. 298. - Art. no. 103502. - DOI: 10.1016/j.artint.2021.103502.
52. Janzing D., Minorics L., Bloebaum P. Feature relevance quantification in explainable AI: A causal problem //Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics. - PMLR. - 2020. - Vol. 108. - P. 2907-2916. - Available: https://proceedings.mlr.press/v108/janzing20a.html.
53. Hope R. M. et al. Workload Classification Across Subjects Using EEG //Proceedings of the Human Factors and Ergonomics Society Annual Meeting. - 2011. - Vol. 55. - No. 1. - P. 202-206. - DOI: 10.1177/1071181311551042.