When EEG Stops Thinking in Squares: Why Linear-Time Models Are Quietly Winning

The hospital problem is not that EEG is too small. It is that EEG refuses to stay the same shape.

A hospital does not run machine learning inside a clean benchmark. It runs it across devices, departments, vendors, technicians, recording protocols, and patients who rarely behave like textbook signals. Electroencephalography, or EEG, makes this especially inconvenient. The signal is long, noisy, clinically useful, and structurally inconsistent. Different datasets may use different electrode counts. Different institutions may follow different montage conventions. A model that looks competent on one electrode layout can become less confident when the scalp is wired slightly differently. Apparently, brains did not agree to standardize themselves for our convenience.

The paper behind LuMamba attacks this problem from three directions at once: it uses LUNA-style channel unification to handle changing electrode topologies, FEMBA-style bidirectional Mamba blocks to process long temporal sequences more efficiently, and a mixed self-supervised objective that combines masked reconstruction with LeJEPA-style representation regularization.¹

That combination matters because EEG foundation models face two bottlenecks that are easy to confuse. One is computational: Transformer-style attention becomes expensive as sequence length grows. The other is structural: EEG channels are not a universal input schema. A model that assumes a stable set of channels is quietly assuming away one of the main reasons EEG generalization is hard.

LuMamba’s contribution is therefore not merely “Mamba for EEG.” That would be a tidy slogan, which is often where understanding goes to retire. The more useful reading is mechanism-first: LuMamba tries to build a stable latent interface for unstable electrode layouts, then run long signals through a linear-time temporal backbone, then shape the latent space so it transfers better across datasets.

The benchmark tables are important. But the architecture is the argument.

The first mechanism: turn changing electrodes into a fixed latent interface

EEG models have a boring but brutal input problem. Many machine learning systems assume that feature 1 means the same thing tomorrow as it did yesterday. In EEG, that assumption weakens quickly. A clinical dataset may use one electrode configuration; a research dataset may use another; a lower-cost device may use fewer channels; a hospital may follow a slightly different montage.

A naive response is to keep only the electrodes shared across datasets. This makes the input easier to model and the data less useful. Another response is to train separate models for separate montages. That makes deployment easier only if the business plan involves maintaining a small zoo of fragile models and calling it strategy.

LuMamba instead borrows the key idea from LUNA: project variable electrode channels into a fixed latent query space. In the paper’s architecture, the input EEG is first tokenized into temporal patches. Each patch receives temporal, spectral, and 3D electrode-position information. A channel-unification module then uses learnable queries to attend over the channel dimension and produce a latent representation with fixed shape. The model no longer needs the raw channel count to define the downstream representation.

A simplified view is:

raw EEG channels -> electrode-aware patch embeddings -> learned latent channel queries -> fixed latent representation

This is the “unified” part of LuMamba. It is not just a preprocessing trick. It changes the interface between data collection and modeling. The model can receive a 16-channel Alzheimer’s dataset, a 26-channel Parkinson’s dataset, or a 20–22 channel pretraining corpus, then map them into a representation space that downstream temporal modeling can consume.

For business use, this is the operationally interesting part. The expensive pain in clinical AI is rarely a single training run. It is maintenance across data sources. If every hospital, device, and protocol requires its own tuned model, deployment becomes a consulting project disguised as software. A latent channel interface does not eliminate that problem, but it points to a less miserable version of it.

The second mechanism: stop paying quadratic rent on long brain signals

The second mechanism is temporal. EEG is a time series, and time series grow. Continuous or long-window biosignal monitoring quickly runs into memory and compute limits when attention-based models scale with sequence length.

The familiar Transformer problem is:

where $n$ is the sequence length. This is manageable until it is not. In biosignals, “not” arrives early.

LuMamba replaces the heavy Transformer-style temporal core with bidirectional Mamba blocks, following the FEMBA line of work. Mamba belongs to the family of state-space sequence models, designed to model long-range dependencies with linear-time scaling:

The bidirectional design matters because EEG events are not always best interpreted from a single forward direction. Artifacts, transient abnormalities, and disease-relevant patterns may be better represented with context on both sides of a window. LuMamba processes the unified latent sequence through two bidirectional Mamba blocks, keeping the temporal modeling efficient while still allowing richer context.

The computational result is not a decorative benchmark. In the paper’s efficiency test, LuMamba requires 26.5× fewer FLOPs than LUNA, 377× fewer than LaBraM, and 3718× fewer than EEGFormer when evaluated at the maximum sequence length each baseline supports before out-of-memory. It also supports sequences 12.6× longer than LUNA, 501× longer than LaBraM, and 2.5× longer than EEGFormer before hitting the memory limit on an NVIDIA A100-64GB GPU with batch size 1.

The absolute numbers should not be read as a direct hospital deployment guarantee. An A100 memory-limit experiment is not the same as running on an edge device in a clinic. Still, the direction is meaningful: the architecture shifts the limiting factor. If long EEG recordings are part of the product roadmap, quadratic attention is not a small nuisance. It is a tax that compounds.

The third mechanism: prettier clusters are not always better representations

The most interesting part of the paper is not the speed claim. It is the representation-learning result.

Most EEG foundation models lean on masked reconstruction: hide parts of the signal and train the model to recover them. This objective rewards the model for learning enough structure to reconstruct missing patches. It often produces visually organized embeddings. When projected with t-SNE, those embeddings can look pleasantly clustered. Everyone enjoys a clean cluster plot. It gives the illusion that the universe has forgiven us.

LuMamba tests a different ingredient: LeJEPA, adapted from image and video self-supervised learning to EEG time series. The adapted setup samples local and global temporal windows from the same signal. The model learns to align local views with a global representation while SIGReg regularizes embeddings toward an isotropic Gaussian distribution. In plain terms, LeJEPA pushes the representation away from overly compact, dataset-specific clustering and toward a smoother latent geometry.

The paper compares three pretraining strategies:

This is where the reader misconception matters. A visually separated embedding space is not automatically a better general-purpose representation. In LuMamba’s results, reconstruction-only pretraining produces clearer t-SNE clusters on TUAR and APAVA. But the mixed LeJEPA-reconstruction objective generalizes better on several downstream tasks, especially those involving unseen electrode montages.

The paper’s t-SNE figure should be treated as exploratory visual evidence, not proof. Its purpose is to illustrate representation geometry: reconstruction produces separation; LeJEPA produces diffusion; the mixed objective sits between them. The downstream task table is the actual evidence for whether that geometry helps.

The objective comparison says “transfer,” not “universal dominance”

Table I is the most useful evidence because it separates the pretraining objective from the rest of the architecture. This makes it an ablation, not merely a leaderboard.

The mixed objective gives the best TUAB balanced accuracy and the strongest APAVA result. It ties reconstruction-only on TDBrain AUROC while slightly improving AUPR. But it loses to reconstruction-only on TUAR and TUSL. That is not a footnote; it is the shape of the result.

The interpretation is that LeJEPA helps when transfer across heterogeneous conditions matters, but it may soften the class-specific structure that helps certain in-distribution or imbalanced tasks. The paper explicitly notes that reconstruction-only performs better on TUAR by about 1–2% and on TUSL by up to 4%. In other words, the mixed objective is not magic powder. It is a trade-off.

For business readers, this trade-off is more valuable than a universal win would be. Real deployments must choose whether they are optimizing for one known task and one stable setting, or for a platform that must survive new sites, devices, and disease-specific datasets. LuMamba is more convincing as a platform architecture than as a claim that every EEG classifier should immediately switch objectives.

The benchmark story is competitive, uneven, and more useful because of that

The paper then compares LuMamba-Tiny with state-of-the-art baselines. These tests serve as comparison with prior work, not as clean ablations, because each baseline differs in architecture, training strategy, and sometimes evaluation setup.

On TUAB, LuMamba-Tiny reaches 80.99 ± 0.22 balanced accuracy, 0.8918 ± 0.0032 AUPR, and 0.8825 ± 0.0038 AUROC. LaBraM-Base remains stronger on AUPR and AUROC, with 0.8965 ± 0.0016 AUPR and 0.9022 ± 0.0009 AUROC, and slightly higher balanced accuracy at 81.40 ± 0.19. LUNA-Base reports 80.63 ± 0.08 balanced accuracy, 0.8953 ± 0.0016 AUPR, and 0.8868 ± 0.0015 AUROC.

The fair reading: LuMamba is competitive on TUAB, especially given its efficiency profile, but it does not dominate LaBraM or LUNA across every metric. That is perfectly acceptable. A model can matter without winning every column. This is a difficult idea for benchmark culture, which often treats tables like horse races with more decimals.

The disease-specific results are more striking, but also more fragile.

APAVA is the headline-friendly result: LuMamba reaches 0.970 AUPR for Alzheimer’s detection and improves over reconstruction-only by more than 20 percentage points in AUPR. That supports the paper’s claim that LeJEPA-style regularization can help under unseen electrode montages.

But small clinical datasets are where overclaiming goes to reproduce. APAVA has 23 patients. The reported split is subject-disjoint, which is good, but the test set is still tiny. This is evidence worth taking seriously as a signal, not evidence sufficient for clinical product claims. “Promising architecture for cross-montage disease detection” is a defensible sentence. “Clinical-grade Alzheimer’s AI” is how one gets invited to regulatory conversations for the wrong reasons.

The evidence map is stronger when each test is read for its actual purpose

A useful way to read the paper is to separate the tests by purpose. Otherwise the article collapses into a list of numbers, and lists of numbers are where interpretation quietly dies.

This map makes the paper more, not less, interesting. The result is not a simple victory lap. It is a design study showing that the three mechanisms pull in different directions: channel unification supports heterogeneous inputs, Mamba supports longer sequences, and LeJEPA reshapes the latent space toward transfer.

The combined system is attractive precisely because EEG deployment is a multi-constraint problem. The model does not need to be the absolute best specialist on every benchmark to be useful. It needs to reduce the cost of moving across datasets, devices, and recording lengths.

What the paper directly shows

The direct claims supported by the paper are fairly clear.

First, LuMamba combines topology-invariant channel unification with a bidirectional Mamba temporal backbone and can be pretrained on a large unlabeled EEG corpus: approximately 21,600 hours from more than 14,000 patients in TUEG. That matters because foundation-model behavior depends on large unlabeled data and downstream reuse, not only on supervised task tuning.

Second, the architecture handles downstream datasets with different electrode configurations, including APAVA with 16 channels and TDBrain with 26 channels, while pretraining uses TUEG recordings with 20–22 channels. The paper’s setup is therefore aligned with the cross-montage problem rather than pretending every dataset arrives in the same shape.

Third, LeJEPA-reconstruction changes the representation trade-off. Reconstruction-only looks cleaner in t-SNE and performs better on some TUH tasks. Mixed LeJEPA-reconstruction performs better on three out of five tasks and is especially strong on APAVA. The paper’s most valuable conceptual point is that latent geometry optimized for visual separability may be less robust under structural shift.

Fourth, LuMamba is materially more efficient in the paper’s FLOPs and memory-limit comparisons. This follows from replacing Transformer-heavy temporal modeling with a linear-time sequence backbone. Efficiency is not just an engineering afterthought here; it is part of the model’s scientific and operational thesis.

What Cognaptus would infer for business use

The business interpretation should be more restrained than the technology press usually enjoys. The paper does not prove that LuMamba is ready for clinical deployment. It does suggest a practical product architecture for biosignal AI.

The first product implication is that cross-device support should be designed into the representation layer, not patched through downstream retraining. A learned latent channel interface can become a reusable “adapter” between messy hardware reality and stable model internals. For a healthcare AI vendor, that means fewer one-off model variants and a cleaner path to serving multiple institutions.

The second implication is that long-sequence efficiency changes product scope. If the model can process longer EEG windows before hitting memory limits, then the product can consider use cases that short-window models make awkward: prolonged monitoring, sleep studies, seizure-risk analysis, artifact-aware review, or clinician-assistive triage over longer recordings. The paper does not validate all of those use cases. It makes them less computationally absurd. A modest but valuable service.

The third implication is that pretraining objectives should be selected by deployment risk, not just validation accuracy. A hospital-network model must tolerate shifts in device, population, protocol, and disease mix. A reconstruction-only objective may create stronger local structure but weaker cross-context behavior. A mixed objective may give up some in-distribution sharpness for broader transfer. That is not a defect. It is a design choice.

The fourth implication is organizational. If biosignal AI becomes a foundation-model platform, the high-value work shifts from building one classifier per dataset to managing a reusable pretraining-and-adaptation pipeline. That pipeline needs data governance, validation protocols, montage-aware evaluation, and monitoring across sites. In other words, the model is only the visible part of the machine. The operational system around it is where most of the adult supervision will happen.

The deployment boundary: this is platform evidence, not clinical certification

The limitations are not decorative. They define the correct use of the paper.

APAVA and TDBrain are small downstream disease datasets. APAVA has 23 patients, split into 15 train, 4 validation, and 4 test subjects. TDBrain has 72 patients, split into 34 train, 8 validation, and 8 test subjects. Subject-disjoint splits reduce leakage risk, but small test sets still make performance estimates fragile.

The paper also evaluates three runs with fixed random seeds. That is reasonable for an early research paper, but it is not enough to establish robustness across acquisition sites, patient subgroups, device vendors, or clinical workflows.

The strongest business claim should therefore be about architecture and cost structure: LuMamba is evidence that topology-invariant, linear-time EEG foundation modeling is technically plausible and computationally attractive. The weaker claim would be immediate diagnostic readiness. The paper does not earn that, and it does not need to.

There is also an important task-specific boundary. LuMamba underperforms stronger specialist baselines on TUAR and TUSL. The authors argue that the gap reflects a generalization-oriented pretraining strategy rather than architectural failure, and Table I partially supports that argument because reconstruction-only performs better on these tasks. Still, for a business deploying a narrow artifact detector or slowing classifier in one known environment, a specialized model may remain the rational choice.

The practical conclusion is not “replace everything with LuMamba.” It is “use this architecture when heterogeneity, sequence length, and reuse matter enough to justify a platform approach.”

The quiet lesson: EEG foundation models need interfaces, not just bigger backbones

The deeper pattern in LuMamba is not limited to EEG. Many enterprise AI systems face the same structural problem: the underlying signal is stable enough to learn from, but the input schema changes across sources. In EEG, the schema is electrode layout. In industrial IoT, it may be sensor placement. In finance, it may be market microstructure and data vendor conventions. In enterprise automation, it may be business-process fields that mutate every time someone redesigns a form and calls it transformation.

The common solution is not always a bigger model. Sometimes it is a better interface: map unstable observed inputs into a stable latent space, then learn temporal or causal patterns there. LuMamba is a clean example of that idea because the components line up neatly. LUNA handles the changing spatial topology. Mamba handles the long temporal sequence. LeJEPA-reconstruction shapes the latent representation so it does not cling too tightly to one dataset’s visible structure.

That is why the paper deserves a mechanism-first reading. If we only ask whether LuMamba wins every benchmark, we miss the useful lesson. The model is interesting because it treats EEG as a deployment problem, not just a signal-classification problem.

The old approach made EEG fit the model. LuMamba tries to make the model tolerate EEG.

That is a quieter kind of progress. It also tends to be the kind that survives contact with reality.

Cognaptus: Automate the Present, Incubate the Future.