What a Spiking Network Learns to Forget

TL;DR for operators

Making a classifier more resistant to corrupted or manipulated inputs can reduce its accuracy on ordinary ones. On CIFAR-10, Spiking Conditional Information Bottleneck (SCIB) raises accuracy under a specified iterative adversarial attack from 0.78% to 47.58%, while clean accuracy falls from 92.94% to 89.69% (four timesteps; PGD at 4/255, Section 5.4, Table 2).¹ SCIB achieves this by training a spike-based neural network to retain information useful for classification while compressing additional input variation within each class, without changing its neuronal dynamics. It also leads 15 of 18 tested environmental-noise configurations, though robustness gains vary by disturbance. On CIFAR-100, clean accuracy drops by 9.07 percentage points in a four-timestep comparison. The reported 24.9% inference-energy reduction is an analytical estimate, not a hardware measurement. For edge-AI operators, the decision is whether improved robustness against expected disturbances justifies the clean-accuracy loss and whether projected computational savings translate into actual deployment benefits.

A Large Robustness Gain Comes with an Accuracy Cost

A perception system has two jobs that can conflict. It must classify ordinary inputs accurately, yet remain useful when sensor readings become noisy or deliberately manipulated. Training for the second requirement can change how the model handles the first.

The trade-off is unusually visible in a recent experiment. Under a seven-step Projected Gradient Descent (PGD) attack, a modified classifier correctly classified 47.58% of CIFAR-10 images, compared with just 0.78% for an unconstrained baseline. On clean images, however, its accuracy was 89.69%, against the baseline’s 92.94%.

That is a 46.80-percentage-point improvement under the specified attack, accompanied by a 3.25-point reduction on clean inputs (Section 5.4, Table 2). The comparison uses four simulation timesteps and an adversarial perturbation budget of 4/255.

The modified classifier uses SCIB, a training method for spiking neural networks (SNNs). These networks communicate through discrete spikes over simulation timesteps, potentially reducing computation when neuronal activity is sparse.

The result raises a practical design question: what information should a classifier discard to become less sensitive to disruption, and how much ordinary predictive performance can it afford to lose?

Compress Input Variation Without Erasing Class Information

The paper’s starting observation comes from information-plane analysis on Fashion-MNIST and CIFAR-10. Conventionally trained SNNs retain substantial information about their inputs even after the information predictive of class labels has largely stabilized (Section 3, Figure 1).

That remaining information is not automatically harmful. It may describe appearance variations, fine-grained details, or other properties that could matter in different tasks.

Conventional information bottleneck training compresses the total dependence between input $X$ and learned representation $S$. The researchers instead use a supervised decomposition:

Under the Markov structure $Y-X-S$, the first term captures information about class label $Y$. The second measures how much additional information the representation retains about the input after the label is known (Section 4.1, Equation 4).

SCIB targets this second term. Its ideal objective preserves label-predictive information while penalizing residual within-class input dependence.

The intuition is that two images belonging to the same class should not produce excessively different representations solely because of irrelevant input variation. This provides a plausible mechanism for reducing sensitivity to noise without compressing all predictive information indiscriminately.

The implementation makes that principle trainable. Each class receives a learnable Gaussian reference distribution with its own mean and diagonal variance. The encoder produces an input-dependent Gaussian distribution, and training penalizes its divergence from the appropriate class reference.

The practical loss is:

Here, cross-entropy preserves classification performance, the KL term encourages within-class representational compactness, and $\beta$ controls compression strength.

Importantly, the KL term is a variational upper bound on conditional mutual information, not a direct measurement of the exact quantity (Section 4.3, Equations 13–15). Training labels select the appropriate reference distributions, but ground-truth labels are not required during inference.

The method uses standard surrogate-gradient training and retains the SNN’s neuronal dynamics. Its contribution lies in the learning objective and representation structure, not a redesigned spiking mechanism.

Robustness Gains Differ Across Noise and Attack Types

The principal experiments use a VGG-9 SNN with leaky integrate-and-fire neurons on Fashion-MNIST, CIFAR-10, and CIFAR-100. The researchers test three distinct sources of disruption.

Environmental noise provides the broadest comparison. SCIB achieves the highest classification accuracy in 15 of 18 configurations involving Gaussian, salt-and-pepper, and Rayleigh corruption (Section 5.2, Table 1).

Under stronger salt-and-pepper noise, with noise density 0.3, SCIB reaches 92.73% on Fashion-MNIST, 87.13% on CIFAR-10, and 58.40% on CIFAR-100. These are the highest reported values in their respective configurations.

The exceptions help calibrate the result. Under strong Rayleigh noise on Fashion-MNIST, conventional information bottleneck training reaches 85.07%, slightly above SCIB’s 84.99%. Other methods also lead two lower-intensity configurations.

The 15-of-18 result demonstrates broad comparative success across the tested conditions, not uniform superiority. The noise-intensity parameters also have distribution-specific meanings and should not be interpreted as one common physical scale.

Simulated hardware noise tests a different failure mechanism. The researchers perturb synaptic weight updates and membrane potentials rather than corrupting input images (Section 5.3, Figures 4–5).

At membrane-potential noise intensity 5 on CIFAR-10, SCIB achieves 64.45% accuracy against 64.08% for conventional IB. Under strong synaptic-weight noise on CIFAR-100, it achieves 62.98%, approximately 0.9 percentage points above the second-best method.

These results indicate improvements in selected high-noise simulations, but the margins are much smaller than those observed in some adversarial comparisons.

Adversarial attacks reveal the largest reported differences. On CIFAR-10 with four timesteps, SCIB reaches 26.65% accuracy under PGD at 8/255, against 1.73% for FEEL-SNN and 0.02% for Vanilla SNN (Section 5.4, Table 2).

SCIB also performs strongly against the evaluated Basic Iterative Method attacks without adversarial-example training. But it does not dominate every attack: on CIFAR-100 with four timesteps and FGSM at 4/255, FEEL-SNN achieves 19.51%, compared with SCIB’s 14.08%.

The clean-accuracy penalty becomes particularly consequential on CIFAR-100: SCIB records 64.73%, against 73.80% for Vanilla SNN.

For an operator, these are different performance regimes, not interchangeable measurements of a single robustness property.

Compression Strength Changes the Outcome

The paper’s sensitivity analysis provides an additional implementation warning.

On CIFAR-10, increasing $\beta$ from 0.2 toward 1.0 produces relatively flat or non-monotonic changes under Gaussian input noise. Under simulated membrane-potential noise, accuracy declines as compression strengthens over the tested range (Section 5.6.2, Figure 8).

This is a sensitivity experiment, not another independent demonstration of overall superiority. It shows why stronger regularization cannot be assumed to produce stronger robustness.

The representation visualizations offer complementary mechanistic evidence. PCA projections and pairwise KL matrices show class-conditional Gaussian priors becoming more separated over training (Section 5.6.1, Figure 7).

That geometry is consistent with the proposed mechanism, but it does not establish that every discarded within-class difference was unnecessary. Nor does the paper provide a separate component-ablation table that isolates the conditional objective from the learnable class-specific prior.

For implementation, the compression coefficient must therefore be treated as a deployment-dependent hyperparameter, not a universally preferable setting.

The 24.9% Energy Reduction Is a Computation Estimate

SNN efficiency depends partly on the number of spikes propagated through network layers. Fewer spikes can mean fewer accumulation operations.

The paper records layer-wise spike activity and estimates single-sample inference computation energy using reference costs of 4.6 pJ per multiply-accumulate operation and 0.9 pJ per accumulation operation.

Its resulting estimates are:

SCIB’s estimate is 24.9% below the unconstrained SNN and 55.8% below the ANN comparison (Section 5.5, Table 3 and Figure 6).

This is encouraging for computation-level efficiency. It is not a measurement of physical-chip energy consumption.

The model excludes memory movement, communication, peripheral circuitry, and other system costs. Consequently, the reported reduction cannot directly support a commercial claim about battery life, device power, or deployment economics.

What an Edge-AI Team Should Validate Before Adoption

For teams developing spiking-based perception or neuromorphic inference, Cognaptus infers a practical evaluation pathway from these results.

First, establish the disturbance profile expected in the actual application. Camera noise, internal neuronal variability, and adversarial manipulation call for separate tests because SCIB responds differently to each.

Second, evaluate clean accuracy and disturbed-input accuracy together across candidate compression coefficients. For applications that depend on fine-grained distinctions, the risk of discarding useful within-class variation deserves particular attention.

Third, compare SCIB with the existing training baseline under matched architecture and evaluation conditions. If mechanism attribution matters, separately test the conditional objective and class-specific priors; the reported experiments do not fully disentangle their contributions.

Finally, measure energy and latency on the target device, including memory and communication costs. SynOps estimates can guide candidate selection, but cannot substitute for those measurements.

Several boundaries remain. The reported work uses image-classification benchmarks rather than real event-stream deployments. Hardware perturbations are simulated, attack coverage is limited to specified surrogate-gradient methods and budgets, and the benchmark tables lack detailed cross-seed uncertainty estimates. Robustness against defense-aware adaptive attacks or dedicated adversarially trained alternatives is not established.

A Better Training Objective, Not a Reliability Guarantee

SCIB demonstrates that changing the information retained during SNN training can improve robustness across several distinct perturbation settings without altering neuronal dynamics. Its class-conditional compression objective is tractable, compatible with existing surrogate-gradient optimization, and associated with lower analytical inference-computation costs.

The remaining decision is application-specific. A model that withstands a particular attack while losing clean accuracy may be preferable for one perception system and unsuitable for another. Its estimated computational savings may also change once actual hardware costs are included.

The paper makes conditional information compression a credible candidate for robustness-oriented SNN training. Turning that candidate into a dependable edge-AI system requires choosing the disturbances that matter, quantifying the accuracy trade-off, and measuring the complete deployed cost.

Cognaptus: Automate the Present, Incubate the Future.