https://cognaptus.com/tags/transformer-architecture/ Recent content in Transformer Architecture on Cognaptus Hugo -- 0.145.0 en-us Sat, 27 Dec 2025 00:00:00 +0000 https://cognaptus.com/blog/2025-12-27-attention-but-make-it-optional/ Sat, 27 Dec 2025 00:00:00 +0000 https://cognaptus.com/blog/2025-12-27-attention-but-make-it-optional/ A mechanism-first look at why some late self-attention layers in dense LLMs can be pruned without calibration data—and why that does not mean attention is suddenly obsolete.