Brains with Gradients: Why Energy-Based Transformers Might Be the Future of Thinking Machines

Brains with Gradients: Why Energy-Based Transformers Might Be the Future of Thinking Machines AI models are getting better at mimicking human intuition (System 1), but what about deliberate reasoning—slow, careful System 2 Thinking? Until now, most methods required supervision (e.g., reward models, verifiers, or chain-of-thought engineering). A new architecture, Energy-Based Transformers (EBTs), changes that. It offers a radically unsupervised, architecture-level path toward models that “think,” not just react. The implications for robust generalization, dynamic reasoning, and agent-based autonomy are profound. ...

July 4, 2025 · 3 min · Zelina