Model Compression

When deploying large language models (LLMs) on mobile devices, edge servers, or any resource-constrained environment, quantization is the go-to trick. It slashes memory and compute costs by reducing model precision from 16-bit or 32-bit floating points to 8-bit or even 4-bit integers. But there’s a problem: this efficiency comes at a cost. Quantization can quietly erode the safety guarantees of well-aligned models, making them vulnerable to adversarial prompts and jailbreak attacks. ...

When it comes to deploying large language models (LLMs) efficiently, few challenges are as stubborn—and misunderstood—as activation outliers. For years, engineers have treated them like a natural disaster: unpredictable but inevitable. But what if they’re more like bad habits—learned and fixable? That’s the provocative premise behind a new framework called Outlier-Safe Pre-Training (OSP). Developed by researchers at Korea University and AIGEN Sciences, OSP proposes a simple but radical shift: instead of patching over outliers post hoc with quantization tricks, why not train the model to never form outliers in the first place? ...