LoRA

If federated fine-tuning feels like trying to teach calculus to a toddler on a flip phone, you’re not alone. While the privacy-preserving benefits of federated learning are clear, its Achilles’ heel has always been the immense cost of training large models like LLaMA2-13B across resource-starved edge devices. Now, a new method—DEVFT (Developmental Federated Tuning)—offers a compelling paradigm shift, not by upgrading the devices, but by downgrading the expectations. At least, at first. ...

When we talk about parameter-efficient fine-tuning, LoRA (Low-Rank Adaptation) is often celebrated as a silver bullet: cost-effective, memory-efficient, and—many assume—safe. After all, it modifies only a small fraction of model parameters, sideloaded as low-rank matrices, while leaving the massive pretrained model backbone untouched. The prevailing belief has been that such minimal intervention can’t possibly memorize or leak sensitive data. This belief is now decisively debunked by LoRA-Leak, a landmark framework introduced in a new paper by researchers from Tsinghua and HKUST. Their findings are a wake-up call for AI developers and policymakers alike: even LoRA-finetuned models are highly vulnerable to membership inference attacks (MIAs)—and ironically, the very presence of the frozen pretrained model amplifies this leakage risk. ...