Cover image

Merge Without Mayhem: How Orthogonal Deltas Could Revolutionize Model Composition

TL;DR for operators Model composition usually sounds harmless until someone asks the obvious production question: “Can we remove that client-specific update without retraining the whole thing?” At that point, many elegant AI stacks quietly become sedimentary rock. The MDM-OC paper proposes a cleaner model lifecycle: keep a shared base model, express every fine-tuned specialist as a task delta, orthogonalize those deltas so they interfere less, merge them with tuned coefficients, and subtract a selected delta later when a capability, customer, or data source needs to be removed.1 The important claim is not “we found another averaging recipe.” The claim is that model updates can be treated as separable components in parameter space. ...

August 2, 2025 · 20 min · Zelina
Cover image

The LoRA Mirage: Why Lightweight Finetuning Isn't Lightweight on Privacy

TL;DR for operators Adapters look small. The privacy surface is not. The paper behind LoRA-Leak argues that LoRA fine-tuning does not magically protect the records used to specialise a language model.1 Even though LoRA trains only low-rank adapter weights while leaving the base model frozen, the resulting model can still leak membership information: an attacker may infer whether a given sample was part of the fine-tuning dataset. ...

July 25, 2025 · 17 min · Zelina
Cover image

OmniAvatar’s Metrics & Training: Under the Hood of Next-Gen Avatars

TL;DR for operators OmniAvatar is best read as a shift from “make the mouth move” to “make the person perform.” The paper introduces an audio-driven avatar video generation system that takes a reference image, an audio clip, and a text prompt, then generates facial and semi-body video with synchronised speech, adaptive body motion, and prompt-controlled scene elements.1 ...

June 24, 2025 · 16 min · Zelina
Cover image

Blind Trust, Fragile Brains: Why LoRA and Prompts Need a Confidence-Aware Backbone

TL;DR for operators LoRA and prompts are attractive because they make model adaptation feel almost too easy: add a few examples, attach a small adapter, nudge the model into a domain, and call it customised. The uncomfortable part is that adaptation changes not only what a model says, but how confidently it says it. A compliance assistant that becomes slightly more domain-specific but far more overconfident has not been improved. It has been promoted beyond its competence, a classic corporate move. ...

March 25, 2025 · 14 min · Zelina