Edge-Ai

TL;DR for operators Federated LLM fine-tuning sounds attractive until someone asks the rude operational question: who is actually paying for the compute, memory, and communication on the devices? The paper behind DevFT proposes a useful answer: do not fine-tune the full model end-to-end from the first round. Start with a compact submodel, train it federatively, transfer the learned LoRA parameters forward, then expand the model in stages until it reaches the full target size.1 The authors call this Developmental Federated Tuning, and yes, the developmental psychology metaphor is a little enthusiastic. Fortunately, the mechanism is more interesting than the metaphor. ...

TL;DR for operators A field robot does not care that your neural network is elegant. It cares whether the model fits on the device, runs without draining the battery, and still recognises the weed before the sprayer makes an expensive little mistake. The paper introduces knowledge grafting, a mechanism for taking selected intermediate features from a larger donor model and attaching them to a smaller deployable model, called the rootstock.1 In the reported DeepWeeds experiment, the authors reduce a VGG16-derived model from 64.39 MB to 7.38 MB, cutting parameters from 16,880,201 to 1,934,665, while reporting 90.45% test accuracy on unseen images. ...

TL;DR for operators Production data does not politely wait for quarterly retraining. Sensor readings shift, fraud patterns mutate, market microstructure changes, network traffic acquires new habits, and customer behaviour performs its usual interpretive dance. This is concept drift: the model is still running, but the world it learned from has moved on. ...

TL;DR for operators RALLY is not a chatbot with propellers. It is a hybrid control framework for UAV swarms where the LLM supplies structured semantic reasoning and the reinforcement-learning layer decides how agents should divide responsibility.1 The practical insight is the separation of labour. A drone swarm does not only need to know where to fly; it needs to agree who should lead, who should coordinate, who should follow, and when those roles should change. RALLY handles that by combining two-stage LLM consensus with RMIX, a role-value mixing network trained to assign Commander, Coordinator, and Executor roles under partial observability and limited communication. ...

TL;DR for operators Quantizing an LLM is not a harmless cost-saving step. It changes the model, and the paper analysed here shows that those changes can weaken safety even when familiar utility scores still look respectable. That is the uncomfortable part: the dashboard can say “performance preserved” while the model has become more willing to comply with harmful requests. Very efficient. Very modern. Very easy to miss. ...

TL;DR for operators If your deployment plan depends on squeezing a language model into cheap inference hardware, this paper is worth reading because it changes the timing of the quantization problem. Most quantization work asks: “How do we repair a model after training so it survives 4-bit inference?” Outlier-Safe Pre-Training asks a more irritating question: “Why did we train a quantization-hostile model in the first place?”1 ...

TL;DR for operators Photonic AI chips are not “GPUs, but shiny.” That is the lazy version, and as usual the lazy version is slightly wrong in the most expensive place. The practical story is narrower and more useful. Two recent Nature papers show that photonic systems can now do more than charming lab tricks. Hua et al.’s PACE system demonstrates a 64 × 64 photonic matrix-vector accelerator with more than 16,000 photonic components, low-latency optical multiply–accumulate operations, and strong performance on Ising-style optimisation workloads.1 Ahmed et al. demonstrate a photonic AI processor capable of running real neural-network workloads, including ResNet, BERT, and Atari reinforcement learning, with near-electronic precision across many tasks.2 ...