Diffusion-Transformers

TL;DR for operators A product that must understand an instruction, preserve a voice or reference sound, and stay synchronized with video is handling several different kinds of control. Audio-Omni’s strongest architectural evidence says those controls should not be forced through the same interface. The system combines a frozen multimodal language model with a trainable audio generator. High-level meaning and transcript information are supplied as flexible context; synchronization and acoustic-reference information are attached directly to the evolving audio representation. In the paper’s conditioning ablation, that allocation performs best across text-to-audio, video-to-audio, text-to-speech, and audio editing. ...

TL;DR for operators The useful surprise in Tail-Aware HiFloat4 is not that a 4-bit video model gets worse. That part is not exactly a Nobel-level plot twist. The useful surprise is where it gets worse. The paper reports a W4A4 HiFloat4 post-training quantization pipeline for Wan2.2-I2V-A14B, and under matched generation settings the unweighted mean score drops from 0.6800 to 0.5880. But the collapse is concentrated: subject consistency falls from 0.9331 to 0.5324, while aesthetic quality is effectively unchanged, overall consistency is comparable, and motion smoothness drops only slightly from 0.9923 to 0.9803.1 ...

A producer rarely begins by asking for a brand-new song from the void. More often, the request is smaller and harder: make this guitar line sound like a flute, move this loop toward jazz, keep the rhythm, preserve the recognisable phrase, and please do not turn the whole thing into synthetic soup. ...