A Missing Modality Needs the Right Neighbors
TL;DR for operators When a multimodal prediction system lacks an expected input, it can estimate a useful internal representation from the information still available. The central design question is which observed representations should contribute to that estimate. TWIN-MoE1 combines context-conditioned pre-imputation with refinement using observed representations assigned to the same mixture-of-experts routing region. On a sentiment benchmark with 50% simulated modality missingness, expert-local reference selection produces higher mean F1 than corresponding cross-expert alternatives in all four tested architecture and aggregation combinations. ...