New Signer, Old Sentence: What Sign-Language Translation Benchmarks Are Actually Testing
TL;DR for operators A held-out test set does not necessarily tell you how a sign-language translation system will behave with a new user. In Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation, Artiaga and colleagues re-evaluate three gloss-free systems after excluding the test signer from training and development.1 ...