Embodied Agents

TL;DR for operators A robot action can look reasonable from the current camera image and still fail because its physical consequence is wrong, its coordinates are slightly off, or familiar skills must be composed in an unfamiliar order. World Action Planner (WAP)1 addresses that gap by putting prediction and search between action proposal and execution: a vision-language model proposes an action, a world model imagines its consequences, semantic feedback can revise the proposal, and local search resolves finer manipulation choices. ...

TL;DR for operators An embodied agent can receive the correct user instruction and still plan toward the wrong objective if the internal description of its environment has been manipulated. Liu et al. test this failure mode by altering planner-visible state semantics rather than changing the instruction, model, planner, executor, or environment itself.1 ...

TL;DR for operators A room-tour video is a deceptively simple test for a video model. The objects do not explode, the camera does not enter a car chase, and nobody asks the model to perform cinematic philosophy. The hard part is duller and therefore more operationally relevant: the model must remember where things were, how rooms connected, what changed, and which earlier view matters now. ...

A robot does not need evil intent to become dangerous. It only needs a bad next action. “Turn on the microwave” sounds ordinary until the microwave contains a fork. “Pick up the knife” may be harmless in a cooking task until the next move is to swing it around. “Turn on the stove” may be safe for one step and unsafe three steps later if the agent forgets to turn it off. Physical risk is annoyingly literal that way. It does not wait for a model to finish reflecting on its values. ...