Code Review

TL;DR for operators A commit may be labeled as a refactor and still change behavior that users can observe. For release, backport, rollback, and dataset decisions, the label matters less than whether the parent and modified versions behave differently. The existing repository may not provide a practical way to reach the changed code: in the benchmark, original callers were absent in 50% of cases. SemaDiff addresses this by creating a small caller—called a generated dependent class—that invokes stable public APIs in both versions. It then runs the same generated tests against each version and treats a divergent outcome as evidence of a semantic-changing commit, a process known as differential execution. ...

Code review is supposed to be the sober adult in the room. A researcher writes code. A reviewer checks the code. A suspicious bug gets caught before it becomes a chart, a memo, a product decision, or—if everyone is having a particularly expensive week—a board presentation. That model works reasonably well when the failure is accidental and the reviewer has more patience than the author. It becomes less reassuring when the author is an AI research agent, the codebase is messy, the experiment is expensive to rerun, and the suspicious line looks less like a bug than a perfectly normal design choice. ...