Mean Field Games

Simulation has a bad habit: it becomes realistic just when it becomes too expensive to run. A simple market model can treat everyone as the same kind of agent and still say something useful. A richer model lets agents differ by wealth, income, health, location, battery level, portfolio position, or whatever state variable the domain demands. Then someone remembers that real agents do not see the whole system. Investors see prices, not everyone’s balance sheet. Households see wages and interest rates, not the full wealth distribution. Drivers see traffic signals and congestion, not the hidden intention of every other driver. ...

TL;DR for operators Simulation is a dangerous place to confuse optimisation with truth. Chen and Zhang’s paper, From Individual Learning to Market Equilibrium, shows that a reinforcement learning agent can optimise very successfully and still fail to reproduce the economic equilibrium it was supposedly simulating.1 That is the useful sting in the paper. The failure is not that the RL agent is too weak. The failure is that the environment quietly gives the agent the wrong economic role. ...