TLDR: AI environments are becoming a core ingredient of model capability: executable worlds that generate trajectories and feedback, not just exams. This essay traces their path from Atari, OpenAI Gym, and AlphaZero through MuJoCo to the RLVR-driven post-training behind Tulu 3 and DeepSeek-R1, then surveys the new generation of agent environments, including SWE-Gym, WebArena, OSWorld, TheAgentCompany, Tau2-bench, and AgentMercury. It then lays out what makes a good environment: rewards that track the real outcome and resist hacking, causal fidelity, affordable and repeatable practice, structured difficulty, and improvement that transfers outside the training world. The frontier, the essay argues, is the supply of varied, consequential practice with dependable feedback. Imagine hiring an engineer who has read every programming book but has never run a program. They can explain databases, discuss distributed systems, and produce an impressively confident architecture diagram. Then you...
learn more
