Much of what we know about learning and decision-making comes from simple lab tasks, yet real life is open-ended, and full of things that could matter at once. This review looks at studies that deliberately include everyday complexity in experimental design, while still using reinforcement learning as a theoretical framework.