You are just a fancy reward function, bro

Reinforcement Learning has emerged as a compelling and powerful method for training agents to make decisions, often surprising humans with moves that initially appear counterintuitive, risky, or outright baffling. Yet, these seemingly insightful choices often lead to exceptional outcomes, driven purely by an agent's relentless optimization of rewards.

But this raises a deeper, philosophical question: Are RL agents truly exhibiting intelligence, or are they simply highly efficient "reward hackers"?

When humans label something as "intelligent", we imply understanding, a grasp of cause-and-effect relationships, purposeful intent, and meaningful abstraction. An RL agent, however, has none of these. It doesn't understand what a "win" truly signifies beyond a numeric increase in its reward function. It doesn't possess intentions or desires in the human sense; it merely follows gradients in policy space, iteratively tuning its actions to maximize returns.

Yet, before dismissing these agents as merely mechanical, we must reflect: What exactly does it mean when we humans claim to "understand" something? Our sense of understanding is deeply tied to causal reasoning, semantics, and the context of accumulated knowledge. But fundamentally, isn't human cognition also driven by reward signals, dopamine hits that reinforce beneficial actions and behaviors? Aren't we, at our core, sophisticated reward hackers, shaped by millions of years of evolutionary pretraining?

Indeed, unlike RL agents who begin training from a random initialization, humans enter the world pretrained. Evolutionary pressures have finely tuned our instincts, biases, and intuitions over thousands of generations. We effortlessly generalize from minimal examples, reason by analogy, and rapidly adapt to new environments, skills RL agents must painstakingly acquire through immense trial and error.

This juxtaposition offers two intriguing possibilities:

First, perhaps RL agents aren't simplistic but merely at the earliest stages of a long evolutionary trajectory toward genuine intelligence. With additional structures, perhaps they could evolve layers of abstraction, theory of mind, and contextual understanding.

Second, and perhaps more unsettling, maybe humans aren't as profoundly intelligent as we like to believe. Perhaps our consciousness, emotions, and complex social structures are sophisticated forms of reward hacking, layers of storytelling and heuristics built atop basic reward-driven optimization.

Thus, the true tension lies not in whether RL agents are intelligent, but rather how we define intelligence itself. Are we setting the bar artificially high, or are we overlooking the simplicity underlying our own complexity.

In the end, reinforcement learning doesn't just challenge how we build intelligent machines, it compels us to reconsider the nature of intelligence altogether.