How do you know whether a machine has truly learned? Is a high score enough to trust it? And what does that single number quietly hide about the way it got there?
Imagine two students who sit the same exam and both score 90%. On paper they look identical — but one understood the material, while the other crammed all night and guessed half the answers. Same score, very different learning. That is the problem I spent my PhD confronting — not in students, but in machines.
My field is reinforcement learning (RL), where a machine learns by trial and error: it acts, receives a reward or a penalty, and slowly works out which actions pay off — much like training a dog with treats, or a child learning to ride a bike.
For decades we have judged these agents by one thing: the reward they eventually collect. Higher score, better algorithm. But like our two students, a single score is a shallow verdict — it reveals the destination and nothing about the journey. Two agents can reach the same reward while one learned efficiently and the other simply stumbled there by luck.
A reward score tells you where an algorithm ended up, but almost nothing about how it learned to get there.
My thesis, Multi-faceted Performance Metrics for Reinforcement Learning, argues for a report card instead of a single grade — a handful of measurements that capture the different faces of good learning.
The first I call the Effort of Sequential Learning (ESL). Picture every attempt as a step across a vast map of possible strategies. A skilled learner walks almost straight toward a good one; a poor learner wanders and doubles back, covering far more ground for the same result. ESL measures the length of that journey — so two agents that finish in the same place are no longer indistinguishable.
Measure the journey, not just the destination. How much ground an agent covers to reach a good strategy reveals how well it truly explored.
A second face is task difficulty. To judge an agent fairly, we should know how hard its task really is. A popular shortcut estimates this by letting an agent flail about at random — and I showed it can mislead badly, labelling hard robotic tasks as easy and easy ones as hard, distorting every comparison built on top of it.
The last face is robustness: does the agent still perform when the world shifts slightly from its training? One that aces training but crumbles at the first surprise has not learned — it has memorised.
In conclusion, judging a learning machine by its score alone is like judging a student by a single exam mark. Measure how it explored, how hard the task was, and how well it copes with surprise — and you finally see whether it learned, or merely scored.
If you would like the full technical account, my complete PhD thesis is available HERE.