Summary: Artificial General Intelligence (AGI) is still a ways off in the future but surprisingly there’s been very little conversation about how to measure if we’re getting close. This article reviews a proposal to benchmark existing AIs against animal capabilities in an Animal-AI Olympics. It’s a real thing and just now accepting entrants.
Artificial General Intelligence (AGI) is something that many AI researchers have an opinion about but with surprisingly little consistency. We believe broadly that achieving human-level AGI requires a system that has all of the following:
OK, so those are the characteristics described in science fiction. We’d probably think we are pretty close if it could just:
There’s also quite a wide range of opinion about when we’ll achieve AGI. Just about a year ago this time we reported on a panel included in the 2017 conference on Machine Learning at the University of Toronto on the theme of ‘How far away is AGI’. The participants were an impressive group of 7 leading thinkers and investors in AI (including Ben Goertzel and Steve Jurvetson). Here’s what they thought:
There’s significant disagreement but the median is 23 years (2040) with half the group thinking considerably longer.
How Do We Measure Progress and Not Just Final Success
Needless to say full achievement of either of those lists is a tall order. Not all researchers agree to what degree these characteristics are necessary and sufficient before we declare victory. After all we are on the journey to achieve AGI and no one has yet actually seen the destination.
Several tests of final success have been proposed most of which you’ve probably heard of.
Curiously there don’t seem to have been any significant new tests of final success added in the last 7 years. Is the matter settled?
Actually what seems not to be settled is how to measure our progress toward these goals. Like most progress in our field we ought to be able to see these successes coming some years in advance as incremental improvements allow better performance. But how do we tell if we’re 50% there or 75%?
The Animal-AI Olympics
One interesting approach was floated this last February in a project partnership between the University of Cambridge Leverhulme Center for the Future of Intelligence, and GoodAI, a research institution based in Prague. Their thought is to benchmark the current level of various AIs against different animal species using a variety of already established animal cognition tasks. Hence, the Animal-AI Olympics.
In June they announced the details of what these tests would be and are now taking submissions from potential competitors. They propose that the following 10 tests represent increasing levels of difficulty and therefore sophistication in reasoning, for both animal and AI.
This strikes me as valuable to know but not particularly definitive in predicting how far we’ve progressed toward AGI. It also seems to focus exclusively on reinforcement learning. My guess is that 8 out of 10 AGI researchers would probably say reinforcement learning is the most likely path, yet we shouldn’t rule out a breakthrough coming from other efforts like spiking or neuromorphic chips or even literal biological wetware chips.
I’m thinking that those General Dynamics robots get to at least number 6 on that scale and maybe a little higher. Still, it’s good to see someone put a stake in the ground and take a shot at this. I’m anxious to see the results.
Other articles on AGI:
Other articles on Spiking / Neuromorphic Neural Nets
About the author: Bill is Contributing Editor for Data Science Central. Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001. His articles have been read more than 2 million times.
He can be reached at: