Following in the wake of recent neuroscientific discoveries revealing the importance of dreams for memory consolidation, Google’s AI company DeepMind is pioneering a new technology which allows robots to dream in order to improve their rate of learning. Not surprisingly given the company behind the project, the substance of these AI dreams consists primarily of scenes from Atari Video games. DeepMind’s earliest success involved teaching AI to play ancient videos games like Breakout and Asteroids. But the end game here is for robots to dream about much the same things humans do – challenging real world situations that play important roles in learning and memory formation.
I got interested since I am an AI enthusiast with projects in place and mind, but this, got me too. I mean, next time your robot decided to startup a few minutes later, may not be a technical or software error, your robot is simply caught up in a nice dream, I wonder what these machines will react when faced with one of those nightmares where someone wakes up screaming, sweating and almost in flight, or worse still, sleep walk….
To understand the importance of dreaming for robots, it’s useful to understand how dreams function in mammalian minds such as our own (assuming the ET readership doesn’t include any aliens eavesdropping on the tech journalism scene).
One of the primary discoveries scientists made when seeking to understand the role of dreams from a neuroscientific perspective was that the content of dreams is primarily negative or threatening. Try keeping a dream journal for a month and you will likely find your dreams consist inordinately of threatening or awkward situations. It turns out the age old nightmare of turning up to school naked is the rule rather than the exception when it comes to dreams. Such inordinate negative content makes little sense until viewed through the lens of neuroscience. One of the leading theories from this fields posits that dreams strengthen the neuronal traces of recent events. It could be that negative or threatening feelings encountered in the dream help to lodge memories deeper into the brain, thereby enhancing memory formation.
DeepMind is using dreams in a parallel fashion, accelerating the rate at which an AI learns by focusing on the negative or challenging content of a situation within a game.
So what might a challenging situation look like for a robot? At the moment the world’s most sophisticated AI’s are just cutting their teeth on more sophisticated video games like Starcraft II and Labyrinth, so a threatening situation might consist of a particularly challenging Boss opponent, or a tricky section of a maze. Rather than pointlessly rehearsing entire sections of the game that have little bearing on the player’s overall score, “dreams” allow the AI to highlight certain sections of the game that are especially challenging and repeat them ad nauseam until expertise is achieved. Using this technique, the researchers at DeepMind were able to achieve an impressive 10x speed increase in the rate of learning.
You might ask why AI “dreams” are necessary given that robots can already dominate humans in most games such as Chess and Go. To grasp this, it is necessary to differentiate between AIs that use supervised vs. unsupervised learning.
Most of the impressive feats so far attained by AI have been made using supervised learning, in which organized “training data” is supplied by the programmers and the AI learns to detect patterns within the data. This is a fairly straightforward approach to teaching machines but decidedly not how humans learn. We use an approach more akin to what programmers call unsupervised learning in which the agent experiments on their own to determine how different courses of action affect their goals. This type of learning if far more time consuming than supervised learning because it involves experimentation.
The folks at DeepMind are primarily concerned with unsupervised learning because it holds the best hope for creating AI with human-like general intelligence. So while it remains uncertain whether androids will one day dream of electric sheep, given the social role that we continue to envision for robots, it does seem increasingly likely that AIs could soon dream of socially awkward situations like showing up to school naked. Now exactly what a naked robot might look like is open to speculation, though Matt McMullen and his teammates at RealDolls seems to have a few vivid ideas of thier own.