AI recreates Super Mario Bros just by watching a two-minute clip of it

12 Sep 2017

Mario’s brother, Luigi. Image: Toca Marine/Shutterstock

A new AI is incredibly adept at seeing how a video game plays and figuring out how its engine is built, and this is big news for designers.

The video game industry is bigger than ever with mobile games, console and PC games, and even virtual reality games. With this comes the pressure of tight deadlines to get a product out to the public in time.

However, a new artificial intelligence (AI) developed by Georgia Institute of Technology researchers could make designers’ lives an awful lot easier, at least during the testing and application stage.

Presenting the findings at a recent AI conference, the team led by PhD student Matthew Guzdial explained that within less than two minutes of watching gameplay, the AI could build its own model of how the game operates.

It does this by studying the frames and making predictions of future events, such as what path a character will choose or how enemies might react.

This helps the AI learn the basic software of the game, which governs everything from character movement to rendering graphics.

Watching a speedrun

Researchers trained the AI by getting it to watch a video of a human player doing a ‘speedrun’ through Super Mario Bros, whereby the player attempts to reach the end of the level as fast as possible, making it very difficult for the AI to follow.

The researchers found that their game engine predicted video frames significantly more similar to those in the original game when compared to the same test on a neural network, giving them an accurate, general model using only the video footage.

“A single video won’t produce a perfect clone of the game engine but by training the AI on just a few additional videos, you get something that’s pretty close,” Guzdial said.

First AI to learn through observation

To test the AI’s game created through observation, the researchers recruited another AI to play, to see how it would do. Lo and behold, the cloned engine proved indistinguishable from the original.

Another researcher on the project, Mark Riedl, said that to the team’s knowledge, it is the first AI to learn a game engine and simulate it simply from watching it play.

He added: “Our model can be used for a variety of tasks in training or education scenarios, and we think it will scale to many types of games as we move forward.”

Mario’s brother, Luigi. Image: Toca Marine/Shutterstock

Colm Gorey was a senior journalist with Silicon Republic