Using algorithms partially modeled on the human brain, researchers from the Massachusetts Institute of Technology have enabled computers to predict the immediate future by examining a photograph.
A program created at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) essentially watched 2 million online videos and observed how different types of scenes typically progress: people walk across golf courses, waves crash on the shore, and so on. Now, when it sees a new still image, it can generate a short video clip (roughly 1.5 seconds long) showing its vision of the immediate future.
“It’s a system that tries to learn what are plausible videos — what are plausible motions you might see,” says Carl Vondrick, a graduate student at CSAIL and lead author on a related research paper to be presented this month at the Neural Information Processing Systems conference in Barcelona. The team aims to generate longer videos with more complex scenes in the future.
But Vondrick says applications could one day go beyond turning photos into computer-generated GIFs. The system’s ability to predict normal behavior could help spot unusual happenings in security footage or improve the reliability of self-driving cars, he says.
If the system spots something unusual, like an animal of a type it hasn’t seen before running into the road, Vondrick explains that the vehicle “can detect that and say, ‘Okay, I’ve never seen this situation before — I can stop and let the driver take over,’ for example.” (Read more from “Deep Learning: Teaching Computers to Predict the Future” HERE)