Facebook Is Building An AI That Can See And Remember Like A Human

Facebook announced a research project Thursday that aims to develop an artificial intelligence capable of perceiving the world like a human being.

The project, titled Ego4D, aims to train an artificial intelligence (AI) to perceive the world in the first-person by analyzing a constant stream of video from people’s lives. This type of data, which Facebook calls “egocentric” data, is designed to help the AI perceive, remember and plan like a human being.

“Next-generation AI systems will need to learn from an entirely different kind of data — videos that show the world from the center of the action, rather than the sidelines,” Kristen Grauman, lead AI research scientist at Facebook, said in the announcement.

The project aims to improve AI technology’s capacity to accomplish human processes by setting five key benchmarks: “episodic memory,” in which the AI ties memories to specific locations and times, “forecasting,” “social interaction,” “hand and object manipulation” and “audio-visual diarization,” in which the AI ties auditory experiences to specific locations and times.

These benchmarks, Facebook hopes, will help the AI become more individualized and personalized.

“Ego4D makes it possible for AI to gain knowledge rooted in the physical and social world, gleaned through the first-person perspective of the people who live in it,” Grauman said. “Not only will AI start to understand the world around it better, it could one day be personalized at an individual level — it could know your favorite coffee mug or guide your itinerary for your next family trip.”

The project uses a dataset consisting of more than 3,000 hours of video footage recorded from a first-person perspective by more than 850 participants from nine different countries. Participants wore GoPro cameras and Augmented Reality glasses, and they recorded footage of themselves accomplishing various tasks.

Facebook said it hopes the AI could help people recall critical information, locate objects and better accomplish tasks.