Mark Zuckerberg builds an AI assistant to run his house — and entertain his toddler

Mark Zuckerberg has a new housemate: Jarvis, an artificial intelligence assistant he created this year that can control appliances, play music, recognize faces and, perhaps most impressively, entertain his toddler.

The Facebook founder spent 100 hours putting together the virtual assistant — named after the artificial intelligence system in “Iron Man” — which understands spoken commands as well as text messages, he wrote in a 3,000-word Facebook post Monday.

Among Jarvis’s skills: adjusting the home thermostat, turning on lights and operating the toaster. The virtual assistant texts Zuckerberg images of visitors who stop by during the day and opens the front door for those it recognizes. It can also tell when Zuckerberg’s 1-year-old daughter, Max, wakes up “so it can start playing music or a Mandarin lesson,” he wrote.

In a tongue-in-cheek video he posted Tuesday on Facebook, Zuckerberg offers an example of Jarvis at work: “Max woke up a few minutes ago. I’m entertaining her,” the virtual assistant (voiced by Morgan Freeman) tells Zuckerberg, before turning his attention to the toddler. “Good morning Max, let’s practice our Mandarin.”

The year-long project was part of an effort to learn about the state of artificial intelligence, Zuckerberg wrote, and also an opportunity to experiment with cutting-edge technology at a time when voice-activated assistants like Amazon’s Echo and Google Home are gaining widespread popularity. (Amazon.com founder Jeffrey P. Bezos owns The Washington Post.)

At this point, I mostly just ask Jarvis to “play me some music” and by looking at my past listening patterns, it mostly nails something I’d want to hear. If it gets the mood wrong, I can just tell it, for example, “that’s not light, play something light”, and it can both learn the classification for that song and adjust immediately. It also knows whether I’m talking to it or Priscilla is, so it can make recommendations based on what we each listen to. In general, I’ve found we use these more open-ended requests more frequently than more specific asks. No commercial products I know of do this today, and this seems like a big opportunity.

[Google’s smart home hub is promising, but still a work in progress]

Building the robot was the easier — and less time-consuming — of his two goals for the year, he said. The other was to run 365 miles in 2016.

“Now I have a pretty good system that understands me and can do lots of things,” Zuckerberg, adding that he’s tried to give his robot a sense of humor. “I’ve taught it fun little games like Priscilla or I can ask it who we should tickle and it will randomly tell our family to all go tickle one of us, Max or Beast. I’ve also had fun adding classic lines like ‘I’m sorry, Priscilla. I’m afraid I can’t do that.’”

But there are also some kinks to work out, particularly around voice commands. When Zuckerberg demonstrated the technology for a Fast Company story, he had to ask the robot to turn off the lights four times before it complied. Shutting down the music took another two tries. (“Wow, that’s like the most fails that it’s ever had,” the 32-year-old told the reporter, visibly embarrassed.)

Next up, Zuckerberg plans to create an Android app for the robot and connect it to more appliances around the house, such as his Big Green Egg grill. The ultimate challenge, he says, is “to build a system that could learn completely new skills on its own.”

”In the longer term, I’d like to explore teaching Jarvis how to learn new skills itself rather than me having to teach it how to perform specific tasks,” he wrote. “If I spent another year on this challenge, I’d focus more on learning how learning works.”