Zuckerberg realizes his dream of building an AI-assistant

Jarvis can communicate through voice or text, can recognize faces and even recommend music

Editor's note: Our Splash Health, Wellness and Wearables event is coming up on March 23 in San Francisco. We'll have Mario Schlosser, Founder & CEO of Oscar Health, Brian Singerman (Partner, Founders Fund), Steve Jurvetson (Draper Fisher Jurvetson), J. Craig Venter (Human Longevity), Lynne Chou (Partner, Kleiner Perkins), Michael Dixon (Sequoia Capital), Patrick Chung (Xfund), Check out the full lineup and register for tickets before they jump! If you’re a healthcare startup and you’re interested in being part of our competition, learn more and register here.

Also, vote for your favorite healthcare startup before February 16! Vote here!

Every year, Mark Zuckerberg comes up with at least one big goal for the oncoming year. In 2014 that was to learn Mandarin, which he is apparently pretty good at now. Last year it was to read two new books every month, allowing the Facebook community to join in.

In January, he set his most ambitious goal yet: to "build a simple AI to run my home and help me with my work." It woud be an assistant that would be able to control his entire house based on Zuckerberg's voice commands. Basically, the goal was to build a real-life version of Jarvis, the AI-butler used by Tony Stark, a.k.a. Iron Man.

With this year coming to a close, Zuckerberg finally revealed on Monday that he had done it: he had built his own Jarvis. In fact, he said, it turned out to be "easier than I expected," taking only 100 hours, which is less time than it took to complete his other goal for the year: running 365 miles.

"So far this year, I've built a simple AI that I can talk to on my phone and computer, that can control my home, including lights, temperature, appliances, music and security, that learns my tastes and patterns, that can learn new words and concepts, and that can even entertain Max," he wrote in a lengthy Facebook post called "Building Jarvis."

Zuckerberg outlined how he went out about building the AI platform, including the most complicated aspect: actually connecting devices that "speak different languages and protocols," including Creston, Sonos, Samsung and Nest, then connecting them to the Internet.

While he built Jarvis to recognize his voice, Zuckerberg also set it up to communicate through text, specifically through a bot on Messenger. While most people probably like the idea of speaking their commands out loud, Zuckerberg found that he prefers to communicate through text, since it bothers other people less.

"If I'm doing something that relates to them, like playing music for all of us, then speaking feels fine, but most of the time text feels more appropriate," he said. "Similarly, when Jarvis communicates with me, I'd much rather receive that over text message than voice. That's because voice can be disruptive and text gives you more control of when you want to look at it. Even when I speak to Jarvis, if I'm using my phone, I often prefer it to text or display its response."

Even as technology advances, some basic human social norms still have to be taken into account. That’s a good lesson for most people to remember.

While Zuckerberg said that he wouldn't have been able to build a system that "could learn completely new skills on its own," he also demonstrated that Jarvis is already getting smarter in at least one respect: it can recognize his taste in music based on his past listening patterns, and can make recommendations based on which person is asking it.

"It also knows whether I'm talking to it or Priscilla is, so it can make recommendations based on what we each listen to. In general, I've found we use these more open-ended requests more frequently than more specific asks. No commercial products I know of do this today, and this seems like a big opportunity," he said.

As for what's next for Jarvis, in the near-term that includes an Android app, and connecting it to more appliances. Longer-term, Zuckerberg does want to teach it to learn new skills by itself.

Most importantly, it also sounds like Jarvis could eventually become a part of Facebook's larger vision. The company has already begun using artificial intelligence; for example, it used AI to add audio photo captions for the blind earlier this year and Zuckerberg is now considering it as a potential new product for his company.

“Over time it would be interesting to find ways to make this available to the world,” he wrote. “I considered open sourcing my code, but it's currently too tightly tied to my own home, appliances and network configuration. If I ever build a layer that abstracts more home automation functionality, I may release that. Or, of course, that could be a great foundation to build a new product.”

Perhaps it won’t be long before every one of us has a Jarvis in the home.

(Image source: jadiberita.com)