Facebook Working On Home Automation
What does the future hold for Facebook? Well Mark Zuckerberg, the Founder of Facebook next big personal project, is looking to automating his home.
In December 2016, Zuckerberg wrote an extensive piece on “Building Jarvis” – a simple AI to run his home. Jarvis will be capable of controlling the lights, temperature, appliances, music, and security. One of the most fascinating things is that Jarvis will be able to learn his tastes, patterns, new words, concepts, and even entertains his son, Max.
If you’re a tech-head, it’s quite an interesting piece. Zuckerberg is building Jarvis in Python, PHP & Object C. He details the complexity in building such a system and realises to continue forward, technology providers need to settle on API standards.
Before I could build any AI, I first needed to write code to connect these systems, which all speak different languages and protocols. We use a Crestron system with our lights, thermostat and doors, a Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max, and of course my work is connected to Facebook’s systems. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.
For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other.
So how would you start to build in Artificial Intelligent robot? Mark broke it down into two steps. Step 1: Connect everything to his computer so it can issue commands and actually turn the light off. Step 2: Add the ability to speak and have the speech translate into text commands that the computer can understand. However, this was more of a difficult task.
It started simple by looking for keywords, like “bedroom”, “lights”, and “on” to determine I was telling it to turn the lights on in the bedroom. It quickly became clear that it needed to learn synonyms, like that “family room” and “living room” mean the same thing in our home. This meant building a way to teach it new words and concepts.
Context is also another important factor. When I say “Turn off the lights in my room”, it will mean something completely different to if my parents or siblings were to say the exact same phrase. Same with when you say “Dim the lights” – It needs to know which room you are in. However, it gets much more complicated as Mark puts it.
Music is a more interesting and complex domain for natural language because there are too many artists, songs and albums for a keyword system to handle. The range of things you can ask it is also much greater. Lights can only be turned up or down, but when you say “play X”, even subtle variations can mean many different things. Consider these requests related to Adele: “play someone like you”, “play someone like Adele”, and “play some Adele”. Those sound similar, but each is a completely different category of request. The first plays a specific song, the second recommends an artist, and the third creates a playlist of Adele’s best songs. Through a system of positive and negative feedback, an AI can learn these differences.
Next, Zuckerberg looks at Facial Recognition in regards to home security. His goal – to say, “only let my family in my home”. Security camera’s will recognise a user’s face, reference that in an approve access list, and then unlock your front door. Who will need keys these days?
What happens when your away from your home? Simple, just use Messenger. Mark created a Messenger Bot so he can text his Jarvis server to issue commands. If he wanted, he could also record audio commands.
When it comes to Artificial Intelligence, there is a lot more to explore.
The AI technology is just getting good enough for this to be the basis of a great product, and it will get much better in the next few years. At the same time, I think the best products like this will be ones you can bring with you anywhere and communicate with privately as well.
Read the full post of Building Jarvis.
