Bridging the Gap between the Digital and Physical Worlds

What happens when we want to look up the meaning of a word, check for the nearest shoe store or find the number for pizza delivery? We run to our computers, and look it up. Have you ever felt helpless when you’re stuck in a situation where you really need to look something up, but a computer isn’t available to you then? If you have, then Pranav Mistry’s invention of the ‘sixth sense technology’ will not fail to fascinate you. This is truly one of the most brilliant scientific inventions in the history of mankind; not because it can perform complicated tasks, but because it simply tries to link the two worlds that are currently separate- the physical world, and the digital world.

Pranav Mistry is an MIT student who pulled this technology together after extensive research. His research was based entirely on a simple concept that intrigued him- how we should be able to interact with the computer the way one interacts with the physical world- through objects and gestures. Initially, his research began around eight years back with relatively simple devices. He opened different computer mouses, took the rollers out of them, and placed them in a line attaching them by some strings, pulleys and springs. What he got finally was a gesture interpretation device. This device interprets the gestures we make in the physical world, and translates it into the digital world. Some of his other early explorations included making a pen that could draw in three dimensions, and a sort of google map in the physical world, where if we place a boarding pass on the map it’ll show us our flight gate or if we place a coffee cup it’ll show us the nearest place to get coffee.

Originally, as his explorations demonstrate, he was interested in making the computer interface more intuitive. However, afterwards, he realized that people are more interested in information rather than computing. Sometime last year, he started thinking of his approach in a different way- how about taking the digital world to the physical world? He created a handy device that one can wear on his head; which will carry around the digital world to wherever one goes in the real world. This device consisted of pixels and a camera that would help us interact with those pixels. It is now come to be known as the sixth sense device.

This device basically helps us use any object as an interface- the palms of our hand, or a blank wall. The camera interprets the gestures we make. So if we make a gesture with our hands like clicking a photo, the device actually takes a photo. By just pinching your fingers, you can zoom in and out of a map that is displayed, unbelievably enough, on a wall. Even more interestingly, this device not only interprets your gestures, but it also interprets what you are holding in your hands. If you are in a library and you pick up a book, the device instantly searches in an online library and find this particular title, then providing you with editions, book reviews, and other information. Rather than pulling out your Iphone and looking up to see if your flight is delayed, scan this device over your boarding pass and the device looks up the information and flashes it on the boarding pass itself!

You can watch movies, play games or even surf the net using just a blank sheet of paper as an interface. One incredibly useful and striking function is being able to pinch information or pictures from different books and placing it together on one interface- either the computer screen itself, or a paper. The days of cut +copy +paste are long gone!

The unique thing about this technological invention is that rather than just moving forward in the digital world, he has tried to bridge the gap between the technical world and the world that has existed for several centuries already- the physical world. A lot of companies are interested in adopting different features of this device; for instance mobile phone companies want adapt one aspect of it, while NGOs in India want to use it as a ‘fifth sense device’ to help people who lack one of the five senses.

So it is only a matter of time before we get to walk into stores and purchase this amazing latest invention. And rather than being complicated unlike most of the latest technologies, this one is actually intuitive, easy to use and links our two worlds together.

Niyantri Ravindaran

[Image courtesy:]