Human-Machine Interface

You were still in wonder of the latest gadget you were wearing. It’s a lot lighter than your current head-mounted-display, but that’s not what has impressed you most. It’s the clarity! Everything looks so natural. You can see overlaid computer generated visuals even at the periphery of your eyes. It’s as if the overlaid features belong to the scene. The effect is even more noticeable as you move your head around. With your head-mounted-display you could only see objects when they entered the display in front of your eyes as you looked towards them and you always struggled with focus. Despite great advances since the early days, it seemed they still couldn’t get the hang of it. But now, this retinal display was such a joy to use.

Virtual retinal display works by drawing a laser raster display directly on your retina. You get to see a conventional display in front of your eyes with a large field of view. If carried out accurately, you won’t be able to tell the difference from what you see normally. Ultimately, there will be no edges, no flicker, no sense of looking at a computer display.

You have arranged a meeting and your visitors have just arrived. You guide them into the appropriate meeting room and after the initial greetings everyone is getting themselves prepared for the meeting to start. Naturally, you want to exchange business cards with everyone and also distribute some flyers about the projects you are doing. Everyone place their mobile phones on the Display Table at the centre of the meeting room. You simply drag all the files by your finger to the mobile phones that are placed on top of the screen. The files are copied wirelessly to their mobiles. The process is secure since only those mobiles that are physically on the screen will get the files and you know that the system will take care of the rest. Sometime later, everyone is standing next to the screen hanging on the wall. Everyone is busy rotating the 3D models so they can get a feel for this new version of the plastic toy you want to manufacture for your customer. Working simultaneously, they use their fingers and use touch gestures to interact with the screen, zooming, rotating and shaping the model as they like.

Display tables are displays that are laid out horizontally with touch sensors. The sense of touch is captured with different techniques such as camera, pressure or temperature sensors.

Since you can use touch, it is believed that you will achieve certain tasks more naturally and intuitively. For example, to do a drag and drop, you may use your finger to touch the display and move one digital object from one place to another place. You can also use the device collaboratively with someone else to exchange digital material. The beauty of this interface is that you won’t need any instructions to operate a new software service. You can use your intuition to find your way around and interact with objects displayed.

1. What is it about

You are sitting in a train. You were lucky to find a seat with a table in front of it. Great! Now, you can get some work done. You are travelling light. You just want to note down some of your thoughts from the previous meeting before you head for the next one. You take out you iTube. It is a cylindrical object with a few buttons on it. You press one of the buttons and roll out a fairly large display until the device clicks. Done! You put it on the table. It’s time to reach for the other gadget in your pocket. This one is also wrapped. You unwrap the keyboard and put it next to your display. You press a button and all devices come out of sleep. The display is now on. You tap on the display to activate an application. No you start typing away. You always preferred your own keyboard. The keyboard and the display are wirelessly connected to the PDA which is somewhere in your pocket. You are online … You are glad you didn’t have to bring that heavy laptop again. This is so much better.

You are in a science museum wondering about the complexity of the mechanical clock in front of you. It’s simply amazing. It has got your attention. You want to know more about it. The clock is almost 100 years old, but its enormous and has so many parts. You are wondering how it actually works. What turns what, and what is that cylinder for? You switch to augmented reality to get a better idea. Now you can see the names of the components overlaid on what you see in the museum. The gears are drawn with a different colour. You make a gesture to open the box behind the face of the clock and now you can see how the gears inside look like. Then you make another gesture and the gears start rotating. Aha! Now you understand the role of that cylinder. So that’s how it works …

Augmented reality is a display system that combined the reality with virtual data. The system is interactive in real-time and virtual objects are generated and inserted into the world as 3D items. Augmented reality is strongly related to Location Based Information Delivery in terms of combining digital information with real world images. The difference is that augmented reality mainly deals with 3D objects and is interactive.

Some believe that reality, augmented reality, augmented virtuality and virtual reality, belong to a Virtuality Continuum where you may go from one extreme to another. In effect they all belong to Mixed Reality.

The scene in Minority Report where Tom Cruise uses gestures to search through a large amount of visual data to find what he is looking for is perhaps the best representation for this technology. Hand gestures and body movements have far more variety than the current mouse and keyboard-based input systems. Gestures can be executed faster and we are more natural at doing them. Controlling any computer device from PCs to mobiles to Display Walls can all benefit from this technology.