1. What is it about

You are in a science museum wondering about the complexity of the mechanical clock in front of you. It’s simply amazing. It has got your attention. You want to know more about it. The clock is almost 100 years old, but its enormous and has so many parts. You are wondering how it actually works. What turns what, and what is that cylinder for? You switch to augmented reality to get a better idea. Now you can see the names of the components overlaid on what you see in the museum. The gears are drawn with a different colour. You make a gesture to open the box behind the face of the clock and now you can see how the gears inside look like. Then you make another gesture and the gears start rotating. Aha! Now you understand the role of that cylinder. So that’s how it works …

Augmented reality is a display system that combined the reality with virtual data. The system is interactive in real-time and virtual objects are generated and inserted into the world as 3D items. Augmented reality is strongly related to Location Based Information Delivery in terms of combining digital information with real world images. The difference is that augmented reality mainly deals with 3D objects and is interactive.

Some believe that reality, augmented reality, augmented virtuality and virtual reality, belong to a Virtuality Continuum where you may go from one extreme to another. In effect they all belong to Mixed Reality.

2. Where is the fun

• Being able to see inside everything as if they are transparent.
• Playing a shoot-em-up game in the real world but with virtual guns and bullets.
• Being able to see Google Earth and rich layers of GIS data placed seamlessly over the real world images. You can literally click on a building to get more information about it while you are passing by.
• Make 3D tele-conferencing with spatial perception so you can actually look at someone’s 3D avatar and talk to them directly and they can see that you are talking to them just by the orientation of your look.
• Incredibly useful for museums, classes, universities, labs and basically any educational environment.

3. What are its Applications

There are many common and specific uses for this technology. Anyone who needs to deal with complex real-world situations that involve consuming a large amount of digital data can benefit from this system.

Mechanical engineers who want to examine the complexity of a device can use CAD models overlaid onto real objects. The technology can help medical doctors and dentists for diagnosis and surgery as well as educating patients on what is about to happen before their surgery. Town planners, real state agents, architects and constructors can view the topography of maps laid out in front of them in 3D. The system is useful for round table discussions involving a map or 3D models.

4. How developed is it

Most of the development effort so far is spent on the hardware design and its portability, which is arguably an important part of the system. Current progress on Location Based Information Delivery and Virtual Worlds can be used to enhance the user experience in augmented reality. These two fields are rapidly progressing and large amounts of money are invested in them. When augmented reality devices become more ergonomic and portable, there will be a rapid rise in their popularity. There is already some progress with the introduction of consumer-based headsets such as Vuzix iWear VR920.

5. How can it be improved

The 3D realism can be improved in line with improvements in Virtual World graphics until a virtual item is inserted into the real world and which looks just as real as the rest of the environment around it. This opens us many possibilities. Suppose you are walking around some ancient ruins and are wondering how it could have looked like years ago. Using augmented reality, you can see the graphical representation of the site, with true realistic look and feel of the town overlaid on top of the ruins. Of course, you can interactively make various parts transparent to see what is inside as you move around. A prime example for this is Pompeii where overlaid 3D buildings can enhance your experience greatly.

Another aspect of the technology that may prove to be slow to pickup is the display mechanism itself. Acceptance of a wearable computer highly depends on its ergonomics. A heavy head-mounted-display may not appeal too much. A light and easy-to-wear-anywhere version of the display is essential for mass popularity of this technology. In addition, it can be used with head-up-displays and display walls when you are not mobile.

Most of the work carried out in this field requires offline pre-processing. For true augmented reality, it is necessary to be able to scan the environment, understand its 3D structure and paint virtual objects over it in real time. This requires next generation processing and graphics capability. The off-line version of augmented reality is what you can already see in special effects used in movies where 3D objects are seamlessly combined with real objects.

Another fine example is when you can combine motion detection and augmented reality in real-time to create an illusion of drawing in 3D with markers. The pen in your hand or glove is sensed and is used to change the 3D objects drawn over the real world you see.

6. What does it lead to

Eventually, this technology leads to a world where most people use augmented reality to interact with certain data types, in particular 3D objects. At their disposal will be a vast amount of digital information, overlaid seamlessly into their world enhancing their experience and educating them as go on about their lives.

With the rise of virtual worlds, the device will become more popular for use with mixed realities. Even when you are not outdoors, the device will be used mainly like a mobile platform, i.e. as a wearable computer. Hence, the conformability and ergonomics of the device will be critical in its acceptability and popularity. However, augmented reality doesn’t equate to using head-mounted-displays necessarily. It is about displaying that overlaid 3D information in some way to you. Exactly how this is achieved and what will work better in the end is yet to be seen. A prime example is perhaps Virtual Retinal Displays. We will know soon.