1. What is it About

You step in the zone. You are in a jungle. You can hear sound of strange creatures around you. You draw your sword. The fun is to about to begin. There will be lots of hack and slashing. Your pulse is getting faster. Your senses are sharp and you are tense. From the corner of your eye you see some movement. This is it! You raise your sword, spin around, make a double duck move while pressing the button on the sword which makes you fly three times higher than normal and land behind the creature. Then, woosh, and its head rolls down … Wooh you have made it. You wipe your sweat off your forehead. Let’s move on.

An immersive 3D capture device can monitor your movements in the real world and map them to the virtual world you are interacting with.

2. Where is the Fun

• At last, you can do away with keyboard, mouse and joysticks when you are in a virtual world. Instead you can use your body by walking, running, natural gestures and general body movements to command your avatar.
• You no longer need to memorise all sorts of commands. If you want to use a sword, you use it like you would in the real world. No more strange key-combinations.
• Tele-conferencing while walking around the office and seeing others walking around theirs makes the experience more natural.
• You can make your avatar dance exactly the same way as you dance. Even better, you can guide your avatar by making dance moves, but the avatar performs a much more professional dance than you do.
• You can build a model of a city using digital 3D blocks by picking them up and placing them in the scene as if they actually exist. It is a natural process and you can do all the placements intuitively. You can even the shape of the blocks dynamically before they are placed.

3. What are its Applications

• You can immerse yourself in any virtual world with a universal interface that is intuitive to you.
• You can use it to interact with complex mechanical machines shown in 3D.
• It can be used in 3D modelling, CAD, animation, art and even music to use your body movements to interact with the computer.
• Sport and exergaming such as Yoga can be performed when you are monitored by the immersive device and your movements matched to a baseline for comparison. The system can give you recommendations and plan your next moves based on your capabilities. EyeToy: Kinetic is a step towards this direction.

4. How Developed is it

There are three main types of input system:

• Non-visual Sensors. Shoe sensors are used to sense every step a user takes and the visuals are updated accordingly. By using minor shifts on the floor you can change direction. Other well known systems such as Wii fall into this category and no doubt will be enhanced in the future. Full body capture devices are already used by animation industry.

  

• Motion capture. European project CyberWalk attempts to use three levels of trackers to achieve the task.

1. Track the user’s head mounted display to show the correct part of the virtual world in the displays. Eye trackers can also be used for this level.
2. Track the user’s 2D position using an overhead camera.
3. Use a series of cameras to track the posture and gestures of the user, without using markers.

Of course motion capture systems using markers can also be used, though they are currently expensive to setup, require body suits with markers and a time-consuming calibration stage.

• Mechanical walking capture. Dedicated mechanical devices can capture your movements while allowing you to walk freely on a treadmill or sliding surface. These devices such as the near-omni-directional treadmills by Virtual Space Devices (see video), or VirtuSphere will have a niche market, thought it is doubtful if they become mainstream due to their large space requirements, cost, difficult installation and safety issues.

In addition there are also other immersive devices that try to enhance the experience of the user by providing ergonomic chairs, displays and vibration facilities. Examples are, Ultimate Game Chair and triple monitors for full visual immersion.

5. How Can it be Improved

The most important requirement for these devices is their practicality. They should be easy to install by anyone, quick to learn and easy to use. Since people expect to use them intuitively, if users find them more difficult than other type of interfaces such as mouse, they many not be so keen to use the immersive devices.

It is also critical not to encourage unnatural walking habits and postures, especially since these systems will be used by children. Hence there are many challenges to overcome both technologically and psychologically. In any case, only those systems will succeed that have been tested and evolved by getting users to participate in the development. Don’t expect a novel mechanism to suddenly come to market. This technology will be developed in baby-steps, incrementally over fair amount of time.

6. What Does it Lead to

Virtual worlds are going to be big and among us very soon. More and more hours will be spent on them as they become an integral part of our life, just as the Internet itself has become now. Devices that can help you use these systems more efficiently will become popular and more resources will be allocated to their development. Ultimately you want to be able to explore these virtual worlds, as if you are actually in them and that your movements are translated realistically to these worlds and vice versa. The biggest challenge will be tactile sensing, to create the illusion of touching something in the virtual world and getting a real physical feeling from it.