TELE-IMMERSION: STEPS INTO THE FUTURE

 

Introduction

Tele-Cubicle

Tele-immersion has the potential to significantly impact educational, scientific, manufacturing, and many other fields. Tele-immersion is an advanced form of virtual reality, which will allow users at different locations to interact in real time in a shared simulated environment. This technology causes users to feel as if they were in the same room. The tele-immersion technology uses a "tele-cubicle" which is equipped with large screens, scanners, sensors, and cameras. The tele-cubicles are linked together in real-time so that they form one larger cubicle as shown on page 2.

Through the virtual environment, participants are able to interact with other people. The virtual objects and data can be passed through the walls between participants, and placed on the shared table in the middle for viewing. 

First Tele-Immersion Experiment
The first experiment of tele-immersion was done in Chapel Hill, North Carolina on May 9, 2000.Think about the Star Trek Holodeck, the projection chamber on the Starship Enterprise where crewmembers interact with projected images as if they were real. "We call it 'being there'," says Welch. [2]
This experiment was the highlight of three years' work done by virtual reality pioneer Jaron Lanier. The test linked three of the members of the group: UNC Chapel Hill, the University of Pennsylvania in Philadelphia, and a non-profit organization called Advanced Network and Services in Armonk, New York, where Lanier is chief scientist. [2]

At Chapel Hill, there were two large screens, hung at right angles above a desk, plus projection cameras and head tracking gear. 

The screens were flat and solid, but once the demo was up and running they looked more like windows. Through the left-hand screen, Welch could see colleagues in Philadelphia as if they were sitting across the desk from him. The right-hand screen did the same for Armonk. When Welch changed his point of view, the images shifted in a natural way. As he leaned in, the images got larger, if he leaned out they got smaller. He could even crane his neck to look round the people. To make it work, both target sites were kitted out with arrays of digital cameras to capture images and laser rangefinders to gather positional information. Computers then converted the images into 3D geometrical information and transmitted it to Chapel Hill via Internet2. There, computers reconstructed the images and projectors beamed them onto the screens. The images were split and polarized to create a slightly different image for each eye; this resembled an old-fashioned 3D movie. Welch wore glasses with differently oriented polarizing lenses so his left eye saw one image and his right eye the other, which his brain combined to produce 3D images. A head-mounted tracker followed Welch's movements and changed the images on the screens accordingly. Like the first phone call ever made to another continent, the quality was scratchy. It was also jerky, updating around three times a second rather than 10, the minimum speed needed to capture the full range of facial expressions. This only worked one-way: the people in Armonk and Philadelphia could not see Chapel Hill. On the other hand, Thomas Cox, UNC video services manager said: "It looks like somebody took a chainsaw and cut a hole in the wall and he's on the other side."[2]

The Problem

The Solution

The Internet2 offers super fast connections, which ensures that the data arrives at its destination without loss and on time. Abilene is a 10,000-mile backbone that was built just for Internet 2. The Abilene network, which has been up and running since 1999. Several key equipment donors made this adventure possible. Nortel Networks supplied optical networking devices and Cisco Systems supplied routers, while long-distance carrier Qwest Communications provided fiber across the United States and operates the network. All three companies have provided human resources to the project. They have also agreed to continue to support the project for at least four more years. The connection speed is 155 megabits per second, which is a hundred times faster than an university lab and 3000 times faster than a dial-up modem. Multicasting is another key to the Internet 2. 

 A live video broadcast can be transmitted across the Internet and split off copies of it to multiple destinations. The speed of the Internet 2 is unlike anything the industry has ever experienced. Take a movie like "The Matrix," the 1999 sci-fi thriller. To come up with some base comparisons, Internet2 staff downloaded the 136-minute DVD version of the film via a standard telephone modem. It took 171 hours. Over an Integrated Services Digital Network (ISDN) line, it took almost 74 hours. A typical cable modem or digital subscriber line (DSL) hookup took about 25 hours. A T1 line as used by many companies did it in about 6 1/2 hours. When they tried the exercise over Internet2, the movie downloaded in about 30 seconds. They repeated the experiment from multiple locations on the Internet2 backbone and duplicated the results. But we're not going to be downloading "The Matrix" for a while yet. That's because the consortium that administers Internet2 agreed it would be used only for noncommercial research purposes. In its initial deployment, therefore, Internet2 is being used only in high-end medical, scientific and academic applications. [3]

 The Internet we have today the originating server transmits a separate data stream to each user. More than 185 universities are using Internet 2 today. The project's leaders say it is a unique test bed for Internet applications of the future, including tele-immersion. A tele-immersion system requires high bandwidth, low latency and low latency variation (jitter), make it one of the most challenging net applications. This application is therefore considered to be an ideal driver for the research agendas of the Internet2community. 

Tele-immersion will require supercomputers to perform the trillions of calculations that are needed to portray environments in 3D. This is achieved using computers that recognize the presence and movements of individuals and objects, tracking those individuals and images, and reconstructing them onto one stereo-immersive surface.The ultimate goal of tele-immersion is not to reproduce a real face-to-face meeting in every detail, but to provide the "next generation" interface for collaborators, worldwide, to work together in a virtual environment (VE) that is seamlessly enhanced by computation and large databases. Presence in the virtual world is typically maintained using an avatar, or a computer-generated representation of a person. These avatars may be as simple as a pointer, but having physical body representations can be very helpful in aiding conversation and understanding in the virtual space because you can see where your collaborators are, and at what they are looking or pointing. Tracking the user's head and hand position and orientation allows articulated avatars to transmit a decent amount of body language, and are very useful in task-oriented situations. When participants are tele-immersed, they are able to see and interact with each other and the objects in a shared VE and can talk to each other via ambient or personal microphones. This kind of computer power would have to be on tap over the Internet. Tele-immersive environments will therefore facilitate not only interaction between users themselves but also between users and computer-generated models and simulations.This will require expanding the boundaries of computer vision, tracking, display, and rendering technologies. As a result, all of this will enable users to achieve a compelling experience and it will lay the groundwork for a higher degree of the their inclusion into the entire system.The following picture shows a picture of what an office might look like in the future.

The picture in the right hand corner shows what the office looks like prior to the projection of the three other people. Tele-immersion may seem like another kind of virtual reality, but Lanier says it is something different. Virtual reality, he says, allows people to move around in a pre-programmed representation of a 3D environment, whereas tele-immersion is more like photography. "It's measuring the real world and conveying the results to the sensory system," he says. But that doesn't mean there's no place for virtual reality within tele-immersion systems. 

The NTII researchers are working on incorporating virtual objects that can be seen, manipulated and altered by all the participants. " Meanwhile, Tom Defanti and his colleagues from the University of Illinois at Chicago are taking the marriage of tele-immersion and virtual reality a step further. In their systems, people share a virtual environment and each is visible to the others as a computer simulated entity, or "avatar". People could choose the way they look in a tele-immersion session-from changing their hair color to looking like a film star the possibilities are endless. [2] However, tele-immersion is not just a research tool. The leading fast-food chain, McDonalds showed interest in a tele-cubicle.McDonalds has a vision of fitting tele-immersion booths in its restaurants so people away from home could have dinner with their family.Is this thought far off? 

The applications for immersive holographic environments are endless. Imagine a video game that has no joysticks, in which you become a part of the game, fighting monsters or scoring touchdowns. Instead of traveling hundreds of miles on Thanksgiving to visit your in-laws you will simply call them up and join them in a shared holographic room.The only thing is you will not be able to pass the turkey across the room.Well it seems hard to believe but with technology, advancing the way it is now anything is possible.This technology could help train soldiers.When the soldiers are out on a ship for long periods of time they can have a tele-immersion session with their family.

All of this, of course, relies on other emerging technologies. Most important is the ability of the Internet to ship vast amounts of data across continents without delay. Luckily for the developers of tele-immersion, their needs are at the forefront of Internet2's thoughts. In fact, the two projects go hand in hand. Internet2 needs a raison d'etre for its increased capacity, and in tele-immersion it seems to have found one. The experiment at Chapel Hill in May was made possible by UNC's Internet2 link. "There is simply no other knownapplication that would push the network to its limits," says Lanier. [2]

3D reconstruction for tele-immersion is performed using stereo, which means two or more cameras take rapid sequential shots of the same object, continuously performing distance calculations, and projecting them into the computer-simulated environment, as to replicate real-time movement. 

Now they use an array of seven cameras for one person seated at a desk, which in practice act as five trios. On the other end, the people that appear as 3-D images are being tracked with an array of seven ordinary video cameras while two other video cameras capture real light patterns projected in each room to calculate distances. Roughly speaking, a cluster of eight two-gigahertz Pentium processors with shared memory. This enables the proper depth to be re-created on the screen.

When a viewer moves his or her head to the right, he or she can see the corresponding images that would be seen if he or she were actually in the room with the person on the screen.

The early prototypes of tele-immersive displays require users to wear special goggles and a head device that tracks the viewpoints 

of users looking at thescreen.Images on the screen are split and polarized to create a different image for each eye. The goggles then combine these images so that the brain recognizes only one 3-D image. This process is similar to how those old 3-D movie glasses work. Early experiments, like the one at UNC in May, have experienced some glitches, similar to those of normal video-conferencing.The scenes being projected are only refreshed three times per second, which creates a jerky image. If that rate could be improved to 10 frames per second, it would create a seamless projected image that would be like looking through a window at another person. Tele-immersion will blur the lines between real and computer-generated images. The key is that in tele-immersion, each participant must have a personal viewpoint of remote scenes--in fact, two of them, because each eye must see from its own perspective to preserve a sense of depth. [2] Furthermore, participants should be free to move around, so each person's perspective will be in constant motion. [2]

Benefits

Collaborative mechanical applications as well as different medical applications are two applications that will benefit from tele-immersion systems. For example, a group of designers will be able to collaborate from remote sites in an interactive design process. They will be able to manipulate a virtual model starting from the conceptual design, review and discuss the design at each stage, perform desired evaluation and simulation, and even finish off the cycle with the production of the concrete part on the milling machines.

In the case of medical applications such as tele-radiology and urgent diagnostics, the availability of such technologies in the places that are physically inaccessible to specialists could potentially save the lives.Off-shore ships and oil rigs are good examples of such environments.Expert surgeons thousands of miles away could be present in an operating room to offer counsel.Imagine that someone was to get hurt on an oil rig and need emergency surgery and there were no surgeons on board, then this could be performed in a tele-cubicle.The situation could happen and the doctor could show someone how to perform the surgery with out being there.This does seem a little risky but it could save some ones life.

With the decline in business today a company could benefit with this technology.Tele-immersion could save a company money in the long run.This technology can cut down or eliminate travel expenses.It will be the ultimate tele-commuting technology, almost eliminating the rush-hour drive to work. Instead of commuting, people could attend board meetings by projecting themselves into the company's conference room.The Internet 2 is able to send information 1000 times faster than we are able to do today.The Internet 2 will bring together institutions and resources from the industry and government to develop new technologies.

Methodology

The research that I did for this project was completed via the Internet.Searching for “tele-immersion” provided many hits for this technology.I also searched for “Internet 2” considering that is what a company or university must have in order to participate in a tele-immersion session.

Gathering Data

I used many search engines to find the relevant information that I needed for this paper.I mostly used Yahoo and Google.I was able to find many sites that pertained to this technology.I printed the information I found that way I would be easier to have a hard copy of the information in my hand for better viewing. 

Findings

The cost of an eight-processor cluster is anticipated to be in the $30,000 to $50,000 range at introduction, and a number of those would be required for each site (one for each trio of cameras)--and this does not even account for the processing needed for other tasks. They do not yet know how many cameras will be required for a given use of tele-immersion, but they currently guess that seven is the minimum adequate for casual conversation, whereas 60 cameras might be needed for the most demanding applications, such as long-distance surgical demonstration, consultation and training.Thousands of surgeries are done daily and people like to know what is going to take place when they go to sleep.With a model of the patient, one could view how the surgery was going to be preformed. entually the researchers would like to make tele-immersion even more naturalistic, perhaps by jettisoning the headgear and glasses altogether. One possibility is to use a screen that transmits different information to each eye, using swivelling pixels that track either the left or right eye. Another idea is to turn the entire tele-immersion room into a screen. Walls, tables, curtains, even floors could be coated with special light-sensitive material. Cameras would photograph the surfaces, computers would calculate their shapes in 3D, and projectors would shine pre-warped images, making it seem as if they filled the room. [2]

Conclusion

Tele-immersion is a way people can communicate on opposite ends of the country or world to feel temporarily as if they are in each other's presence. If a computer network can support tele-immersion, it can probably support any other application.Tele-immersion demands as little delay as possible from flows of information (and as little inconsistency in delay), in addition to the more common demands for very large and reliable flows. Expert surgeons thousands of miles away could be present in an operating room to offer counsel, actors in New York and Los Angeles could rehearse lines together and distance learning could become as real and en-gauging as a traditional classroom experience.

Automobile designers from Detroit and Germany could meet to conceive the next generation of sport utility vehicles.It will be the ultimate tele-commuting technology, almost eliminating the rush-hour drive to work. Instead of commuting, people could attend board meetings by projecting themselves into the company's conference room. In addition, if your job requires you to travel, you could still be home for dinner by tele-immersing yourself into the family kitchen.This would be the next best thing to being there.Executives or researchers on different continents could hold face-to-face meetings without ever boarding a jet.This is a definite plus for the world today.Although few would claim, that tele-immersion will be absolutely as good as "being there".According to researchers, business travel might even be replaced to a significant degree by this new technology in 10 years. This is not only because tele-immersion will become better and cheaper but because air travel will face limits to growth because of safety, land use and environmental concerns.Environmental issues are a big concern in our society today.The fuel exhaust is a big concern.So many people worry about flying due to the recent tragedies and this would be a safe way to meet. 

People do not have a lot of time to go places today and companies could save expenses by meeting letting their employees meet virtually. Tele-immersion will not replace e-mail, phone calls, or existing teleconferencing systems. This technology is still in the early stages of development and the possibilities are truly endless to step into the future.