ARGOS: A Display System for Augmenting Reality

David Drascic

Human Engineering Research & Consulting (HERC)

241 Logan Avenue,

Toronto, Ontario, Canada M4M 2N2

drascic@ie.utoronto.ca

Julius J. Grodski

Defence & Civil Institute of Environmental Medicine (DCIEM),

P.O. Box 2000,

North York, Ontario,

Canada M3M 3B9

jul@dretor.dciem.dnd.ca

Paul Milgram, Ken Ruffo, Peter Wong, Shumin Zhai

ETC-Lab: Ergonomics in Teleoperation and Control

Department of Industrial Engineering,

University of Toronto,

4 Taddle Creek Road,

Toronto, Ontario,

Canada M5S 1A4

{paulm,ken,peter,shumin}@argos.rose.utoronto.ca

This is the extended abstract for a video published in:

ACM SIGGRAPH Technical Video Review, Volume 88: InterCHI 1993 Conference on Human Factors in Computing Systems
Running Time: 11 minutes. A HERC Production.

This abstract was published in INTERCHI'93: ACM Conference on Human Factors in Computing Systems, p 521, Amsterdam, Netherlands, 1993.

ABSTRACT

This video describes the development of the ARGOS (Augmented Reality through Graphic Overlays on Stereovideo) system, as a tool for enhancing human-telerobot interaction, and as a more general tool with applications in a variety of areas, including image enhancement, simulation, sensor fusion, and virtual reality.

KEYWORDS: stereoscopic displays, 3-D, virtual reality, remote manipulation, teleoperation

NOTE: Portions of this video are in alternating field stereoscopic video format. Appropriate viewing equipment is needed to perceive the stereoscopic effect. [5]

Human-Robot Interaction

This project began by examining the human-machine interface of a particular telerobot, identifying two key problem areas: information flow from the robot to the operator (feedback), and information flow from the operator to the machine (commands/instructions).

Visual Feedback Interface

Most telerobots use monoscopic video (MV) as the primary feedback link for the operator. MV lacks binocular depth cues, however, hampering the user's perception of locations of objects in the remote world. Using stereoscopic video (SV) for remote manipulation can reduce task execution time, error rates, and training times. [2] Recent studies have shown that operators strongly prefer SV, and rate it more comfortable and usable than MV. [1]

Control Interface

Most telerobots are controlled manually, requiring continuous attention from highly skilled operators. Fully autonomous telerobots are not yet possible in unstructured environments, but it is feasible to transfer some of the workload from the operator to the machine [6]. A semi-autonomous robot can carry out simple movement commands if a method of communicating precise three dimensional coordinates to the robot is available. Humans are poor at absolute judgement of position, but can make accurate relative position judgments using SV displays. [3] If a pointer with an accurately known position is available, operators can use it to specify arbitrary points in the remote world, using their own sense of relative position.

We have created a virtual pointer using calibrated stereoscopic computer graphics (SG), and with our ARGOS system combine the SG image with the SV, so that the virtual pointer appears within the remote world. The operator can move it around freely and, by aligning it with objects in the remote world, can determine their position. [3] ARGOS combines human abilities of perception and comprehension with computer abilities of precise calculation and graphic displays to create an augmented system with greater functionality than either alone.

ARGOS Applications

We call our system ARGOS, meaning "Augmented Reality through Graphic Overlays on Stereovideo". Rather than trying to create a virtual or artificial reality, ARGOS serves to give operators sufficient and necessary information for carrying out tasks in a natural, spatial manner. By enhancing the information content of the SV displays with SG images, we augment their usability and functionality.

A extension of the virtual pointer is the virtual tape-measure, which can be used to measure sizes and distances in the remote world. [5] For example, the virtual tape-measure can be used to measure the sizes and positions of landmarks on a microscopic scale, such as within cells.

With sufficient graphic and computing power, it is possible to create and animate virtual objects of any complexity and realism desired. Working in structured environments, ARGOS can enhance video images by overlaying wire-frame edges on known objects. [4] In unstructured environments, ARGOS can integrate information from other sensors such as radar and sonar with the SV. Architectural visualisation, training, medical imaging and simulation are some of the fields that we are currently exploring. Machine vision capabilities are expanding the potential of our ARGOS system even further. [6]

ACKNOWLEDGEMENT

This work, and the production of this video, were carried out under contract for, and in cooperation with, the Defence and Civil Institute of Environmental Medicine.

NOTE

Portions of this video are in an alternating field stereoscopic format. Special glasses are required in order to be able to see the stereoscopic effect. Unfortunately, the published video was mangled in a conversion from NTSC to PAL and back again, so even with the special glasses you STILL can't see it in 3-D. But on the original tapes it looks very good. Honest!

References

1. Drascic, D., Grodski, J.J. "Stereoscopic Video and Defence Teleoperation", SPIE Vol 1915 Stereoscopic Displays and Applications IV, Feb. 1993

2. Drascic, D. "Skill Acquisition and Task Performance in Teleoperation using Monoscopic and Stereoscopic Video Remote Viewing", Proceedings of the Human Factors Society 35th Annual Meeting, 1367-71, Sep. 1991

3. Drascic, D., Milgram, P., "Positioning Accuracy of a Virtual Stereographic Pointer in a Real Stereoscopic Video World", SPIE Vol 1457 Stereoscopic Displays and Applications II, March 1991

4. Milgram, P., Drascic, D., Grodski, J.J. "Enhancement of 3-D video displays by means of superimposed stereo-graphics", Proceedings of the Human Factors Society 35th Annual Meeting, 1367-71, Sep. 1991

5. Milgram, P., Drascic, D., Grodski, J.J. "A Virtual Stereographic Pointer for a Real Three-Dimensional Video World", INTERACT `90: Third IFIP Conference on Human-Computer Interaction, August 1990

6. Zhai, S., Milgram, P. "A telerobotic virtual control system", SPIE Vol 1612, Cooperative Intelligent Robotics in Space II, Nov. 1991

This rest of this page is in a prototype form only, but it will be a while before I can finish it. Hopefully the pictures pique your interest enough to return at a later date when I will install a transcript of the video as well. MPEGs are certainly possible, but not really necessary. If you are interested in them, please e-mail me and I'll see what I can do. Any feedback is welcome.

David Drascic, drascic@ie.utoronto.ca

Controlling a robot equipped with monoscopic video is hard. Using stereoscopic video makes it much easier.

Various stereoscopic camera configurations

The operator wears special glasses in order to see the 3-D image

Combined left and right views look like this; the glasses separate them for the left eye and the right eye.

This shows the Virtual Pointer in operation, measuring the distance from the foreground chair to the back. Sorry the colours aren't better.

Here the Virtual Pointer draws a path in space for the robot to follow.

Here the Augmented Reality system is shown to enhance the view of the shuttle bay by drawing in a wire-frame outline of the edges within the bay, making them more clearly visible. (In the video it does, anyway.)

An example of a simple wire-frame table and chair, drawing in stereo by an Amiga. It looks much more impressive in stereo.

The operator controls the virtual robot in real time, and the real robot follows along when the operator tells it to. Good for long time delays or senstive tasks. The stereoscopic view of the "phantom robot" makes it easy to position it precisely in space, much better than is possible with similar monoscopic systems.

Here the operator moves a real robot by "grabbing" the end with a virtual hand.

Here the operator controls a virtual robot in the same manner.

Although this is hard to make out, it looks much better in the video in stereo. This frame was grabbed from a copy of a copy of copy. It shows a computer drawn desk that rises out of the floor and then flies towards the viewer, looking very impressive in the original, if I say so myself. :-)