Defence Teleoperation and Stereoscopic Video

David Drascic

Human Engineering Research & Consulting
241 Logan Avenue, Toronto, Ontario, Canada M4M 2N2
drascic@ie.utoronto.ca

Julius J. Grodski

Defence and Civil Institute of Environmental Medicine
PO Box 2000, Downsview, Ontario, Canada M3M 3B9
jul@dretor.dciem.dnd.ca

Proc SPIE Vol. 1915, Stereoscopic Displays and Applications IV,
pages 58-69, San Jose, California, Feb 1993.

(c) Copyright 1993.

Abstract

This paper examines whether the potential benefits outweigh the expected costs of using stereoscopic video (SV) instead of monoscopic video (MV) for hazardous materials teleoperation. The first part presents the various benefits ascribed to SV, discusses why these benefits should apply to teleoperation, and outlines the expected costs. The second part presents two laboratory experiments showing that using SV can dramatically decrease learning time and improve performance. The third part presents two operations-like experiments conducted with trained telerobot operators of a variety of skill levels; it demonstrates that operators strongly prefer SV, consider it significantly better for most teleoperation tasks, and rate SV to be more useful and more comfortable to use than MV.

Introduction

There are many tasks which are hazardous to human life that could be accomplished remotely using telerobotic manipulators. The need to work with hazardous materials, whether nuclear, chemical, or military, and in extreme and hostile environments, is becoming more and more relevant to modern society and its economy. Efficient deployment of teleoperated and telemanaged robots will be essential for future operations in all of these areas. Currently, most telerobotic systems are completely manual, requiring constant human attention for both high level tasks, such as image interpretation and decision making, and low level tasks, such as driving from one location to another and simple obstacle avoidance. Work is progressing in off-loading some of these low level tasks from the human to the machine, so that the telerobot is semi-autonomous (Drascic et al, 1993, Zhai & Milgram, 1991). In more distant future, fully autonomous robots will be available that can carry out complex operations in unstructured environments without human intervention.

The goal of this work is to examine whether the benefits of using stereoscopic video (SV) for defence-related teleoperation outweigh the costs involved. The particular task being studied is that of Explosive Ordnance Disposal (EOD), more commonly known as bomb disposal. This includes handling and disarming both military explosives and improvised explosive devices (IEDs). This paper begins with an analysis of the various benefits ascribed to SV, and an outline of the associated costs. We then discuss the relevance of laboratory-based experimental results to real world teleoperation, and present a methodology that stresses the importance of expert evaluation as a robust and powerful analytic tool for field trials. Finally, we present two experiments that used trained operators of a variety of skill levels, seeking confirmation that the expected benefits of SV apply to real world operations.

Limitations of Monoscopic Video

The main source of information to the operator in most existing telerobots is one or more monoscopic video (MV) displays. MV displays eliminate all binocular depth cues (i.e., eye convergence and retinal disparity), as well as some monocular depth cues (i.e. texture gradient). The loss of these important depth cues results in situations where the location of objects in the remote scene is ambiguous. Motion parallax or multiple views can sometimes resolve these ambiguities, but operating conditions typically render these options unfeasible.

A related problem is the difficulty in estimating sizes with an MV system. It is difficult to determine whether an obstacle is too steep to climb, a depression is deep enough to present a hazard, or a door is wide enough for the telerobot to pass through. These deficiencies led to one British study (Robinson, 1984) reporting that standard MV systems made bomb squad personnel reluctant to use their remote manipulator.

Benefits of Stereoscopic Displays

Stereoscopic displays can be used to overcome the limitations of monoscopic displays. Careful implementation is required, where symmetry, precise camera matching (luminance, chrominance, focal length, etc.), and precise camera system alignment must be achieved and maintained. (Starks, 1992, Beldie & Kost, 1991, McGovern, 1987) When implemented properly, SV provides an immediate and compelling sense of depth, which can greatly simplify teleoperation tasks, particularly those requiring delicate manipulation. The literature reports a variety of benefits in using SV displays, including:

Particular Benefits for Defence Teleoperation

Defence related teleoperation tasks, such as bomb disposal and hazardous materials management, are characterised by the need to be conducted successfully on the first attempt while operating in an unpredictable and changing environment. The luxury of repeating a task several times is usually not an option. Thus it is reasonable to expect the benefits of SV to be significant and important even for very simple tasks. For difficult tasks, SV will likely mean the difference between success and failure.

Two potential benefits of using SV for defence teleoperation not previously discussed include:

Costs of Stereoscopic Video for Teleoperation

While the benefits of using SV for teleoperation appear quite convincing, it is important to consider at what price these benefits come. The costs associated with replacing an existing MV set-up with SV can be broken down into hardware, operational, user, and social costs.

Hardware Costs

Operational Costs

User Costs

Social Costs

These costs are clearly not inconsequential, and may outweigh the benefits of SV, depending on the particular circumstances of the teleoperation task.

Balancing the Costs and Benefits

The benefits of SV expected for defence teleoperation include faster and more accurate performance with fewer errors, with considerably less training required over an operator's career. The cost of failure in a bomb disposal task or hazardous materials handling tasks can be very high, in terms of damage to the telerobot, the environment, and life around the site. Furthermore, the expensive training and practice time for the telerobot operators to acquire and maintain sufficient skill at interpreting the monoscopic cues of an MV display will likely be considerably reduced with SV. If these expectations prove valid in actual field use, it would be prudent to invest in SV for defence teleoperation.

The Experiments

Two experiments were conducted using trained telerobot operators with a variety of skill levels, seeking confirmation that the expected benefits of SV apply to real world field operations. The first experiment was conducted under field-like conditions with typical operators, who as Munitions Technicians have training but little daily experience using the telerobot. The second experiment was conducted under more controlled conditions with expert telerobot operators.

Human-Machine System Performance Evaluation

All of the benefits of SV described above were obtained either by theoretical analysis or by experiments conducted in a laboratory under limited and controlled laboratory conditions. The experimental tasks used were limited in scope and for the most part highly repeatable. Real EOD tasks are broad in scope and unique in circumstance: no two field incidents are ever identical. The question arises of whether or not it is reasonable to expect laboratory results to be relevant for actual field conditions. These laboratory experiments were designed to examine some of the skills operators need to exercise in carrying out teleoperation tasks. While these skills comprise only a portion of the total effort, their contribution is very important for the successful completion of the task. For example, a peg-in-hole task is not something typically done by EOD operators in the field, but many of the same skills are required for various portions of an EOD event. Therefore, it is reasonable to expect that the benefits of SV found in the laboratory will pertain the field.

In order to properly evaluate the benefits of SV for EOD, it is important to avoid the two typical flaws committed by many researchers conducting field trials (Hennessy, 1990). First, there is an assumption that people behave in a regular, repeatable, predictable, and explicable manner, much like machines. While tight laboratory controls may allow this assumption to stand, it fails completely in a field environment where events are unpredictable and operators must draw on resources and experience not codified in advance by the experimenter.

The second flaw occurs when an experimenter attempts to use laboratory techniques in field situations (Hennessy, 1990). Standard statistical methods were developed to study highly repeatable natural and mechanical systems that did not have an active and unpredictable human intelligence. When analysing human-machine systems, laboratory techniques require tight controls and a strictly limited scope. They are insensitive to variations in circumstances and the unpredictability of the field and most importantly, they ignore situation and context when attempting to understand system behaviour. Attempting to use these techniques to measure performance causes the largest part of the data variability to remain inexplicable. These techniques cannot isolate various contributing factors, and are not comprehensive: they can miss important factors.

The proper way to appraise performance in the field is to use the assessment of human experts with experience in the task being studied (Hennessy, 1990). The expert ratings can then be subjected to statistical analysis to test for consistency in response. If the experts are in agreement, their opinions and interpretations are a reliable and powerful analytic tool. In these experiments, the human experts referred to are the EOD technicians themselves. Previous investigations into SV have used this approach, both directly by gathering human ratings (Gorski, 1992), and indirectly, by observing which system is used by an operator when both are available (Spain & Holzhausen, 1991, Dumbreck et al, 1990). Our experiments use the direct approach of gathering operator ratings.

Stereoscopic Video System Configuration

The field of view of the SV system was matched to that of the standard black and white camera with which the telerobots were equipped, in order to allow the operators to make as fair a comparison as possible between the SV and the MV systems. We used Panasonic GP-KS102 1/3" CCD cameras equipped with 6 mm c-mount lenses. The standard displays of the telerobots used, Hitachi 9" low resolution monochrome displays. In order to give suitable stereoacuity, the cameras were separated by approximately 12 cm, converging at the tip of the manipulator arm, approximately 80 cm from the cameras.

The SV system used the alternating field encoding approach for transmission and display. (Milgram et al, 1990) The video field alternator used was developed by Dr. Paul Milgram of the University of Toronto and the authors. A "3D-TV StereoDriver Model 1000" was used to control Nintendo (3D-TV Model N) liquid crystal shuttering spectacles (Starks, 1993).

Experiment 1: Operators in Field Conditions

Subjects

The first experiment had 8 subjects of self-rated moderate to good skill level with the telerobot. This group of seven men and one woman were munitions technicians, with various degrees of training and experience with the telerobot. They constituted a typical pool of operators. As is typical for field trials, this experiment was conducted under very strict time, personnel, and equipment limitations.

Method

A full factorial within-subjects design was used to ensure that all subjects experienced all conditions, and that the exposure was counter-balanced. Subjects performed all tasks in both MV and SV conditions, in order to be able to make a fair comparison. Each subject participated in two experimental sessions of approximately two hours on separate days. They would use either MV or SV for the first session, and the other for the second session. In each session they would perform each of the tasks described below once.

The two telerobots were used in this experiment were considerably different, due to circumstances typical of field trials. We chose to use the older, considerably less dependable telerobot for the SV display in order to bias this experiment against the expected benefits, using a well-behaved new one for the MV display. In this way, if the expected benefits were found, it would indicate that they were robust to such problems. As reported by our operators, the older robot was the direct cause a significantly higher number of performance errors with SV than might otherwise have been expected. While standard laboratory experimental analysis would be ruined by this uncontrollable influence, the field trial approach used was able to isolate this factor from the effects of the video system. This technique is clearly a more powerful, robust, and reliable analysis method for field situations.

Tasks

Three tasks were chosen to represent many of the skills required to successfully control the telerobot in the field. They were based on EOD procedures, but did not adhere strictly to them.

Expectations

Given the previous work showing that SV has considerable advantages for almost all teleoperation tasks, we expected that the EOD operators would respond favourably to it, and would feel more capable of controlling the telerobot. We hypothesised that the subjects would rate both systems equally comfortable. Since subjects were instructed to perform the tasks according to standard procedure, there was no time pressure, and so we expected that there would be no significant difference in task time between the MV and SV. Prior to the experiment, we hypothesised that there would be fewer performance errors with the SV system. However, given the difference between the two telerobots used in the trials, this hypothesis was discarded.

Experiment 2: Experts, Controlled Conditions

Subjects

The second pool of 8 subjects of self-rated good to expert skill levels were experts at EOD. Several were instructors.

Method

A partial-factorial within-subject design was used. The subjects participated in a single experimental session. The identification task described below was performed twice at the beginning of the session, once with MV and once with SV. The other three tasks were each performed once as a set with MV, then as a set with SV, forming one block of two sets. Each block was repeated 5 times. The entire session took between 3 and 4 hours to complete. A single telerobot, equipped with a switch-able MV/SV video system as described above, was used for both video conditions.

Tasks

Measures

The main measures in this experiment were the rating by the operators regarding the usability and comfort of the MV and SV systems, and a rating of preference. Other measures include task completion time, number of errors, and subject ratings of performance and confidence.

Expectations

It was expected that these subjects, being more skilled than the previous group, might show less of a preference for SV, but would still prefer it. Again, we hypothesised that there would be no difference in comfort rating. The repetition of the tasks under both video conditions 5 times imposed some measure of laboratory control on the experiment, and we hypothesised that tasks would be performed more accurately and with fewer errors with SV.

Observations

The subject ratings of Experiments One and Two are presented in Figures 1 through 4 and discussed below. In summary, all of the operators responded very positively to the SV system. They expressed a strong desire to have the entire telerobot fleet equipped with SV. They reported that SV was more useful than MV, and at least as comfortable to use. Statistical analysis of the responses indicate a high degree of consistency in most measures (Table 1). There were no significant effects of order of exposure: those who used SV in their first session gave essentially the same ratings as those who used MV.

As expected, there were no significant differences in the task time or error rates for the different tasks in Experiment One. However, performance advantages for SV were found in Experiment Two. These are summarised in Table 2.

Operator Assessment of SV for EOD

The operators in both experiments were asked a number of questions to determine their objective evaluation of the two video systems. The operators showed the typical naive trends of tending toward the mean and avoiding the extremities, no matter how much these extremes may be warranted. The questions that were asked and their corresponding answers are presented below. A summary of the results is given in
Table 1: Mean Operator Ratings
Question 1: "How would you rate the video system you used with respect to usability, i.e. how well it helped you get the job done, where 1 is terrible, 4 is acceptable, and 7 is excellent?"

FIGURE 1: Usability Ratings
Question 2: "How would you rate the video system you used with respect to comfort, where 1 is terrible, 4 is acceptable, and 7 is excellent?"

FIGURE 2: Comfort Ratings
Question 3: "If you had to choose between having either an MV or a SV system, but not both, how would you rate your decision, where 1 means you strongly prefer MV, 4 means you have no preference, and 7 means you strongly prefer SV?"

FIGURE 3: Forced Choice Between MV and SV
Question 4: "If the telerobot was equipped with both MV and SV system which you could toggle between with the touch of a switch, how much of the time do you think you would have the telerobot in the SV mode, where 1 is never, 4 is half the time, and 7 is always?"

FIGURE 4: Estimated SV Usage if both are available

Discussion

Usability (Figure 1) is an imprecise term: it implies how well something affords, or facilitates, its intended use. It is not quantifiable per se, and cannot be used with traditional automatic data gathering techniques. However, it is a term that all the operators understood clearly. The operators that participated in Experiment 1 are highly trained munitions technicians, but since the EOD telerobot is rarely used by them, few had much day-to-day experience. The expert operators of Experiment 2 were instructors, with considerably more experience with the telerobot. It is clear from the ratings of both experiments that SV was considered much more useful for the EOD tasks than was MV, regardless of the skill level of the operator.

The comfort rating (see Figure 2) in Experiment 1 that indicated that SV was more comfortable than the similar MV system was unexpected. The explanation arose in the debriefing interviews, when several subjects explained that it was easier to see things with the SV display, so they spend much less time peering closely at it than they had at the MV display. That the comfort ratings in Experiment 2 showed no difference was anticipated by several subjects in Experiment 1, who indicated that the flicker of the SV was acceptable for the relatively short exposure of 1 hour, but could be less acceptable for a longer exposure. In situations requiring extended operation, the results of Experiment 2 suggest that the operators will suffer no more discomfort with SV than they would with MV.

Table 2: Experiment Two Task Results
Debriefing interviews revealed further insight into the issues explored above. In response to the question "Did you notice/suffer any headaches that you would attribute to the SV system?", all 16 subjects said "No."

In response to the question "Did you notice any visual fatigue you would attribute to the SV system?":

All subjects were asked how they would improve the displays. All said they would get rid of the flicker and the goggles. So while the ratings say that the flicker of a 60 Hz alternating field SV display is acceptable for both casual and extended use, their comments suggest that a flicker-free display would be preferred by the users.

When forced to choose between MV and SV (Figure 3), all subjects expressed a strong preference for the SV. When presented with the option of having both available and being able to switch freely between them (Figure 4), all operators said they would use SV most or all of the time. In conjunction with the usability and comfort ratings above, these ratings indicate that SV would be of considerable advantage to EOD teleoperation.

Given the nature of field trials and the inappropriateness of laboratory statistical techniques for them, it is neither surprising nor a failing of the technique used that no differences were found in Experiment 1 for the "traditional" measures of performance (i.e. task time and error rate), particularly in light of the confounding of video system with quality of telerobot. That they were found at all in Experiment 2 is an indication of how strong the benefits must be. Experiment 2 was designed to be disruptive, with no task condition re-occurring until all others had passed, and with the video system switching between MV and SV throughout, in order to prevent short term learning. Despite this, several important performance advantages to SV were found (Table 1). The estimate of weapon position, which comes from more accurately perceived slopes with SV, is particularly important: the angle with which the weapon is aimed is critical for the success of the mission. If aimed incorrectly, the weapon could accidentally detonate the IED, instead of disabling it.

Conclusion

In order to evaluate the potential benefits of using stereoscopic video (SV) rather than monoscopic video (MV) for defence teleoperation, two experiments were conducted, where operator ratings were used as the main measure of system performance. These experiments found that both naive and expert operators expressed unanimous acceptance and preference for SV over MV, and agreed that SV was superior to MV for all tasks requiring precision operation. They also rated SV more useful and more comfortable than MV.

The performance benefits of SV found in these experiments include better object identification, improved manipulation, improved weapon positioning, and improved perception of the remote environment. It was seen that SV can be used to extend the range of tasks for which the EOD telerobot can be used.

The ratings of the trained telerobot operators, and the performance advantages found, indicate that SV would be of considerable benefit to defence teleoperation.

Acknowledgement

The work described herein was done under contract W7711-0-7114/01-XSE with Supply and Services Canada for the Defence and Civil Institute of Environmental Medicine. The authors would like to thank Captain John Eng and the Canadian Forces School of Electrical and Mechanical Engineering at CFB Borden for their participation and support in this work.

References

Beldie, I P, Kost, B, "Luminance asymmetry in stereo TV images," SPIE Vol. 1457 Stereoscopic Displays and Applications II, 242-247, 1991

Clapp, R E, "Stereoscopic displays and the human dual visual system," SPIE Vol. 624 Advances in Display Technologies VI, 41-52, 1986

Drascic, D, "Skill Acquisition and Task Performance in Teleoperation using Monoscopic and Stereoscopic Remote Viewing," Proceedings of the Human Factors Society 35th Annual Meeting, 1367-1371, 1991

Drascic, D, Grodski, J, Milgram, P, Wong, P, Ruffo, K, Zhai, S, "ARGOS: A Display System for Augmenting Reality," in the Video Proceedings of INTERCHI `93 Conference on Human Factors in Computing Systems, Amsterdam, 1993

Dumbreck, A A, Abel, E, Murphy, S P, "3-D TV System for remote handling: development and evaluation," SPIE Vol. 1256 Stereoscopic Displays and Applications, 226-236, 1990

Eichenlaub, J B, "Developments in autostereoscopic technologies at Dimension Technologies, Inc.," SPIE Vol. 1915 Stereoscopic Displays and Applications IV, 1993

Gorski, A M, "User evaluation of a stereoscopic display for space-training applications," SPIE Vol. 1669 Stereoscopic Displays and Applications III, 236-243, 1992

Hennessy, R T, "Practical Human Performance Testing and Evaluation," Chapter 15 of MANPRINT: An Approach to Systems Integration, H R Booher, ed., Van Nostrand Reinhold, New York, 1990

McGovern, D E, "Current development of needs in the control of teleoperated vehicles," SANDIA National Laboratories Report SAND87-0646 UC-15, Albuquerque, New Mexico, 1987

Merritt, J O, "Often-overlooked advantages of 3-D displays," SPIE Vol. 902 Three-Dimensional Imaging and Remote Sensing Imaging, 46-47, 1988

Merritt, J O, "Perceptual training with cues for hazard detection in off-road driving," SPIE Vol. 1457 Stereoscopic Displays and Applications II, 133-138, 1991

Milgram, P, Drascic, D, Grodski, J, "A Virtual Stereographic Pointer for a Real Three Dimensional World," INTERACT `90, Third IFIP Conference on Human Computer Interaction, Cambridge, 1990

Pastoor, S, Beldie, I P, "Subjective assessments of dynamic visual noise interference in 3D TV pictures," Proceedings of the Society of Information Display, 30(3), 211-215, 1989

Pepper, R L, Smith, D C, Cole, R E, "Stereo TV improves operator performance under degraded visibility conditions", Optical Engineering, 20(4), 579-585, July/Aug. 1981

Robinson, M., "Remote control vehicle guidance using stereoscopic displays," Proceedings of the Annual Meetings of the Human Factors Society, 1984

Scheiwiller, P M, Murphy, S P, Dumbreck, A A, "A compact zoom lens for stereoscopic television," SPIE Vol. 1457 Stereoscopic Displays and Applications II, 2-8, 1991

Spain, E H, Holzhausen, K P, "Stereoscopic versus Orthogonal View Displays for Performance of a Remote Manipulation Task," SPIE Vol. 1457 Stereoscopic Displays and Applications II, 103-110, 1991

Starks, M, "Low cost universal stereoscopic virtual reality interfaces," SPIE Vol. 1915 Stereoscopic Displays and Applications IV , 1993

Starks, M, "Stereoscopic Video and the quest for virtual reality," SPIE Vol. 1669 Stereoscopic Displays and Applications III, 216-227, 1992

Zhai, S, Milgram, P, "A telerobotic virtual control system," SPIE Vol. 1612 Co-operative Intelligent Robotics in Space II, 1991