This thesis is concerned with design factors that influence human performance in manipulating the location and orientation of three dimensional (3D) objects with six degrees of freedom (6 DOF). The need for this research has emerged from the development of a variety of advanced technologies. Technologies such as virtual and augmented reality (Barfield and Furness, 1995) , telerobotics (Sheridan, 1992b), computer aided design (Majchrzak, Chang, Barfield, Eberts, and Salvendy, 1987), scientific data visualisation (Card, Robertson, and Mackinlay, 1991) , and 3D computer graphics and animation (Foley, van Dam, Feiner, and Hughes, 1990) all require designing interfaces to let human users control 6 degrees of freedom of objects (robot, data, or viewpoint) in 3D space.
Many devices, including various instrumented gloves, position trackers, and hand controllers have been developed for applications in these areas. Figure 1.1 shows a collection of such 6 DOF devices. Ideally, the development of such devices should have been guided by the knowledge of human characteristics and performance. However, in reality, the development of most of these products has been driven primarily by sensor technologies. Whenever sensors that can detect displacement or force/torque in six degrees of freedom are developed, they tend to be quickly incorporated into new 6 DOF control devices for human input into computers and other machines. Little systematic human factors research on 6 DOF manipulation has been available for guiding the development of 6 DOF interfaces.
In order to transform this trend of "technology push (or gadget-driven)" into "demand pull (or human-driven)" in the development of interactive devices (Jacob, Leggett, Myers, and Pausch 1993) , systematic studies of human capabilities and limitations as a function of various design dimensions are much needed. Design variations in many parts of an interactive system can influence how a user conducts input control tasks. To identify the key human factors issues in designing 6 DOF inputs, Figure 1.2 illustrates the major components* involved when a user exchanges information with a computer system (or any machine in general).
Block 1 in Figure 1.2 is the physical control interface between
the user's limb and the machine (computer). This physical interface
is also called a manipulandum in many fields. The currently most
common 2 DOF example of such a physical interface is the computer
mouse. A physical interface functionally consists of two parts,
one part
Figure 1.1 A sample of input devices for 6 DOF manipulation;. (a) The "Bat", designed by C. Ware (1990), consists of a Polhemus™ tracker and a handle. (b) The Spaceball™ is an isometric device manufactured by Spaceball Technologies Inc., Boston, MA, USA. (c) The SpaceMaster™ is an elastic device with a small range of movement (5 mm in translation and 15° in rotation), manufactured by BASYS GmbH, Erlangen, Germany. (d) The Cricket™, manufactured by Digital Image Design Inc., New York, NY, USA, is a free moving device consisting of a tracker inside of a handle. (e) The Space Mouse™ is an elastic device with slight movement (5 mm in translation and 4° in rotation). It is patented by DLR, the German aerospace research establishment, manufactured by Space Control Company, Malching, Germany and marketed by Logitech, Fremont, CA, USA. (f) The MITS Glove, designed by the author, consists of a Bird™ tracker and a clutch mounted on a glove.
Figure 1.2 The human-machine interaction system
for sensing signals from the user's limb and one part, such as a handle or a glove, for the user to grasp. When narrowly defined, the term input device often refers to this physical interface. Interestingly, the information transfer between the human limb and the ìinputî device is in fact bilateral. In one direction, the user's motor actions manipulate the device and these actions are transformed into instructions for the computer. In the other direction, the user also receives certain control feel information via proprioception from the physical device. This bilateral nature of an input device can not be overlooked. An important issue is what resistance* the device should have in order to give the user a proper control feel. Should the designer choose a freely moving device (zero resistance, isotonic) such as an instrumented glove, a device with a certain type of movement resistance such as an elastic device, or a device with infinite resistance (isometric) such as the Spaceball™? How do different types of resistance affect the user's performance? To what extent does this resistance induce user fatigue? Two extreme cases, along the continuum of control resistance, an isotonic device versus an isometric device, will be studied in Chapter 2. Elastic resistance will be studied in Chapter 3.
The design of the physical size and shape of a physical device also has implications towards changing the particular muscle groups (limbs) used in manipulating the device, including the wrist, the arm, the hand, and the fingersesso d the hand, and the fingers (Card, Mackinlay, and Robertson, 1991). A relatively small handle may afford the user to use the fingers. On the other hand, when a tracker is mounted on the palm or the back of the user's hand, the 6 DOF manipulation will be done by the wrist, the arm, and the shoulder. Are some of these body joints more suitable than others for 6 DOF manipulation? This will be addressed in Chapter 4.
Obviously, the interaction process goes beyond the physical device itself. Block 2 in Figure 1.2 represents the transformation from user's output to the computer display interface. There are many alternatives in designing this transformation to map the output of the physical device to object movement. An important decision to make is selecting the control order of this transformation. A zero order transfer function does not involve any integration (or differentiation) and the user's input variable (such as displacement or pressure) is directly mapped to a cursor's position. Zero order transformation is hence called position control. A first order transfer function has one integration which maps the user's input variable to the cursor movement by an integral. This means that the user's input is proportional to the velocity of the cursor movement. First order transformation is hence also called rate control. Control order can be also second order (acceleration control) or higher. The research questions here therefore are: How does the transfer function affect the user's performance? Does one type of transfer function make the input control more easily to learn than others? Is there any kind of relationship between transfer function and other dimensions, such as device resistance? Some of these questions will be addressed in Chapter 2, in conjunction with the study on isotonic and isometric resistance.
Following the transformation operation, the input will then be displayed by means of some visual representation, as illustrated by Block 3 in Figure 1.2. This visual representation (a cursor) can take many forms. For conventional 2D interfaces, it might be an arrow, a cross-hair, a dot, or any other symbol that reveals the user's input actions relative to other "target" objects, such as a button, a window or a file icon. For 6 DOF input, the unique challenge is to display the spatial relationships between the user's translational and rotational input actions and other objects in the depth dimension. Chapter 5 discusses ways to take advantage of the human perceptual system and present the depth information effectively. In particular, it studies a novel 3D cursor technique based on the partial occlusion effect.
The various components of the interaction system on the computer side have to be investigated on the basis of the understanding of perceptual, motor and cognitive functions of the human user, as illustrated by the left side of Figure 1.2. This thesis attempts to incorporate some research results from the behavioural sciences, such as psychomotor behaviour, neuromotor control, psychophysics and human perception. Literature in these fields that concerns the issues in input control will be reviewed in related parts of this thesis.
us examine input, tionsks. In summary, the alternatives associated with the human factors design of a 6 DOF interaction system form a very large design space spanning multiple dimensions, including but not limited to:
As a general human factors principle, the final composition in the multi-dimensional design space of 6 DOF interface should aim at matching human capabilities and limitations so that the user can learn the 6 DOF manipulation task quickly, perform the task efficiently and work for long periods of time if necessary with minimal fatigue. However, the effects of variations along these design dimensions on learnability, performance and fatigue are often beyond designers intuition. The goal of this thesis is to provide designers with a human factors basis and rationale for the process of designing and selecting 6 DOF interfaces.
In addition to the above mentioned dimensions, there is also a more general distinction in terms of manipulation metaphor in high DOF input design. Two opposing metaphors present themselves with regard to manipulation in 3D space. One is to use the direct manipulation metaphor in which the user's hand motions are projected into the display space as isomorphically as possible (isomorphism). A glove input with 1 to 1 control-display mapping between the userís hand and displayed hand in a virtual reality (VR) environment is an example of direct manipulation input. The opposite view to isomorphism is that the input devices should be designed as tools that transform human actions indirectly into the manipulation task. One such example is a rate control device, which converts the userís motor action into an objectís movement velocity, rather than displacement. Obviously isomorphism may produce a more intuitive, more natural interface that directly takes advantage of the repertoire of skills acquired in daily life. Some researchers, however, suggest a ìhands off my VRî approach and believe tools are more appropriate for manipulation in 3D space (Green, Bryson, Poston and Wexelblat, 1994). Tools may enable users to go beyond their physical limitations, to produce input control with better quality and with less fatigue.
Of course, rather than concentrate on the extrema, one should realise that there is a continuum between the completely isomorphic control interfaces and the totally artificial tool-like interfaces. What is important is to determine how each approach quantitatively affects human performance as learning progresses, so as to provide a basis for designers to choose compromising solutions for specific applications. Isomorphism versus tool-using is not independent of the design dimensions discussed above, particularly the transformation in the input process. For instance, perfect isomorphism requires position control with 1 to 1 control display ratio, whilst rate control is perhaps more suited for interfaces designed as tools. This notion of isomorphism versus tool using is therefore addressed throughout this thesis.
The design of controls and displays for human input to machines
is a classic human factors research topic. It is by definition
at the very heart of the study of human machine systems. Although
6 DOF manipulation is relatively new, a large body of literature
related to 1 or 2 DOF input control is scattered across a variety
of journals, conference proceedings, and technical reports in
the areas of experimental psychology, human motor control, aviation
and aerospace, teleoperation, human-computer interaction and so
on. The present study builds upon this large body of literature,
attempts to consolidate it to some extent and, in some respects,
to advance it. Detailed references and reviews of the most relevant
literature are made in each chapter of this thesis. In order to
put the current study into a global context, a very general overview
of representative works on input control is presented in the following.