Figure 6.1 The human-machine interaction system
Developments in virtual environments, telerobotics, computer aided design, scientific data visualisation, and many other technologies and applications demand human factors knowledge with regard to the design of interaction systems that allow 6 degree of freedom manipulation. Few answers have been readily available for addressing basic questions such as what the controller resistance should be and what kind of transfer function to provide for efficient human-machine system performance. This thesis is one attempt to provide both an empirical and a theoretical basis for the design and selection of 6 DOF interaction techniques, based upon surveying literature in many related domains, applying theories of human motor control, and conducting experiments with elemental manipulation tasks.
As discussed in Chapter 1, there are multiple components and stages involved when a human user exchanges spatial information with a computer (Figure 1.2, redrawn as Figure 6.1 for convenience). The user's motor actions act upon a physical device (manipulandum) which feeds a certain type of control feel (via proprioception) back to the human through resistance. The information output from the manipulandum can be transformed by various types of transfer functions. The result of the transformed input is then visually displayed. New motor control actions are generated through the interaction between exteroception, proprioception and central resources, including attention, pre-stored motor programs formed from past experience, and so on.
When designing any interactive system, the system characteristics should conform with human capabilities and limitations. On the basis of a review of a large body of literature and a series of experiments involving 6 DOF manipulation tasks, this thesis has advanced the knowledge base with respect to human performance as a function of the designing of each of the components in that interaction system as shown on the right hand side of Figure 6.1.
Experiment 1 revealed the compatibility principle between the
transfer function of the controller dynamics (Block 2 in Figure
6.1) and the mechanical properties of the physical device (Block
1 in Figure 6.1). Two 6 DOF devices with different resistance,
one isometric and one isotonic, and two types of transfer functions,
position control and rate control, were examined in a 6 DOF docking
task. A strong interaction was found between the resistance mode
and the transfer function mode. In the position control mode,
the subjects had shorter mean completion times with the isotonic
device than with the isometric device. In the rate control mode,
the completion time scores for the isotonic and isometric devices
were reversed. Analysis showed that the strong self-centring effect
of isometric devices is the key in understanding this interaction
pattern.
Between the two compatible modes, namely the isotonic position control and the isometric rate control, the former appears to be more natural and easier to learn but the latter is less fatiguing and generates smoother trajectories when the subject is well practised, due to the low pass filtering effect in rate control. In comparison to position control, rate control has both advantages and disadvantages. On the one hand, it is less direct than position control, therefore imposing a possibly higher cognitive load on the user (circle A in Figure 6.1). On the other hand it compensates for the physical limitations of the hand by enabling an essentially infinite operating range.
Experiment 2 focused on isometric versus elastic rate control. The major disadvantage with the isometric device is the insufficiency of kinaesthetic feedback (Circle C in Figure 6.1) to the user. The user can not feel the effect of her control actions very well on the basis of just the force cue alone involved in the isometric control. The major advantage of the isometric device, on the other hand, is its strong self-centring effect which Experiment 1 showed to be necessary for rate control. An elastic device, which provides movement proprioceptive cues in addition to force cues, is also self-centred. As the stiffness of the self-centring elasticity increases, the self-centring effect increases accordingly, hence enhancing compatibility with rate control. On the other hand, stiff elastic controllers allow less movement, hence reducing movement proprioceptive feedback. These two factors, compatibility (between Block 1 and Block 2 in Figure 6.1) and proprioception (circle C in Figure 6.1), therefore act in opposition in determining the magnitude of the elastic resistance. The optimal elasticity is thus a result of the trade-off between these two factors. Experiment 2 investigated the performance differences between an elastic rate control device and an isometric rate control device and found that the difference was related primarily to the experience subjects had acquired with each device. Slight advantages of the elastic device were found in early, but not later, learning stages of the experiment.
Experiment 3 pursued the same issue as in Experiment 2, but with a more demanding task, 6 DOF dynamic tracking. A more substantial difference was found between the elastic rate control and the isometric rate control, but the general trend was the same as in Experiment 2: the elastic device was easier to learn than the isometric device. Consistent with many existing human motor control theories, the results of Experiment 2 and Experiment 3 imply that the basis of human motor skills shifts from closed-loop feedback driven behaviour to open-loop motor-program driven behaviours. When richer proprioceptive feedback is available, users (of elastic devices) may more easily acquire the skills related to performing the task.
A detailed analysis of subjects' tracking performance was done through dimensional decomposition (Appendix 2), which revealed a number of interesting phenomena about 6 DOF tracking. First, a satisfactory level of performance in the depth dimension (relative to results reported in the literature) was found when interposition, perspective, stereoscopic disparity and partial occlusion cues were all incorporated into a 3D display system. Subjects' tracking errors in the depth dimension were only about 35% to 45% larger than those in the horizontal and vertical dimensions, much less than previously reported in the literature. Second, it was also found that the subjects had larger tracking errors in the vertical dimension than in the horizontal dimension in the early stages of the experiment, most likely due to their attention allocation strategy. Third, the issue of controllability of 6 degrees of freedom with one hand was addressed. With a certain priority order, subjects tended to concentrate on fewer degrees of freedom in early stages of the experiment and progressed to co-ordinate with more degrees of freedom in later stages of the experiment. Between the translational and rotational aspects, translation took higher priority. Between the horizontal, vertical, and depth dimensions, the horizontal dimension took priority. It was found that after 40 minutes of practice more than 80% percent of the subjects were able to control all 6 degrees of freedom simultaneously.
The issue of which muscle groups (Circle B and Block 1 in Figure 6.1) ought to be involved in 6 DOF manipulation was studied in Experiment 4. Two isotonic position control techniques were tested in a 6 DOF docking task. One technique, the glove, was operated with the user's wrist, elbow and shoulder. The other technique, the Fball, was operated additionally by the user's fingers. The results showed that the Fball outperformed the glove, even if the effects of a confounding factor, the use of a clutch with the glove, was removed from the analysis.
An important part of input control is the visual representation (Block 3 and Circle D in Figure 6.1) of the task. The user needs timely and revealing exteroceptive feedback about her control actions in relation to target objects. When interacting with 3D environments, a key issue is displaying a user's actions in the depth dimension. Experiment 5 investigated the use of semi-transparent surfaces in exhibiting the relationship between the user's input and the target location. The interposition cue, i.e., the fact that closer objects obscure farther objects, is known to be one of the most powerful sources of depth information in human perception. However, it is difficult to use the interposition cue in interactive computer graphics because foreground images completely block the view of the background. Experiment 5 showed that the partial occlusion cue, introduced by semi-transparent surfaces, does not have the disadvantage of the total interposition effect but is still very effective in revealing the states of the user's input in relation to target objects.
Collectively the series of experimental and analytical studies in this thesis make a significant contribution to the understanding of human factors in 6 DOF manipulation. Prior to this research, it was realised that there are many dimensions in the 6 DOF input design space that influence user behaviour, but little was known about the performance implications of many of these dimensions. This thesis provides a basis for understanding some of those performance implications. The highlights of the results can be summarised very briefly as follows (see Figure 6.1).
(1) The physical property of a 6 DOF input device should provide rich proprioceptive feedback so that the user can easily feel her control actions so as to learn the task quickly.
(2) The controller transfer function used in any interaction technique should be compatible with the characteristics of the physical device.
(3) Fine, small muscle groups and joints (i.e. fingers) should be included in the operation of input devices when possible.
(4) Visual display of user actions in relation to target objects should be designed to allow immediate exteroceptive feedback, and the inclusion of semi-transparency well serves this purpose by revealing the depth relationship between a cursor and target objects.
The studies in the thesis also have significance beyond the explicit
empirical findings with respect to users' performance in 6 DOF
manipulation techniques. First, many of the user interface designs,
such as the MITS glove, the EGG, the Fball and the "Silk
Cursor", all have novel features with practical values. Second,
the experimental paradigms, including 6 DOF docking, 6 DOF tracking,
"virtual fishing" and their performance measurements,
as well as the large software system developed in the course of
this research, make methodological contributions to research in
6 DOF input and 3D human machine interaction. Third, some of the
results, such as the role of proprioceptive feedback, also contribute
to the understanding of human motor skills.