(a)
2.8.1 Data Transformation
In the following data analysis, statistical model residuals were
first analysed and it was found that the residual distribution
was skewed towards lower scores (Figure 2.8a). This is typical
when completion time is used as performance measure. Log transformation
was then imposed, in order to meet the residual distribution requirement
for analysis of variance (ANOVA) (Howell, 1992) . Figure 2.8 (a)
and (b) show the residual distribution before and after the log
transformation. The same procedure and treatment were carried
out for the subsequent experiments in the following chapters;
however, for the sake of brevity, the residual plots are not shown
further. For ease of comprehension, all figures that illustrate
results will still be drawn according to the original, untransformed
scale.
2.8.2 General Results
Figure 2.9 displays the means and standard errors of each control technique over the four phases of the experiment. The detailed results of a repeated measure analysis of the entire data set are summarised in Table A3.1.1 in Appendix 3. The performance difference caused by different techniques was statistically significant: F(3, 21) = 40.18, P < 0.0001. In addition, subjects' performances improved significantly with practice: F(3, 21) = 54.6, P< 0.0001. No other independent variable or interactions between the variables were statistically significant (p>0.05).
The ranks of the four techniques, as measured by average completion time over all four phases, was as follows: isotonic position (6.71 sec), isometric rate (6.97 sec), isotonic rate (10.55 sec), and isometric position (16.93 sec). Statistical comparisons between the four techniques, as summarised in Table A3.1.2 in Appendix 3, shows that the performance differences between every pair of techniques were statistically significant, except the difference between the isotonic position and the isometric rate mode.
Figure 2.9
General results of Experiment 1
2.8.3 Performance in Early Experiment Phase
Whereas the preceding section analysed the overall performances of the four techniques over all four phases of the experiment, the present section focuses on the initial test. Repeated measure variance analysis (Table A3.1.3 in Appendix 3) showed that the major conclusion drawn from Test 1 was the same as that drawn from the overall data: Technique resulted in a significant performance difference. Pairwise comparisons between conditions (Table A3.1.4 in Appendix 3) showed that all pairs were significantly different from each other, except the isotonic position versus the isometric rate (See also Figure 2.9). The mean values of these two techniques were 7.68 and 9.07 seconds respectively, i.e. the isotonic position mode produced shorter mean completion times than the isometric rate control in phase I (after 10 minutes of practice). However, this difference was not statistically significant (F(1,7) = 0.654, p = 0.43), possibly due to the relatively small number of degrees of freedom in the repeated measure analysis. Repeated measure analysis is a conservative test, in which subject and effect interactions are used as error terms.
When a full factorial analysis was used, the difference between
the isotonic position and the isometric rate mode was detected
as statistically significant (Fisher's Protected LSD post-hoc
test: p < 0.01). Full factorial analysis, using Subjects as
one of the factors and using model residual as error term, is
much more sensitive. The implications of full factorial analysis
and repeated measure analysis are different, however. Factorial
analysis computes the probability that the differences between
means are caused by chance, as reflected by model "residual"
after removing all the variance caused by all other independent
variables and their interactions, including subjects (individual
differences). Repeated measure analysis, on the other hand, does
not remove all the variance caused by individual differences and
other factors from its error term. The results of repeated measure
analysis are therefore more generalisable to larger populations
of users.
2.8.4 Performance in Final Phase of Experiment
Figure 2.10 shows the results for Test 4, which took place after the subjects had had 40 minutes of intensive practice. Repeated measure analysis of variance (Table A3.1.5 in Appendix 3) showed that technique was still a statistically significant factor. Pairwise comparison showed that the difference between every pair of techniques was significant, except the isotonic position vs. the isometric rate (Table A3.1.6 in Appendix 3).
2.8.5 Ease of Use
After the entire experiment was over, subjects were asked to rate each of the techniques according to "Ease of Use", "Fatigue" and "Overall Preference"*. Figure 2.11 summarises the subjective ratings on ease of use. The isotonic position mode and the isometric rate mode received higher average ratings than the other two modes. Technique caused a statistically significant difference on these ratings: F(3, 21) = 12.9, P <0.0001. Fisher's Protected LSD post-hoc test (Table A3.1.7 in Appendix 3) showed that the difference between every pair of techniques was significant, except the isotonic position and the isometric rate mode. Note that the results were based on data collected after 40 minutes of practice with each mode. It was observed that users usually found isometric rate control more difficult to use than isotonic position control at their very initial stage of experiment.
2.8.6 Fatigue
Figure 2.12 summarises the subjective ratings of fatigue. Technique caused a statistically significant difference on the ratings on fatigue: F(3, 21) = 10.9, P = 0.0002. Fisher's Protected LSD post-hoc test (Table A3.1.8, Appendix 3) showed that the differences between every pair of techniques were significant, except the isotonic position and the isotonic rate mode.
The isometric position technique was felt to be the most fatiguing
and the isometric rate technique the least fatiguing. It is important
to note that the isotonic position technique was significantly
more fatiguing than the isometric rate technique, even though
it produced scores similar to the isometric rate control on some
other measures (e.g. ease of use and time performance). This is
due to the fact that with isotonic sensing users have to perform
unsupported hand movements.
2.8.7 Subjective Preference
Overall, users preferred the isotonic position control and the isometric rate to the other two techniques. A significant difference in preference ratings existed between techniques: F(3, 21) = 19.3, P < 0.0001. Fisher's Protected LSD post-hoc test (Table A3.1.9, Appendix 3) showed that the differences between every pair of techniques were significant, except for the isotonic position versus isometric rate mode comparison. Users' overall subjective preferences were very similar to the result from time performance measurement .
2.8.8 Interaction between Resistance and Transfer Function
Figure 2.14 illustrates one of the most important results from this experiment. A strong interaction was found between the transfer function and the resistance of the control techniques by a repeated measure variance analysis ( Table A3.1.10, Appendix 3). Even though both resistance (F(1,7) = 8.6, p < 0.05) and transfer function (F(1,7) = 12.8 p < 0.01) significantly affected completion time, the interaction between these two variables was much more significant (F(1,7) = 182.4, P < 0.0001), suggesting that simply to compare resistance (isometric versus isotonic) or transfer function (position versus rate control), as was found in some of the literature reviewed, is misleading. This important conclusion is illustrated further in Figure 2.15, which is a revised version of Figure 2.14 plotted in 3D.
