Well, with the Virtual Pointer, you do. When you look at the screen now, you see a pointer hovering about around the suitcase, which could be far away from you. You can move it around, so you line it up with one end of the suitcase, and press a button. Now as you move it, you see dragging along behind it a Virtual Tape-Measure. Press the button again, and the computer tells you that the suitcase is 42 cm wide, which will just fit into your bomb-disposal bin. You've saved the day!
Of course, this clever pointer can be used in all sorts of ways. For example, if your robot is the least bit intelligent, you can use it to point out a destination, and let the robot drive itself there while you attend to more important matters. There are some pretty good techniques for avoiding obstacles and working out how to get to a destination, so this is something many robots today could do.
But if your robot isn't all that clever, you could use the Virtual Pointer to draw a path for the robot to follow.
So that's the idea behind the Virtual Pointer. The way we accomplish this is discussed in several of our papers. We've implemented it on a number of different systems, including Amigas, PCs, and various SGI platforms.
Of course, we aren't limited to just drawing pointers. We can draw stereoscopic graphics that are calibrated to the real world so that things look very realistic. The images that we draw can be used to enhance or augment the image of the real world. Thus we arrive at the topic of Augmented Reality. We call our technology ARGOS, meaning Augmented Reality through Graphic Overlays on Stereo-video. By combining stereoscopic video with stereoscopic graphics, we can add information to the scene that would otherwise be invisible.
Drascic, D., Grodski J.J., Milgram P., Ruffo K., Wong P., Zhai S., "ARGOS: A Display System for Augmenting Reality", ACM SIGGRAPH Technical Video Review, Volume 88: InterCHI `93 Conf on Human Factors in Computing Systems, Extended Abstract appears in Proceedings of INTERCHI'93: ACM Conference on Human Factors in Computing Systems, p 521, Amsterdam, April 1993.
Drascic, D. & Milgram, P., "Positioning Accuracy of a Virtual Stereographic Pointer in a Real Stereoscopic Video World", SPIE Vol 1457 - Stereoscopic Displays and Applications II, San Jose, California, Feb. 1991.
Grodski, J.J., Milgram, P., & Drascic, D., "Real and Virtual World Stereoscopic Displays for Teleoperation", NATO Defence Research Group Seminar: Robotics in the Battlefield, March 1991.
Milgram, P., Drascic, D., Grodski, J.J., "Stereoscopic Video-Graphic Coordinate Specification System", U.S. Patent #5,175,616, Dec. 29, 1992.
Milgram, P., Drascic, D., Grodski, J.J., "Enhancement of 3-D Video Displays By Means of Superimposed Stereographics", Proc. Human Factors Soc. 35th Annual Meeting, San Francisco, 1457-1461, Sept. 1991.
Milgram, P., Drascic, D., & Grodski, J.J., "A Virtual Stereographic Pointer for a Real Three Dimensional Video World", in D. Diaper, D. Gilmore, G. Cockton & B. Shackel (eds): Human-Computer Interaction - Interact' 90, Cambridge, UK, Aug. 1990.
Milgram, P., Drascic, D., Grodski J.J., "Stereoscopic Video + Superimposed Stereographics: Applications in Teleoperation", Proc. 2nd Canadian Workshop on Military Robotic Applications, Kingston, Aug 1989.
Milgram, P., Zhai, S., Drascic, D., Grodski, J.J., "Applications of Augmented Reality for Human-Robot Communication", Proc. IROS'93: Int'l Conf. on Intelligent Robots and Systems, Yokohama Japan, 1467-1472, July 1993.
Rastogi, A., Milgram, P., Drascic, D., Grodski, J.J., "Virtual Telerobotic Control", 1993 DND Workshop Advanced Technologies in Knowledge Based Systems and Robotics, Ottawa, Nov. 1993.