The HoloSketch VR Sketching System (Deering)

The creators of the HoloSketch system wanted to create a high-quality, high-accuracy VR display system to allow non-programmers to be able to create accurate virtual objects more quickly than with more common 3D drawing systems. Their 3D sketching system uses a CRT monitor display, a 3D mouse which works like a one-fingered data glove, and a set of glasses that enables head tracking. They implemented a "fade-up" menu that replaces the current view after a small delay (using a fade effect) on a wand button press with a pie menu, including sub-menu capability. Drawing is done by holding down a different wand button and using the wand tip as a sort of 3D pen tip. Prepackaged objects can be created using the menu to select one, pushing the wand button to create a new instance, and moving the wand to affect size or shape or form of the object. Other kinds of objects include making a material trail behind the wand, like wire-frame lines or virtual "toothpaste". HoloSketch supports editing operations once an object is selected by moving the wand tip inside an object and pressing the middle wand button, and to enable complex operations without accidental wand button presses doing accidental things, keyboard buttons must be held down at the same time. They have a "10X reduction mode (meta key)" to remove issues with jitter. The system allows animations to be added to objects.

They found that it is "surprisingly quick and easy to create complex forms in [3D] using HoloSketch." They find that the most common mistake is people not expecting head motion to make a difference in the display because people have been trained to expect otherwise by standard computer use. They had a single non-programmer artist use trials of the system over several months. They found that elbow support was necessary for extended work, and that as an expert user she could draw two-handed to rapidly create complex shapes with the toothpaste primitive.

Discussion:

I wonder what the cost of reproducing a system like this would be today. It seems like it would be easier to work with an LED monitor than a CRT because the screen is flat and there is no layer of thick glass and there shouldn't have to be corrections for that.

I imagine pie menus would be even more useful in 3D space than in 2D with a pen, because if the user is just pointing without physical contact to steady her hand it would be hard to point to a specific list item easily (I have noticed this with the Wii and doing tasks like "typing" a name by pointing and clicking at a keyboard image.) It would be nice to better understand what they are talking about in this paper when they discuss how they do jitter reduction (their "10X reduction mode" -- see page 5, or 58) -- I think this is very important in any hand-tracking-based application requiring precision.