3D Interaction Controls Revisited

Is it time for new interaction controls in 3D modeling applications such as Computer-Aided Engineering (CAE) analysis tools? Interacting with 3D models on a 2D screen has always been less than intuitive. However, a recent flurry of innovation in human-computer controls holds out much promise for the future of 3D model control.

E3 2009: Project Natal Xbox 360 Announcement

The Nintendo Wii game console and its associated position-aware controller (Wii Remote) has popularized an intuitive, motion-based interaction control. Interestingly, Apple recently filed a patent that describes a Wii-like 3D control system, but without the controller. Apple's design uses a standard camera to sense the motion of the user and then interprets that motion to interact with a 3D model.

Similar to Apple's patent is a system being developed for the next generation of games by Microsoft called Project Natal, that uses a special camera-like sensor.

The popularity of Apple's iPhone, with its intuitive finger-based multi-touch interface, has sparked renewed interest in touch-based laptops and monitors. For instance, SpaceClaim are experimenting with multi-touch controls in their CAD application.

SpaceClaim MultiTouch

Gesture-based controls are another area that holds the promise of easier and more efficient human-computer interactions, as demonstrated by SpaceClaim (notice a pattern here?), Firefox, and Opera.

There have been many predictions of the imminent doom of the mouse-driven controller, yet it is still the primary controller for 3D applications - "rumors of my demise have been greatly exaggerated." Given the flurry of innovation currently focused on human-computer interactions, maybe control evolution is now imminent.


3D Interaction Controls

Rich - the answer to your question is, of course, yes and no.

Yes because there is indeed room for innovation in this area. The 2D mouse doesn't truly provide a natural control for moving in 3D. This is one reason why we've finally added support for 3D mice to Pointwise. Use the 3D mouse in your left hand to control pan, zoom, rotate (and more) of the 3D scene while the right hand uses the 2D mouse to interact with the GUI. This however, is nothing new. People have been asking for this since back in the day when the devices were called "spaceballs" (not the movie).

No, because I don't see any of the examples cited as being practical for someone who has to use them 8 hours a day. Recall the scene in Minority Report where Tom cruise is standing and manipulating a display with large sweeping movements of his arms. Now do that for 8 hours straight - quite a workout. The complaint is the same for voice-activated GUI's; can you imagine working in an open plan office where those were in common use?

Good Points

John, thanks for your well considered comment. I just came across another interesting UI concept design (see video below) that combines multi-hand-multi-touch with a large touchpad. The concept encourages ergonomic hand positions (more in line with the 3D-mouse-2D-mouse combination you mentioned) that wouldn't require Popeye forearms to make it through an 8 hour day.

10/GUI from C. Miller on Vimeo.

Touch Screen CAD from ZWCAD


Here's another example of touch screen technology in a CAD system, specifically ZWCAD.