Why be confined to using an archaic mouse and keyboard configuration when all you need to interact with your desktop PC is your own two hands? That’s what researchers at Microsoft’s Applied Sciences Group will be hoping to pose to its potential consumer base in the near future, that is if their prototype 3D display – which allows users to manipulate on-screen objects with varying hand gestures - ever sees the light of day. 

Thanks in part to Samsung’s spectacular transparent OLED displays, the research team demonstrated the Minority Report-esque technology in a video published Monday; showing a user interacting with menus, icons and objects in 3D space all with flicks of the wrist or subtle hand movements. Viewing the display from different angles also sees the 3D perspective shifting to give the impression of a 3D overlay. The set-up incorporates a see-though Samsung display (held aloft with the use of a pop-up monitor arm) and Microsoft’s motion-sensing peripheral Kinect – first released for the Xbox 360, and later for PC development - to detect hand movements, gestures and monitor where the user is viewing the display from, making the 3D effect possible - what ASG call “view-dependent, depth-corrected gaze”. The user puts their hands directly behind the screen to interact with the desktop. Placing hands back down onto a hard surface, and out of view of the Kinect, results in the display diverting back to a typical 2D user interface to navigate using mouse and keyboard. 

While the demo shows some interesting if not entirely mind-blowing uses of how users can manipulate objects in 3D space – flicking through digital files and documents, zooming into a screen – the potential is extraordinary, and not only in the context of PC, but gaming too. Imagine an RTS game where your legion of troops can be moved around the battlefield with a simple tap, pinch, drag and drop; and where your view of the war beneath can switch on the fly with subtle head movements. 

Don’t expect to see a Microsoft-patented see-through 3D display in your local PC World store any time soon however, the Applied Sciences Group’s research is still very much a conceptual idea showing what is possible with modern and freely available technology. So, we can’t actually pretend to be Tom Cruise just yet, but we can certainly dream…