Virtual touch screen

A virtual touch screen (VTS) is a user interface system that augments virtual objects into reality either through a projector or optical display using sensors to track a person's interaction with the object. For instance, using a display and a rear projector system a person could create images that look three-dimensional and appear to float in midair.[1] Some systems utilize an optical head-mounted display to augment the virtual objects onto the transparent display utilizing sensors to determine visual and physical interactions with the virtual objects projected.[2]

History

Microsoft Hololens augmenting the Windows 10 interface for the user through the optical head-mounted display.

Virtual touch screens were originally inspired by holography.

Development

During a 2009 study at Korea Electronics Technology Institute, a virtual touch screen system was built using the touch point tracking technique based on the time of flight (TOF) camera and the Kalman filter.[7] To address the widespread problem in conventional virtual touch screen systems (i.e. responding sensitively to even the slightest hand movement and failing to recognize the touch point accurately), the study had applied the Kalman filter tracking technique to predict and track the touch point consecutively. Being forecast and recognized, the predicted touch point was bound to exist all the time, but an abrupt movement of touch point was likely to be recognized only insensitively. A commercial version based on this technology was never implemented.

Usage

Currently, virtual touch screens (VTS) are used primarily for entertainment and research purposes. A popular virtual touch screen system is the Kinect system made commercially available by Microsoft. The Kinect system is primarily used for consumer entertainment such as playing games and using the menus found on the Xbox system.[8][9] However, in recent years the Kinect has been used for research projects in the field of robotics and 3-D interactions outside of the field of games such as browsing the web, and medical practices.[10][11][12]

There is a potential for the technology where the 3D visualization and physical interaction with the objects is necessary, such as teaching students introductory calculus.[13]

Virtual touch screen technology can also be applied to turn non-touch displays into interactive, touch-capable surfaces using a Kinect sensor in combination with a projector.[14] This type of application allows virtual touch technology to enhance personal display usage as well as better communication and presentation of data to be used in education or business settings.

The Kinect sensor, commonly used in many virtual touch screen applications

In medical practices, virtual touch interaction has been applied to allow users to view and manipulate digital data such as 3D images used in medical scans, allowing medical staff to interact with the data without any physical contact, thus avoiding the need for re-sterilization.[15] Additionally, Microsoft's Kinect sensor has also employed the use of virtual touch screen technology to help stroke patients recover and improve motor function in their limbs through a game-like system involving interpretation of gestures combined with adjustable difficulty levels based on the user's performance.[16]

Human factors and ergonomics

One of the key issues with virtual environments that utilize touch are the haptics. For example, utilizing virtual touch screen for medical training could cut costs down for training medical staff for touch delicate operations such as surgery. However, the haptics of such training does not provide sufficient physical touch feedback necessary for a virtually trained practitioner to perform a real surgery. Incorporating sufficient haptic feedback to virtual touch screen interactions is an active area of research.[17]

See also

Notes

  1. ↑ Gary B. Shelly, Thomas J. Cashman, Misty E. Vermaat, Jeffrey J. Quasney,; et al. (2008). Discovering computers 2008 : introductory. Boston, MA: Thomson/Course Technology. ISBN 1423912047.
  2. ↑ Koh, Eunjin; Won, Jongho; Bae, Changseok. "Vision-based Virtual Touch Screen Interface". 2008 Digest of Technical Papers-International Conference on Consumer Electronics (2008): 1–2.
  3. ↑ "Displair company from Astrakhan demonstrated interactive screenless display". kiosks.ru. 2012-04-17. Retrieved 2012-06-08.
  4. ↑ ""Project Natal" 101". Microsoft. June 1, 2009. Archived from the original on June 1, 2009. Retrieved June 2, 2009.
  5. ↑ Kain, Erik (21 January 2015). "Microsoft Augments Reality With Windows Holographic". Forbes. Retrieved 21 January 2015.
  6. ↑ Myerson, Terry (21 January 2015). "The next generation of Windows: Windows 10". Blogging Windows. Microsoft. Retrieved 21 January 2015.
  7. ↑ AHN, YANG-KEUN; KIM, MIN-WOOK; PARK, YOUNG-CHOONG; CHOI, KWANG-SOON; PARK, WOO-CHOOL; SEO, HAE-MOON; JUNG, KWANG-MO (2009). "Implementation of 3D Virtual Touch Screen" (PDF). Korea Electronics Technology Institute.
  8. ↑ Microsoft's new Kinect is official: larger field of view, HD camera, wake with voice
  9. ↑ Dave Tach (May 21, 2013). "Kinect is always listening on Xbox One, but privacy is a 'top priority' for Microsoft". Polygon. Retrieved May 22, 2013.
  10. ↑ Wortham, Jenna (November 21, 2010). "With Kinect Controller, Hackers Take Liberties". The New York Times. Retrieved November 25, 2010.
  11. ↑ "Medical practice finds use for Kinect hack". QuickJump Gaming Network. December 27, 2010. Retrieved May 23, 2011.
  12. ↑ "radRounds". Xbox Kinect-based radiology & medical image exploration. Retrieved December 4, 2011.
  13. ↑ Boggess, Jason; Harding, Chris. "Improving Introductory Calculus Education with 3-D Visualization and Virtual Touch (Haptics)". Virtual Reality Applications Center, Iowa State University, Ames, Iowa (2007).
  14. ↑ "Ubi Interactive". Make the World Your Touchscreen. Retrieved 2015-04-21.
  15. ↑ "GestSure". Take control of your operating room. Retrieved 2015-04-21.
  16. ↑ Chang, Eric (2015). "Microsoft Research". Stroke Recovery With Kinect. Retrieved 2015-04-21.
  17. ↑ Robles-De-La-Torre, Gabriel (2006). "The importance of the sense of touch in virtual and real environments". IEEE. 13 (3): 24–30. doi:10.1109/MMUL.2006.69. ISSN 1070-986X. Retrieved 2015-04-21.

References

This article is issued from Wikipedia - version of the 8/6/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.