4.7: Augmented Reality (AR)

AR Technologies

Augmented reality (AR) aims at simplifying user life by creating direct or indirect real-time view of a real physical world. To do this computer-generated perceptual information superimposed to real world digitally (“Augmented Reality,”n.d.).

Paul Milgram and Fumio Kishino defined the Milgram’s Reality-Virtuality Continuum, which shows the span of a real-to-virtual environment (Azuma, Baillot, Behringer, Feiner, Julier, & Macintyre, 2001). In this demonstration Augmented Reality is one part of the Mixed Reality (MR) (Figure 3).

Figure 3 Milgram’s Reality-Virtuality Continuum (taken from Azuma et al., 2001).

According to Azuma (1997), augmented reality technology has three main requirements: Combination of real and virtual content, interactive in real time and registered in 3D. Augmented reality devices require to include these three key elements. (Billinghurst, Clark, & Lee, 2015). Regardless of using similar devices such as head mounted displays (HMD) (or head-worn displays HWD), tracking systems, computer interfaces augmented reality differs from Virtual Reality. Different then VR, AR technology removes real objects or manipulates/replaces objects from the real environment (Carmigniani, Furht, Anisetti, Ceravolo, Damiani, & Ivkovic, 2010). Removing objects from the real environment require covering those objects with an artificial computer-generated information. This virtual information matches to environment background to help the user to ignore the real objects (Carmigniani et al., 2010) There are some other main differences between AR and VR in terms of system requirements (figure 4). For example, AR system does not need a wide Field of View (FOV), since AR technology display can be non-immersive.

Figure 4 Virtual Reality and Augmented Reality technology requirements (taken from Billinghurst et al., 2015).

Augmentation of the real world can apply to all senses such as touch, smell, hearing (Krevelen & Poelman, 2010), haptic and to the somatosensory system (“Augmented Reality,”n.d.; Carmigniani et al., 2010) but the vast majority of research focused on visual enhancements of reality.

AR Devices

There are many different devices for AR systems, but some devices such as tracking, computers, displays, and input devices are the most used ones in research (Carmigniani et al., 2010). In this review, I restricted AR definition to specific display technologies. Head mounted displays provide imagery in front of user’s eyes. There are two common types of HMD (figure 5), optical see-through and video see-through. Optical see-through uses the semi-transparent mirror. This mirror combines real-world image and augmented images, with this way users see augmented/enhanced scene through a transparent display. Video see-through systems include cameras, and opaque mirror since visual information from real world pass through camera system, the computer process the information and augmented scene displays on this opaque mirror (Azuma et al., 2001). Since the augmented scene information already processed by the computer, video see-through systems have more control over the combined display (Carmigniani et al., 2010).

Figure 5 Two types of HWD’s. Optical see-through display (left), Video see-through display (right) (taken from Azuma et al., 2001).

Figure 6 First display is a video-see-through, the middle one is an optical see-through and both of them are binocular HMD from Trivisio. The last one is a monocular HMD from Google Glass (taken from Billinghurst et al., 2015).

In addition to HMD, handheld displays are becoming popular. Users can hold the AR employed devices in their hands, which makes the AR applications highly portable and more socially accepted then the HMD. Users can use Smart-phones, Tablet PC, and PDAs to run AR applications (Billinghurst et al., 2015). For example, Google’s ARCore software development kit allows users to build AR applications, by using several technologies such as light estimation, motion tracking, and environmental understanding (“ARCore,” 2018; “Google Developers, ARCore Overview,” 2018” ).

 

Figure 7 Playing a multi-player game with two PDAs (taken from Wagner et al., 2005)

Figure 7 shows two people playing The Invisible Train game on PDA’s. Trains do not exist on the wooden miniature railroad track, players only see them through video see-through displays. According to Wagner et al. (2005), handheld AR interface allows untrained users to accomplish collaborative spatial tasks.

Another version of AR display is Spatial Augmented Reality (SAR). Common versions are limited in mobility but users can use them without carrying or wearing the display. Usually, SARs installed at a fixed location and projects graphical information onto the physical objects or video see-through displays provides the augmented imagery. For example, mirror-like large screens enable customers to try on clothes without undressing by using Radio-frequency identification (RFID) (“Interactive retail systems,” n.d., “Keonn Technologies”) (figure 8).

Figure 8 User try on clothes 3D. Hand gestures allow the user to select and try different clothes.