One of the crucial future technologies in surgery is Augmented Reality (AR). Most experts agree that AR will increase safety and efficiency, improve surgical training and decrease costs. But what exactly is AR? AR means enhancing reality with an additional layer of virtual information. The additional layer should be at the correct location, i.e. geometrically registered; it should be interactive and be available in real-time. 

In addition to visual information, AR can offer audio, haptic, olfactory and even gustatory information. There are three different AR modalities: 

• video see-through, 
• optical-see-through and 
• projector-based AR.

Video see-through 

In video see-through a video feed of the real world is displayed on a screen with digital information added. This modality is widely used in healthcare, particularly in research projects and clinical studies but also routinely in laparoscopy and endoscopy. Currently, the disadvantage of video see-through is the limited resolution of the camera, but the technological progress could sooner or later overcome this hurdle. Another issue is the screen between the user and the patient.

Optical see-through

In optical see-through systems the images of the real world are not displayed on a screen but are projected onto a slanted semi-transparent mirror. The disadvantage of body-worn AR solutions is the fact that the glasses are still rather large. Wearing them over an extended period of time is uncomfortable. Moreover, the optical see-through medium absorbs light which makes reality appear darker than it actually is. . 

Projector-based AR

Projector-based AR does not rely on a body-worn or hand-held device. Rather the projector is integrated in the OR environment and the information is projected onto the objects of interest. Thus, reality is augmented with digital information. This modality is still in an early research phase and no devices are commercially available.

Cooperation and Training

In addition to surgeons, IT experts and, above all, psychologists play important roles. This is about a man-machine interface, thus a number of psychological issues need to be clarified. When new technologies are launched too early, people might trust and rely on these systems too much. Another issue is distraction: the AR overlays distract from what’s happening in reality. Thus a surgeon might be so focused on the digital information that he overlooks something more relevant that occurs unexpectedly. 

While AR images might not be overly important for an experienced surgeon, they can be crucial for junior physicians simulators can guide trainees through highly immersive multi-user environments. They can be exposed to different scenarios and train behaviour in complex situations or the communication with the OR team. The physicians wear a headset which allows them to enter a multi-user environment. This VR scenario is linked to a laparoscopy or anaesthesia simulator to train certain situations. The success of the intervention depends to a large extent on the communication between the surgeon and the anaesthetist – a scenario that can be trained in VR. 

Professor Hansen, Professor for Virtual and Augmented Reality at Otto von Guericke University Magdeburg, and his team are experimenting with this kind of training in a joint project with the University Hospital Mainz., funded by the German Federal Ministry of Education and Research. They combine VR systems with relevant simulation and training software.