Review Smart Medicine – Reality

// 08.10.2018,ZWT Graz//

Review: Smart Medicine – Reality

 The current medication status in the individual rooms can be seen together with the patient occupancy on the multi-touch application. And the patients receive an “additional visit” via the data glasses. The new possibilities of digitisation also offer great potential for the medical sector. The associated opportunities and challenges were discussed at the invitation of ZWT, HTS and Med University Graz in cooperation with Silicon Alps at “Medicine Meets Technology – Smart Reality” on October 9th.

“Smart Reality – Smart Vision – Virtual Reality: the new possibilities of digitisation are advancing into all areas of medicine”, Josef Smolle from the Institute for Medical Informatics and Documentation at the Medical University of Graz summed up at the MED CAMPUS event. It was opened by Caroline Schober-Trummler, Vice Rector of the Medical University of Graz, as Managing Director of ZWT-GmbH, and Anke Dettelbacher welcomed the guests.


VR biopsy and vitality visualisation of structures 

Smolle on the diverse applications in medicine: “The spectrum ranges from virtual anatomy to high-quality and semi-automated clinical imaging, computer-guided VR biopsy systems and the intra-operative use of data glasses to the vital display of structures in the (formerly) submicroscopic range. Much has already been developed and some of it will become a fixed part of the routine.”


AR systems for the hospital bed 

An application from the Graz company Evolaris  is already being tested in an Upper Austrian hospital. This makes it possible for doctors to look after patients not only as real persons but also via data glasses. An example of use: doctors can support the nursing staff at the bedside by wearing the glasses. “Digital assistance systems such as this live remote system should never replace the doctor, but can give him or her an advantage”, explains Christian Kittl from Evolaris. “We have been using this augmented vision system for a long time in the industry, where support staff support the workers on site via the data glasses and, for example, project circuit diagrams into the data glasses.” Data protection is the top priority, especially in the medical sector. “For example, we are working on automated procedures to ensure that the right to one’s own image remains guaranteed.”

  • This is a classical application case of Augmented Reality (AR): the real image, e.g. the patient in the hospital bed, can be seen via the data glasses, and additional, relevant information (e.g. health parameters) can be seen via the data glasses.
  • With Virtual Reality (VR), on the other hand, the entire visual content is virtual (e.g. a 3D video seen with VR glasses).
  • Smart Reality or Smart Vision is an umbrella term for VR, AR and similar technologies.


“VR is no longer a toy.”

Markus Karlseder from Mindconsoleabout a concrete application: “In America there are already VR operation simulators for orthopaedic operations or laparoscopic operations. Virtually, I can very easily depict and train scenarios that are no longer possible with a training dummy – such as when vital parameters derail dangerously or the patient moves.” It is important that the whole process is realistic. “VR is not a toy but has immense potential, especially for the training sector. Through storytelling, VR can build an immersive bridge between conventional theory and practice.”


Linking MR image with reality

Silviu Reghin from Codeflügelexplained another application example regarding AR: “You can see the patient at the operating table via the MS HoloLens and, at the same time, the MR over the body with 3D mapping. The image data provides more precise information and enables the MR image to be linked to reality.” Reghin also emphasized these different approaches: with AR you can either work with classic devices like Smartphones or Tablets or with data glasses like the MS HoloLens, with which you “have your hands free”.


The hospital at a glance

The Fraunhofer Institute, which is also located in Graz, deals with data visualisations. Eva Eggeling: “With ‘Health at Hand’, we have developed a multi-touch application with which you can see the hospital and all relevant data at a glance: for example, how many drugs are still present in the cabinet in room X, which patients are in which rooms and also the health data of the individual patients. This is, of course, much more efficient for orientation than Excel lists or similar projects.” Data protection plays an important role here. “There are different permission; each one sees only the information that is relevant to them.”


Thought-controlled grasping neuroprosthesis 

Eduardo Veas from the Know-Center: “Among other things, we are building a prototype for collaborative monitoring of biomarkers, which will enable experts to identify the markers at an early stage. We are also working on an individually mind-controlled grasping neuroprosthesis in the ‘MoreGrasp’ project. Commands from the brain are transmitted to a stimulator, triggering a movement action.” Especially with VR and AR technologies, however, there are still numerous challenges. “Data must be credible and from a secure source. At the moment it is also the case that many displays have too narrow a field of vision. It is important not to overload people with applications – and to develop them bit by bit. As a know-how center, we support companies and researchers throughout the entire process: from data collection and analysis to the development of new data-driven services and business models.”