How Augmented reality tools help Doctors
In the past few decades, the medical industry has made tremendous progress, including new advances in imaging diagnostics , such as contrast-enhanced ultrasound, mammography, computed tomography (CT), and magnetic resonance imaging (MRI). and many more. Not only do these technologies benefit from advances in other areas, but they also advance advances in technologies such as miniature cameras, microprocessors, and real-time data streaming, all of which revolutionize the way doctors use images during surgery.
Now, basically one or several of the above medical images are used before each surgery . Even in the emergency room, doctors use ultrasound or CT to help guide the treatment process. Whether it is a large surgery or a small surgery, these images can now be presented in real time during the surgery.
Although medical imaging technology has made great progress, until now, the display of video content is basically no different from the 1950s. Both the image and the data are presented on a 2D flat screen, forcing the medical staff to take their eyes off the patient and even remove them from the hands of the doctor who are undergoing surgery. More importantly, these images are not presented from the perspective of the viewer, but rather depend on the device used for display. During the surgery, doctors must use their skills and imagination to understand the data and project the images onto the patient through their own imagination.
In addition, different visual data will be displayed separately, so doctors have to assign extra attention to integrate different images and data content, such as integrating angiography and CT images into the affected area of a patient. To master this skill, it takes years of training.
Augmented Reality (AR) is a technology that can superimpose digital information into the real world. In our opinion, this technology has the potential to change the field of medical imaging. According to our research at the Augmentarium project at the Maryland Blended Reality Center, in our intended application scenario, a doctor can use AR heads, such as Microsoft’s HoloLens, to see digital images and other data. Directly superimposed on his field of view, in such a scene, the head display may show the patient’s floating echocardiogram, as well as extremely important data about the patient’s aneurysm, all of which are directly present in the surgical field. The doctor does not need to look at the different display devices in the area other than the patient.
The AR is able to present both image data and other important patient information, a potential that saves lives and reduces medical incidents. This is especially important for those treatments that are performed outside the operating room. The operating room is perhaps the safest place in the hospital, where there are 4 to 8 full-time health care workers around the patient. Since everyone has seen medical imaging content before surgery, the surgical procedure is usually properly planned. The anesthesiologist monitors the patient’s physiology and is responsible for pain management and life-saving medications. Surgical nurses need to ensure that all necessary equipment is available immediately, and doctors need to be fully immersed in the surgical task.
However, the time in the operating room is extremely expensive and the operating room is also scheduled in strict accordance with the selected case. These selected surgeries are one of the basic sources of revenue for all hospitals, so it’s important to ensure that the operating room is fully occupied and functioning properly. Those small, or sudden, surgeries are difficult to arrange in the operating room. As a result, many of these procedures are performed outside the operating room, such as in an intensive care unit or emergency area. It is these patients who do not have surgery in the operating room who take the most risks. At this point, we can see the greatest benefits that AR brings to the medical field.
Unlike the surgery performed in the operating room, the surgery performed outside will receive less support from the medical staff, usually only one doctor and one nurse, who also have other responsibilities. Usually, the patient’s vital signs are unstable. If it is late at night, the doctor who performed the surgery may have just received training, and he may also assume the role of an anesthesiologist and doctor. This room is also not designed for surgery, especially without the complicated equipment for presenting images, not to mention the confusing devices that monitor patient vital signs.
It is easy to miss the most important clues in tracking a patient’s multiple image and data information. At this time, only one AR display function is needed to integrate all the images and patient data, and the doctor can focus on the patient, which obviously improves the quality and safety of the surgery, and reduces the surgical procedure. Related complications, which in turn save costs.
In addition to the cost reductions due to safer surgical procedures, it is also possible to reduce costs by reducing the need for redundant screens. At present, ultrasound, endoscopy and bronchoscopy require hospitals to purchase a complete system, each with its own display. Each of these systems can cost hundreds of thousands or even millions of dollars. It’s not because different imaging functions require special displays, but the current economic model. These entire systems are sold with incompatible displays.
Electronic medical records have their own displays, hemodynamic monitors and ventilators also have their own screens, and their data cannot be fused. AR can provide a shared screen that gradually reduces the need for specific displays needed to monitor each patient’s data, while providing a real-time integrated display of vital sign data from multiple sources.
To achieve these functions, current AR technologies still need to evolve, and doctors need to understand this concept. Doctors need to be comfortable and safe when wearing these hardware. In some scenarios, these images need to be as opaque as possible, while others may need to be more transparent. If the projected image is to be used for surgical procedure guidance, it needs to be presented with great accuracy. Although there are various technical challenges, we believe that no single challenge cannot be overcome.
At the Mixed Reality Center in Maryland, we’re committed to researching AR as a primary tool to help healthcare professionals save lives while reducing healthcare costs. Teams from Stanford University, Duke University, and Johns Hopkins University are working to integrate and project visual data while developing AR display devices for the medical industry.
This will require talented computer scientists and visionary doctors to work together to make augmented reality a reality in the medical field. We are passionate about the future, and by then, our use of AR technology in the medical field will be as common as the use of stethoscopes today.
