Learn how GE Healthcare used AWS to build a new AI model that interprets MRIs

MRI images are quite complex and data heavy.

Because of this, developers train large language models (LLMs) for MRI analysis have been cutting captured images into 2D. But this results in only an approximation of the original image, thus limiting the model's ability to analyze complex anatomical structures. This creates challenges in complex related matters brain tumorsskeletal disorders or cardiovascular diseases.

But GE Healthcare appears to have overcome this major hurdle, introducing the industry's first full-body MRI (FM) 3D research base model this year. AWS re: Invent. For the first time, models can use full 3D images of the entire body.

GE Healthcare's FM was built on AWS from the ground up – there are very few models designed specifically for medical imaging such as MRIs – and is based on more than 173,000 images from more than 19,000 studies. Developers say they were able to train the model with five times less computing than was previously necessary.

GE Healthcare has not yet commercialized the basic model; it is still in the evolutionary research stage. Early assessor, General Brigham's Massready to start testing it soon.

“Our vision is to put these models in the hands of technical teams working in healthcare systems, giving them powerful tools to develop research and clinical applications faster, and also more efficiently cost-wise,” GE HealthCare AI chief executive Parry Bhatia told VentureBeat.

Enables real-time analysis of complex 3D MRI data

Although this is a new development, AI and generative LLMs are not a new area for the company. The team has been working with advanced technologies for more than 10 years, Bhatia explained.

It is one of its main products ON Recon DLa deep learning-based reconstruction algorithm that allows radiologists to perform crisp images faster. The algorithm removes noise from raw images and improves signal-to-noise ratio, cutting scan times by up to 50%. As of 2020, 34 million patients have been scanned with AIR Recon DL.

GE Healthcare started working on its MRI FM at the beginning of 2024. Because the model is multimodal, it can support image-to-text search, link images and words, and segment and classify diseases. The goal is to give health care professionals more detail in a single scan than ever before, Bhatia said, leading to faster and more accurate diagnosis and treatment.

“The model has great potential for real-time analysis of 3D MRI data, which could improve medical procedures such as biopsies, radiation therapy and robotic surgery,” said Dan Sheeran, GM for healthcare and life sciences. at AWS, to VentureBeat.

Already, it has outperformed other publicly available research models in tasks including classifying prostate cancer and Alzheimer's disease. It has demonstrated up to 30% accuracy in matching MRI scans with text descriptions in image detection – that may not sound that impressive, but it's a huge improvement over the 3% capability which is presented by similar models.

“It's gotten to a point where it's giving really strong results,” Bhatia said. “The implications are dire.”

Doing more with (much less) data

The MRI process requires multiple types of data to support different methods that map the human body, Bhatia explained.

The so-called T1-weighted imaging technique, for example, highlights fatty tissue and reduces water signal, while T2-weighted imaging enhances water signals. Both methods are complementary and create a complete picture of the brain to help clinicians detect abnormalities such as tumors, trauma or cancer.

“MRI images come in all different shapes and sizes, just like books come in different shapes and sizes, right? ” said Bhatia.

To overcome the challenges presented by different databases, developers introduced a “resize and adapt” strategy so that the model could handle and respond to different changes. Also, data may be missing in some areas – an image may be incomplete, for example – so they taught the model to simply ignore those situations.

“Instead of getting involved, we taught the model to bridge the gaps and focus on what was available,” Bhatia said. “Think of this as solving a puzzle with some pieces missing. .”

The developers also used semi-supervised student-teacher learning, which is especially helpful when there is limited data. With this method, two different neural networks are trained on both labeled and unlabeled data, with the teacher creating labels that help the student learn and predict labels in real time. coming

“We're now using a lot of these self-directing technologies, which don't require a lot of data or labels to train large models,” Bhatia said. “It will reduce the dependencies, where you can learn more from these raw images than in the past. “

This helps ensure that the model performs well in hospitals with fewer resources, older devices and different types of data, Bhatia explained.

He also emphasized the importance of the multi-modality of the models. “A lot of technology in the past was unique,” Bhatia said. “It would only look into the image, into the text. But now that they're becoming multi-modal , they can go from image to text, text to image, so you can bring in a lot of things that were done with separate models in the past and integrate the workflow.”

He emphasized that researchers only use databases to which they have rights; GE Healthcare has partners that allow anonymous data sets, and are careful to adhere to compliance standards and policies.

Using AWS SageMaker to address computational, data challenges

Of course, there are many challenges when building such sophisticated models – such as limited computing power for 3D images that are gigabytes in size.

“It's a huge amount of 3D data,” Bhatia said. “You have to incorporate it into the model's memory, which is a very complicated problem.”

To help overcome this, GE Healthcare built on it Amazon SageMakerwhich provides high-speed networking and distributed training capabilities across multiple GPUs, and introduced Nvidia A100 and tensor core GPUs for large-scale training.

“Because of the size of the data and the size of the modules, they can't fit it into a single GPU,” Bhatia explained. SageMaker allowed them to customize and scale operations across multiple GPUs that could interact with each other.

Also used by developers Amazon FSx in Amazon S3 object storage, which allowed faster reading and writing for databases.

Bhatia pointed out that another challenge is cost optimization; with Amazon's elastic computing cloud (EC2), developers were able to move unused or infrequently used data to lower-cost storage tiers.

“Leveraging Sagemaker to train these large models—especially for efficient, distributed training across multiple high-performance GPU clusters—was one of the critical components that really helped us move faster,” Bhatia said.

He emphasized that all components were built from a data integrity and compliance perspective that took into account HIPAA and other regulations and regulatory frameworks.

Ultimately, “these technologies can streamline, help us innovate faster, as well as improve overall operational efficiency by reducing the administrative burden, and ultimately drive better patient care – because now you provide more personal care.”

Serves as the basis for other specialized models

Although the model is currently specific to the MRI field, researchers see good opportunities for expansion to other medical fields.

Sheeran pointed out that historically, AI in medical imaging has been hampered by the need to develop custom models for specific situations in specific groups, requiring expert annotation for each image used in training.

But that approach is “inherently limited” due to the different ways that diseases manifest across individuals, and include common challenges.

“What we really need are thousands of such models and the ability to quickly create new ones as we encounter new information,” he said. High quality labeled data sheets are also required for each model.

Now with next-generation AI, instead of training separate models for each disease/organ combination, developers can pre-train one basic model that can serve as the basis for other specific models that well tuned down the river.

For example, the GE Healthcare model could be extended to areas such as radiation therapy, where radiologists spend a lot of time manually identifying organs that may be at risk. It could also help reduce scan time during x-rays and other procedures that currently require patients to sit still in a machine for long periods of time, Bhatia said.

Sheeran marveled that “we're not just expanding access to medical imaging data through cloud-based tools; we are changing how that data can be used to drive AI advances in healthcare.”