A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks.

in Nature biomedical engineering by Yue Sun, Limei Wang, Gang Li, Weili Lin, Li Wang

TLDR

  • The study develops a foundation model for motion correction, resolution enhancement, denoising, and harmonization of MR images using a tissue-classification neural network and 'tissue-aware' enhancement network.
  • The model outperforms state-of-the-art algorithms in improving image quality, handling pathological brains, and generating high-resolution images.

Abstract

In structural magnetic resonance (MR) imaging, motion artefacts, low resolution, imaging noise and variability in acquisition protocols frequently degrade image quality and confound downstream analyses. Here we report a foundation model for the motion correction, resolution enhancement, denoising and harmonization of MR images. Specifically, we trained a tissue-classification neural network to predict tissue labels, which are then leveraged by a 'tissue-aware' enhancement network to generate high-quality MR images. We validated the model's effectiveness on a large and diverse dataset comprising 2,448 deliberately corrupted images and 10,963 images spanning a wide age range (from foetuses to elderly individuals) acquired using a variety of clinical scanners across 19 public datasets. The model consistently outperformed state-of-the-art algorithms in improving the quality of MR images, handling pathological brains with multiple sclerosis or gliomas, generating 7-T-like images from 3 T scans and harmonizing images acquired from different scanners. The high-quality, high-resolution and harmonized images generated by the model can be used to enhance the performance of models for tissue segmentation, registration, diagnosis and other downstream tasks.

Overview

  • The study aims to develop a foundation model for motion correction, resolution enhancement, denoising, and harmonization of magnetic resonance (MR) images.
  • The model uses a tissue-classification neural network to predict tissue labels, which are then used by a 'tissue-aware' enhancement network to generate high-quality MR images.
  • The study validates the model's effectiveness on a large and diverse dataset consisting of 2,448 deliberately corrupted images and 10,963 images spanning a wide age range and acquired using various clinical scanners.

Comparative Analysis & Findings

  • The model consistently outperformed state-of-the-art algorithms in improving the quality of MR images, particularly in handling pathological brains with multiple sclerosis or gliomas.
  • The model generated high-quality, high-resolution images that can be used to enhance the performance of models for tissue segmentation, registration, diagnosis, and other downstream tasks.
  • The model was able to generate 7-T-like images from 3 T scans and harmonize images acquired from different scanners, demonstrating its ability to handle varying imaging protocols and scanner types.

Implications and Future Directions

  • The development of this foundation model has the potential to significantly improve the quality of MR images, enabling more accurate diagnosis and treatment of various medical conditions.
  • Future studies can explore the application of this model to other types of medical imaging modalities, such as computed tomography or positron emission tomography scans.
  • The model's architecture and training data can be further refined and extended to improve its performance and adaptability to different imaging protocols and scanner types.
Read Full Article