Approved Research

3D reconstruction of Human Anatomy Structure from Medical Imagining and Text Description

Lay summary

Traditional medical imaging, such as X-rays, CT scans, and MRI scans, have been instrumental in diagnosing and treating a wide variety of medical conditions. However those 2D image or 3D volume data are difficult to interpret and visualize in a way that accurately represents the full 3D structure of the body. This can make it challenging for medical professionals to get a complete picture of a patient's anatomy and identify abnormalities. This inspired lots of research in the past 20 years on medical image segmentation and 3D visualization. Given high resolution imagining and dense scan, the state-of-the-art techniques can reconstruct the 3D model accurately.

However, many medical imaging methods, such as X-rays and CT scans, use ionizing radiation to create images. While the doses used are generally considered safe, repeated exposure can increase the risk of cancer or other radiation-related health problems. Some medical imaging methods can be time-consuming and require patients to lie still for an extended period of time. This can be difficult for some patients, such as those with claustrophobia or mobility issues. Besides, medical imaging methods can be expensive in terms of the equipment and staff cost. This leads to only a very limited imaging could be acquired for each patient, which is normally not enough for the 3D reconstruction to show a complete visualization of the patient's problem.

Our goal is to reduce the number of medical image captures and supply a 3D visualization of patient's anatomy using machine learning techniques. Medical data are normally multimodality (image, text, audio, video etc), complex with information from various channels. Machine learning technique have been proven very efficient in processing large data, learn their structures, extract useful features, perform automated analysis, enabling faster and more consistent reasoning. This can improve efficiency in medical workflows, reduce errors, and facilitate more informed decision-making by healthcare providers.

Our project aims to develop a few new machine learning models to 1) train a parametric 3D human skeleton model that can represent individual patient skeletal structure, and 2) efficiently and accurately reconstruct 3D human anatomy from medical images and report.

Our project is expected to last three years and will have a significant impact on productivity in the NHS at scale. By enabling doctors to make more informed decisions and helping professionals and patients understand medical conditions and treatments, our technology has the potential to improve healthcare outcomes.