Lifelong learning for Medical Artificial Intelligence

TU computer scientists create standardised framework for developing and validating clinical AI solutions

2023/08/01 by

Doctors often rely on imaging techniques such as computed tomography (CT) for a quick and correct diagnosis. For some time now, they have been supported in the analysis of CT scans by artificial intelligence (AI). This can be trained, for example, to segment different types of tissue in the image file, i.e. to distinguish them precisely from one another and to mark them in colour. Computer scientists at TU Darmstadt want to accelerate the further development of these AI solutions in the hospital environment in a unique interdisciplinary project.

Two years ago, the joint project EVA-KI, funded by the Federal Ministry of Health and headed by TU computer scientist Dr. Anirban Mukhopadhyay, was launched. What makes it unique is that for the first time, computer scientists, doctors from two German university hospitals, experts from the medical software industry and experts on the approval of medical products are working hand in hand. Together they want to establish a platform for the development and validation of AI solutions in clinical routine.

From static to continuous learning

When translating promising research results into clinical practice, an important hurdle must be overcome: the transition from static to continuous learning, i.e. training models throughout their entire life cycle. Continuous learning continues to gain momentum, but still has a long way to go in the health sector. Both the methods and the legal regulation need to be rethought and tested for this.

Dr. Anirban Mukhopadhyay,,
Leader of MEC-Lab at the Department of Computer Science

Medical knowledge is dynamic, always adjusting to newer knowledge, updated medical practices or when new diseases arise (e.g. the pandemic). Yet the only type of learning models that are currently allowed to help patients in US or EU are the static learning models. We created the EVA-KI consortia to take a holistic approach of bringing continual learning in healthcare, from the technical, clinical and regulatory perspectives.

Picture: TU Darmstadt/GRIS

Through the joint project, the researchers have very well-functioning collaborations between the technical, clinical and regulatory working worlds. They can train the algorithms with optimally prepared data and then have them tested under realistic conditions in the clinics. In return, the medical professionals receive an AI solution that is constantly improved and adapted to their needs.

Cooperation between the technical, clinical and regulatory working worlds

Mukhopadhyay and his team receive anonymised medical image data from the university hospitals, which experts have already uniformly structured with additional information using special software. For each CT scan of a pulmonary embolism, for example, they receive certain basic data as well as an evaluation of the lung lesions to be detected. All information has a predefined structure. The AI learns from this data which structures it should classify and how. Using trained AI image recognition, clinicians can also analyse new images more quickly and accurately. In turn, user statistics and user feedback help the researchers to improve the algorithms. An ideal cycle.

To ensure that the AI solutions developed can be used outside of research, the scientists are also working closely with a medical regulation company. This is to ensure that the requirements for approval in various countries are already taken into account during development. The hurdles that have to be overcome are described by the project participants on the basis of the fictional company TuMEDa.

Standardised framework Lifelong nnU-Net

In future, other researchers and clinicians should also benefit from the results of the joint project EVA-KI: With their publication “Lifelong nnU-Net: a framework for standardised medical continual learning”, Mukhopadhyay's team presented a library of code and trained models created in the project. It was published in June 2023 in the renowned open access journal Nature Scientific Reports.

The standardised framework Lifelong nnU-Net enables researchers and clinicians to perform sequential training and testing of models for continuous segmentation. By releasing their code and trained models to open-source datasets on GitHub, the Darmstadt researchers are creating a benchmark for evaluating future continuous learning methods for segmentation models. The goal of Lifelong nnU-Net is to ensure high technical standards and reproducible results. Building on the proven segmentation method nnU-Net it ensures broad applicability and lower the hurdle for the continuous evaluation of new methods.

Currently, research is very focused on improving the performance of AI models in synthetically designed test scenarios. However, these models are often unable to meet the dynamic requirements of the real world. We want to find a middle ground that allows us to safely test and develop innovative AI solutions in real clinical scenarios

Camila González, Alumna of MEC-Lab and Post-doc at Stanford University

Recently, the US Food and Drug Administration (FDA) published a draft on Predetermined Change Control Plan in place for regulating Continually learning AI as a Medical Device. It aims to support the iterative improvement of machine learning-enabled device software functions while assuring their safety and effectiveness. The EVA-KI partners will therefore continue the project with a focus on bringing together the silos of technical and regulatory affairs and will prepare several white papers. This effort will be critical to align EU AI-act with the EU Medical Device Regulations.

Joint project EVA-KI - Establishment of a platform for the development and validation of AI solutions in clinical routine

The consortium of researchers from Darmstadt, Cologne, Essen, Göttingen and Munich is led by Dr Anirban Mukhopadhyay, head of the independent research group MEC-Lab at the Department of Computer Science at TU Darmstadt.

TU-Alumna Camila González, a computer scientist who played a key role in the development of the MEC-Lab during her doctorate, moved to Stanford University as a post-doctoral researcher.

The Federal Ministry of Health has been funding the project since the end of 2020 with 1.7 million euros as part of the digital innovations for improving patient-centred care in the healthcare system.

Read more