Two years ago, the joint project EVA-KI, funded by the Federal Ministry of Health and headed by TU computer scientist Dr. Anirban Mukhopadhyay, was launched. What makes it unique is that for the first time, computer scientists, doctors from two German university hospitals, experts from the medical software industry and experts on the approval of medical products are working hand in hand. Together they want to establish a platform for the development and validation of AI solutions in clinical routine.

From static to continuous learning

When translating promising research results into clinical practice, an important hurdle must be overcome: the transition from static to continuous learning, i.e. training models throughout their entire life cycle. Continuous learning continues to gain momentum, but still has a long way to go in the health sector. Both the methods and the legal regulation need to be rethought and tested for this.

Through the joint project, the researchers have very well-functioning collaborations between the technical, clinical and regulatory working worlds. They can train the algorithms with optimally prepared data and then have them tested under realistic conditions in the clinics. In return, the medical professionals receive an AI solution that is constantly improved and adapted to their needs.

Cooperation between the technical, clinical and regulatory working worlds

Mukhopadhyay and his team receive anonymised medical image data from the university hospitals, which experts have already uniformly structured with additional information using special software. For each CT scan of a pulmonary embolism, for example, they receive certain basic data as well as an evaluation of the lung lesions to be detected. All information has a predefined structure. The AI learns from this data which structures it should classify and how. Using trained AI image recognition, clinicians can also analyse new images more quickly and accurately. In turn, user statistics and user feedback help the researchers to improve the algorithms. An ideal cycle.

To ensure that the AI solutions developed can be used outside of research, the scientists are also working closely with a medical regulation company. This is to ensure that the requirements for approval in various countries are already taken into account during development. The hurdles that have to be overcome are described by the project participants on the basis of the fictional company TuMEDa.

Standardised framework Lifelong nnU-Net

In future, other researchers and clinicians should also benefit from the results of the joint project EVA-KI: With their publication “Lifelong nnU-Net: a framework for standardised medical continual learning”, Mukhopadhyay's team presented a library of code and trained models created in the project. It was published in June 2023 in the renowned open access journal Nature Scientific Reports.

The standardised framework Lifelong nnU-Net enables researchers and clinicians to perform sequential training and testing of models for continuous segmentation. By releasing their code and trained models to open-source datasets on GitHub, the Darmstadt researchers are creating a benchmark for evaluating future continuous learning methods for segmentation models. The goal of Lifelong nnU-Net is to ensure high technical standards and reproducible results. Building on the proven segmentation method nnU-Net it ensures broad applicability and lower the hurdle for the continuous evaluation of new methods.

Recently, the US Food and Drug Administration (FDA) published a draft on Predetermined Change Control Plan in place for regulating Continually learning AI as a Medical Device. It aims to support the iterative improvement of machine learning-enabled device software functions while assuring their safety and effectiveness. The EVA-KI partners will therefore continue the project with a focus on bringing together the silos of technical and regulatory affairs and will prepare several white papers. This effort will be critical to align EU AI-act with the EU Medical Device Regulations.