Toolbox for microscope data analysis
Proof of Concept: Professor Molina-Luna awarded 150,000 euros grant by the European Research Council
2020/04/27
Two scientists at the TU Darmstadt have been awarded a “Proof of Concept” grant of 150,000 euros by the European Research Council (ERC). Professor Leopoldo Molina-Luna received the award for his project “STARE”. We shall be presenting the other recipient of the grant, the “LONGSENSE” project by Professor Heinz Koeppl, in the next few days.
“STARE” aims to develop a comprehensive software toolkit that includes all the necessary procedures for a user-friendly analysis of transmission electron microscopy data in real time and during ongoing experiments. The idea for “STARE” came from a current ERC grant project by Professor Molina-Luna.
Professor Molina-Luna, the European Research Council, ERC, has decided to support your Proof of Concept project “STARE – Machine learning based Software Toolkit for Automated Identification in atomic-resolution operando nanoscopy” with 150,000 euros. What is the project actually about? What is the role played by artificial intelligence?
Transmission electron microscopy (TEM) is a fundamental technology due to its unsurpassed ability to image and characterise matter on sub-nanometer length scales. TEM has significant implications for various industrial segments, including electronics and semiconductors, as well as metallurgical alloys, polymers, medical and biological systems. In these times, for instance, Cryo-TEM plays an essential role in the structural investigation of the coronavirus. TEM enables continuous progress in the development of improved, advanced materials and opens up complex relationships between structure and property.
Today, there is a strong, unmet need for the development of real-time technology, automated identification algorithms in TEM for the analysis of atomic structure, phases and defects. For example, TEM plays an important role in the fundamental understanding of strengthening in materials by clarifying the role of dislocations. It is not usually trivial to obtain or extract meaningful scientific information from the raw digital data of TEM output. It requires a lengthy signal processing routine and the material understanding and expertise of an experienced microscopist.
Thanks to the rapid development of information technology and computer science, automated computer-aided analyses of electron microscope images or data have become a reality. Over the last ten years, various tools have been developed and applied to digital data analysis. In the meantime, the rapid development of microscopy has led to novel techniques and instrumentation, such as in situ/operando TEM and pixelated detector techniques that require great skill to execute and analyse data. While existing solutions are time-consuming, from the manual analysis to software solutions, few of them use machine learning or, to use a different expression, image feature pairing.
Proof of Concept projects are intended to contribute to the practical implementation of a research result from an ERC grant project. What is the significance of your ERC grant project FOXON, from which STARE has now developed?
In order to improve the performance of the functional materials at a lower cost, TEM is expected to analyse large amounts of data in a shorter period of time. It is based on the use of high-performance computers. This trend is expected to continue, as all current and short-term computational approaches require integrated multi-material systems with a wide range of material requirements that go beyond the design of pure computing functionality.
From a technical perspective, TEM has made significant progress in recent years. That said, a major bottleneck in the penetration of TEM in routine practice has still not been addressed, and this is reflected in a lack of means of efficient analysis. The analysis of large data sets (big data) is becoming increasingly important in various fields, such as materials science and engineering. The aim of FOXON is to generate a real-time analysis of multidimensional data during the execution of an experiment.
What are the possibilities of your research, which is now to be developed for market readiness with the STARE project? What concrete applications do you see in the medium or long term?
The proposed software toolkit differs from other existing packages in the following aspects. First, this toolkit is not based on a “black box”, but rather on an open-source python environment. Second, it will include all the necessary analytical methods, including noise reduction, data filtering, customisation and location identification. One important USP of our software idea is the fact that we have created a comprehensive framework for obtaining real-time feedback from the experiment. Unlike other commercial software solutions, we are striving for a module that can be integrated in existing systems.
STARE stands for an end-to-end TEM image analysis framework that provides real-time feedback on the experiment. Furthermore, one of STARE's goals is to improve the user interface of the software and make it more accessible to non-experts. There has also recently been an extremely exciting and promising cooperation with the TU’s IT Department and the group of Professor Kristian Kersting (Department of Artificial Intelligence and Machine Learning, Department of Computer Science), from which we are receiving genuine expert support in the development of our idea. We are also working with various partners in industry.
About the individual
Leopoldo Molina-Luna is Assistant Professor of Electron Microscopy in the Field of Material Sciences, Department of Materials and Geosciences, at TU Darmstadt. After completing his doctorate in physics at the Eberhard Karls University of Tübingen, he was a postdoctoral fellow at one of the world's leading centres for electron microscopy, the EMAT in Antwerp. His postdoctoral fellowship at the University of Antwerp was funded by an ERC Advanced Grant. Since 2018, Professor Molina-Luna's project „FOXON – Functionality of Oxide based devices under Electric-field: Towards Atomic-resolution Operando Nanoscopy“ has been funded by an ERC Starting Grant worth approximately 1.8 million euros.
Proof of Concept
Proof of Concept is a supplementary grant to the research grants of the ERC. It is aimed exclusively at scientists who already hold an ERC grant and who want to pre-commercialise a research result from a current or completed project. This is the first step towards a technology transfer. The aim of a Proof of Concept project is to check the market potential of such an idea. So rather than funding research activities with it, the ERC finances measures for further development with a view to the maturity of application, commercialisation or marketing of the idea. In the most recent funding round, 55 researchers from 17 countries were awarded an ERC Proof of Concept grant. The grants are part of the EU's Horizon 2020 research and innovation programme.
Questions asked by Silke Paradowski.
