Doctoral Students

Contact

Supervisors:

Martin Herold (GFZ)

Nadja Klein (HU)

Weeds are significant contributors (about 12% of global crop production) to crop yield and quality decline. Farmers use different approaches such as chemical or biological herbicides to eliminate weeds. However, excess use of herbicides leads to the pollution of soils, water, and air, putting the above and below ground wildlife biodiversity at risk. Alternative weed mitigation strategies must be designed and promoted. The site-specific weed management (SSWM) approach has been proposed consisting of varying weed management strategies within a crop field to suit the weed population's variation in density, location, and composition. The first step in implementing a SSWM strategy is accurate and timely detection and mapping of weeds. The high temporal, spatial, and spectral remote sensing information is required to capture detailed within-field variability, which can only be met by using the Low Altitude Remote Sensing platform and lightweight hyperspectral imaging sensors that can be deployed locally and at varying conditions.

This research aims to use hyperspectral multi-temporal Low altitude remote-based time-series imagery combined with field data and advanced machine learning (ML) techniques to detect and discriminate weeds in croplands. To achieve this aim, the project will i) monitor the weed population during different growth stages and preprocess of hyperspectral data and field data for better detecting weeds, ii) test a combination of ML and image processing algorithms for detecting weeds across a range of field and growing stage conditions, iii) apply the framework for multi-temporal analysis to track weed distribution and conditions for theuptake by precision farming techniques in our study regions.

Full-length publications

1. E. Celikkan, M. Saberioon, M. Herold and N. Klein (2023). Semantic Segmentation of Crops and Weeds with Probabilistic Modeling and Uncertainty Quantification. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 582-592.

Conference presentations

-

Contact

Supervisors:

Yuri Shprits (GFZ)

Guillermo Gallego (TU)

Space weather is a term used to describe hazardous events in the near-Earth space environment that can have adverse effects. Power grids, telecommunication infrastructure and space assets show significant vulnerability to space weather events originating from the Sun. While these effects are largely invisible to the naked eye, in the 21st century operations in space and on the ground significantly depend on the accurate knowledge and forecast of the conditions in the space environment.

Using ground observations of the Sun and physics-based models of the Solar System, it is possible to quantify space weather hazards (e.g., solar wind speed and density) and risks on Earth’s technology. The output of this method provides an estimate of the desired variables. These methods, however do not utilize the vast amount of observations that are available from space, and they are computationally demanding and dominantly physics-based, making them difficult to run in real-time and use all available measurements.

We propose to leverage the growing number of highly detailed multi-spectral images of the Sun to improve predictions of the solar wind streams arriving at the Earth. We propose to exploit the capabilities of modern computer vision and machine learning (ML) techniques to register solar images, analyze them and assimilate them in an empirical (data-driven) model. Through our approach we will develop a novel, data-driven framework directly connecting solar disturbances to its consequences for space weather.

Full-length publications

-

Conference presentations

1. D. Collin, S. Bianco, G. Gallego, and Y. Shprits. Forecasting solar wind speed from solar EUV images. (Oral and poster presentation), International Workshop on Machine Learning and Computer Vision in Heliophysics, Sofia, Bulgaria, 19-21 April 2023.
2. D. Collin, S. Bianco, G. Gallego, and Y. Shprits. Forecasting solar wind speed by machine learning based on coronal hole characteristics. (Poster presentation), EGU General Assembly , Vienna, Austria, 24–28 April 2023. https://doi.org/10.5194/egusphere-egu23-6968
3. D. Collin, S. Bianco, G. Gallego, Y. Shprits (2023). Forecasting solar wind speed from solar EUV images. (Oral presentation), IUGG General Assembly, Berlin, Germany, 11-20 July 2023. https://doi.org/10.57757/IUGG23-2070

Contact

Supervisors:

Ulrike Herzschuh (AWI)

Guido Grosse (AWI)

Birgit Kleinschmit (TU)

Our goal is to employ vomputer vision and data science methods to advance the data handling, analyses and interpretation of the wealth of Big Data 3-D remotely sensed environmental data acquired by the AWI on polar expeditions. Objective: Data acquisitions from drone-borne and airplane passive and active optical imaging sensors over large areas in Siberia and Alaska resulted in multi-temporal datasets of billions of remotely sensed points in spatially explicit point clouds. Computer vision already enables notable advancements in 2-D Big Data environmental data science. In addition, a wide range of 3-D sensors is employed to investigate polar terrestrial environments and permafrost landscapes: ground-based, drone-borne, and airplane active optical laser scanning devices and passive optical densely overlapping imaging from different view angles provide 3-D point cloud data consisting of billions of individual measurements of surface structures, including vegetation and permafrost landscape topography in the circumpolar region. Approach: 3-D point clouds containing billions of remotely sensed points over large areas are the products from the drone-borne and airplane LIDAR as well as high-resolution Red Green Blue and Red Green Near Infrared cameras that allow stereo photogrammetric derivation of point clouds. We need machine learning on the Big Data 3-D point clouds that will also allow us to analyse the characteristics and interactions among the points in 3-D space to enable recognition of tree species and terrestrial degradation features (classification), to enable segmentation to identify the meaning of the environmental objects, and to develop advanced automated change detection tools for multi-temporal point cloud datasets. We will apply two use cases using high latitude biodiversity and permafrost landscape diversity.

Full-length publications

-

Conference presentations

1. V. Döpper et al. Unlocking the Potential of Arctic to Boreal Multi-Source Point Clouds: Deep and Transfer Learning for Automated Segmentation and Classification. (Poster presentation), SilviLaser 2023, London, UK, 6-8 September, 2023.

Contact

Supervisors:

Mahdi Motagh (GFZ)

Setareh Maghsudi (Uni Tübingen)

Enabled by recent technological advances, the field of radar remote sensing has entered the era of explosively-growing wide-swath Synthetic Aperture Radar (SAR) missions with short revisit times (1-6 days), such as Sentinel-1 and the planned Tandem-L and NISAR missions, providing an unprecedented wealth of topography and surface change time-series using interferometric SAR (InSAR) technique. Such data volume can be characterized with (i) huge volume and large variety; (ii) complexity and high dimension; (iii) partial unreliability; and (iv) correlation or similarity. Thus, for the retrieval of geophysical signal from InSAR time-series, the data should be classified, clustered and cleaned, performed by data analytics, to reliably detect anomaly changes and improve susceptibility in the areas affected by deformation due to natural and manmade hazards. In the state of- the-art literature for exploiting InSAR time-series data, the separation of geophysical signals from noise artifacts such as atmosphere and decorrelation can be divided into two main parts: (i) data processing for efficient estimation of the phase considering long stack of the data; and (ii) data analysis and decision.

In brief, the main goal of this project is to develop a generic framework for real-time InSAR data analytics using the theory and methods from online machine learning and sequential decision making, and to design efficient algorithmic solutions for the retrieval of geophysical signals from SAR measurement. In particular, the concentration is on SAR data from Sentinel-1 satellite. On the online learning and classification side, the methodology is concentrated on online machine learning algorithms. Specific attention is given to submodular optimization. On the online decision making side, the basic method is sequential optimization with limited feedback, especially multi-armed bandit.

Full-length publications

1. B. Ghosh, M. Motagh, M. Haghshenas Haghighi, M. Stefanova Vassileva, T. Walter, and S. Maghsudi (2021). Automatic detection of volcanic unrest using blind source separation with a minimum spanning tree based stability analysis. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. https://doi.org/10.1109/JSTARS.2021.3097895

2. B. Ghosh, M. Haghshenas Haghighi, M. Motagh, and S. Maghsudi (2021). Using Generative Adversarial Networks for extraction of InSAR  signals from large-scale Sentinel-1 Interferograms by improving tropospheric noise correction.  ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-3-2021, 57–64. https://doi.org/10.5194/isprs-annals-V-3-2021-57-2021
3. B. Ghosh, S. Garg, and M. Motagh (2022). Automatic flood detections from Sentinel-1 data using deep learning architectures. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-3-2022, 201–208. https://doi.org/10.5194/isprs-annals-V-3-2022-201-2022.

Conference presentations

1. B. Ghosh, M. Motagh, S. Maghsudi, and M.H. Haghighi. Reduction of tropospheric noise delay from large-scale interferograms using Generative Adversarial Networks. (Oral presentation), 40th Annual Scientific and Technical Conference of the DGPF, Stuttgart, Germany, 4 - 6 March 2020.

2. B. Ghosh, M. Motagh, S. Maghsudi, and M.H. Haghighi. Automatic flood monitoring based on SAR intensity and interferometric coherence using machine learning. (Oral presentation), EGU General Assembly, Online, 4-8 May, 2020.

3. B. Ghosh, M. Motagh, M.H. Haghighi, and T. Walter. Using minimal spanning tree based ICA optimization for volcanic unrest determination. (Oral presentation), EGU General Assembly, Online, 19-30 April, 2021.

4. B. Ghosh, M.H. Haghighi, M. Motagh, and S. Maghsudi. Using generative adversarial networks for extraction of InSAR signals from large-scale Sentinel-1 interferograms by improving tropospheric noise correction.  (Oral presentation), ISPRS Congress, 4-10 July 2021.

5. B. Ghosh, M. Motagh, S. Garg, M. Sips, and D. Eggert. Deep learning, remote sensing and visual analytics to support automatic flood detection. (Oral presentation), EGU General Assembly, Vienna, Austria & Online, 23-27 May 2022.
6. B. Ghosh, S. Garg, M. Motagh. Automatic flood detection from Sentinel-1 data using Deep learning architectures. (Oral presentation), ISPRS Congress, Nice, France, 6-11 June 2022. 

Contact

Supervisors:

Natalia Kliewer (FU)

Carolin Ulrbrich (HZB)

Rutger Schlatmann (HZB)

The number of photovoltaic (PV) systems installed worldwide is steadily increasing, along with their share in the energy produced in the power grid. Getting the maximum benefit from these PV systems involves two primary considerations: maximizing power generation over the entire lifetime of the PV modules and optimally integrating them into the power grid.

Getting the maximum output from a given PV installation requires continuous monitoring to detect underperformance and failures as early as possible. However, this type of monitoring is seldom implemented, particularly outside of utility-scale photovoltaic plants or research-focused installations. These facilities are equipped with all the instruments required for accurate monitoring, in contrast to residential and commercial installations.

The capabilities of modern data analytic methods can help raise the proportion of PV installations that can be monitored. In addition, data-driven modeling can make monitoring a PV system simpler: data-driven models do not require detailed information about the system, and they can also combine data from sources that are available outside of plants and research centers, but which standard, physics-based modeling approaches can't take advantage of.

One thing that could hinder the integration of PV systems into the power grid is the stochastic nature of solar energy. This stochasticity could lead to adverse effects such as over- and under-production and challenge the stability of power grids. This means that in addition to monitoring, the integration will also require the most accurate possible forecasting of expected power to control the grid optimally and avoid unnecessary stresses on it. Data-driven methodologies can help make the implementation of such best practices more accessible and more widely available, benefiting both PV system owners and power grid managers.

In this project, we investigate data-driven monitoring and forecasting systems for photovoltaic installations. As an application, we study the potential and benefits of PV power monitoring and forecasting for load management in support of electric mobility integration in the public transport system. We focus specifically on electric buses and the load generated from charging their batteries. Suppose the charging tasks are performed in an uncontrolled way. In that case, the large electric loads of the batteries might cause instability of the power grid and reduce the environmental benefits of such electric vehicles. Contrary to this, load management coordinates the charging processes to benefit as much as possible from renewable energy sources while preserving the power grid stability.

The coordination of energy generation from PV systems and energy consumption by electric vehicles is a key to a green transition in the transport sector.

Full-length publications

1. P. Graniero, M. Khenkin, H. Köbler, N.T. Putri Hartono, R. Schlatmann, A. Abate, E. Unger, T.J. Jacobsson and C. Ulbrich (2023). The challenge of studying perovskite solar cells’ stability with machine learning. Front. Energy Res., Sec. Solar Energy, 11. https://doi.org/10.3389/fenrg.2023.1118654
2. N.T.P. Hartono, H. Köbler, P. Graniero, et al. (2023). Stability follows efficiency based on the analysis of a large perovskite solar cells ageing dataset. Nat Commun 14, 4869. https://doi.org/10.1038/s41467-023-40585-3

Conference presentations

1. P. Graniero, A. Louwen, R. Schlatmann, and C. Ulbrich. Comparison of different data sources for  Machine Learning algorithms in photovoltaic output power estimation. (Poster presentation), 37th EU PVSEC, Online, 7-11 September 2020.
2. P. Graniero, D. Rößler, C. Ulbrich, and N. Kliewer. Potentials and challenges for integration of electric bus fleets and PV-systems. (Oral presentation), 31st European Conference on Operational Research (EURO 2021), Athens, Greece / Online, 11-14 July 2021.
3. P. Graniero. Data driven mitigation measures in advanced PV plant monitoring. (Oral presentation), Intersolar Conference, Munich, 6-7 October 2021.
4. P. Graniero. Comparison of Unsupervised Algorithms for PV Fault Detection, and Data Sources for Power Nowcasting. (Oral presentation), Cost Action Pearl PV’s Conference- Enabling the Terawatt Transition,  Enschede, The Netherlands, 14-16 March 2022.
5. P. Graniero, G.A.F. Basulto, R. Schlatmann, R. Klenk, and C. Ulbrich. Online Implementation of a Multiple Linear Regression Model for CIGS Photovoltaic Module Performance. (Poster presentation), WCPEC-8, Milan, Italy,  26-30 September 2022.

Contact

Supervisors:

Julia Boike (AWI)

Guillermo Gallego (TU)

The Arctic is among the most vulnerable regions to recent warming trends in Earth’s climate. The sparsity of available historical data in the region necessitates the use of numerical models to better understand the effects of climate change on Arctic landscapes and ecosystems. There remains a notable gap, however, between realistically modeling the highly dynamic nature of Arctic landscapes and efficiently scaling to longer time-frames or global studies.

Physics-informed machine learning has the potential to help bridge this gap by providing tools that better leverage a wide variety of available data sources, thereby potentially providing new insights into the processes driving changes in Arctic environments.

This project aims to 1) build a modern, data-driven framework for land surface modeling in the Arctic and 2) apply these tools to better quantify and explain major sources of uncertainty in modeling permafrost processes.

Full-length publications

1. S. Westermann, T. Ingeman-Nielsen, ..., J. Boike, B. Groenke, ..., and M. Langer (2023). The CryoGrid community model (version 1.0) – a multi-physics toolbox for climate-driven simulations in the terrestrial cryosphere. Geoscientific Model Development, 16(9), pp. 2607-2647.
2. B. Groenke, M. Langer, J. Nitzbon, S. Westermann, G. Gallego, and  J. Boike, (2023). Investigating the thermal state of permafrost with Bayesian inverse modeling of heat transfer. The Cryosphere, 17, pp. 3505–3533. https://doi.org/10.5194/tc-17-3505-2023

Conference presentations

1. B. Groenke, M. Langer, G. Gallego, and J. Boike. Learning soil freeze characteristic curves with universal differential equations. (PICO presentation), EGU General Assembly, Online, 19–30 Apr 2021. https://doi.org/10.5194/egusphere-egu21-13409
2. B. Groenke, M. Langer, G. Gallego and J. Boike. A model-driven approach to quantifying uncertainty in permafrost temperature trends. (Poster presentation), 6th Data Science Symposium, Bremen, Germany, 8-9 Nov 2021.
3. B. Groenke, F. Miesner, M. Langer, G. Gallego and J. Boike. An energy conserving method for simulating heat transfer in permafrost with hybrid modeling. (Oral presentation), Climate Informatics 2022, virtual, 9-13 May 2022.
4. B. Groenke, M. Langer, G. Gallego and J. Boike. A probabilistic analysis of permafrost temperature trends with ensemble modeling of heat transfer. (PICO presentation), EGU General Assembly, Vienna, Austria, 23–27 May 2022. https://doi.org/10.5194/egusphere-egu22-10509
5. B. Groenke, M. Langer, G. Gallego, and J. Boike: Exploring physics-informed machine learning for accelerated simulation of permafrost processes. EGU General Assembly, Vienna, Austria, 24–28 April 2023. https://doi.org/10.5194/egusphere-egu23-10135
6. B. Groenke, M. Langer, J. Nitzbon, S. Westermann, G. Gallego, and J. Boike (2023). Explaining uncertainty in the thermal state of permafrost with Bayesian inversion of hydrothermal dynamics. 6th European Conference on Permafrost, Puigcerdà, Catalonia, Spain, 18-22 June 2023.
7. B. Groenke, M. Langer, G. Gallego, and J. Boike (2023). Applications of physic-informed machine learning in accelerating dynamical models of permafrost processes. IUGG23 General Assembly, Berlin, 11-20 July, 2023.

Personal homepage

Contact

Supervisors:

Uwe Ohler (MDC)

Markus Schuelke-Gerstenfeld (Charité)

Single-cell genomics can obtain molecular data for tens of thousands of cells simultaneously. A typical experiment is carried out on a complex sample that contains different cell types, and can measure different cellular properties, such as the number of messenger RNA molecules per cell. Typical tasks include identifying distinct cell types (e.g. via unsupervised embeddings, [1]) or inferring a pseudo-temporal ordering of cells along developmental stages. A particular opportunity arises from single-cell genome accessibility data, which provides information about which of several million gene switches, so called regulatory regions, are accessible/on or inaccessible/off [2]. These data can be analyzed at multiple resolutions: At the level of whole regions, to identify where active switch regions are and to infer which genes they may regulate, or at the level of short DNA sequence patterns within the regions, which are recognized by proteins to specifically activate the switches in e.g. different cell-types. Models to utilize the power of single-cell genomics data, and accessibility in particular, are still in their infancy. The main challenge is that the higher number of cells (i.e. samples) is accompanied by high dropout: the readout covers only a few percent of all variables, and the resulting discrete count data is sparse. Additionally, ground truth experimental data only exists for a handful of scenarios, making it hard to develop practically useful methods that work beyond simulated data.

The project will utilize data from the Schuelke lab to develop deep neural network approaches in the Ohler lab that enable flexible multi-resolution analyses: the goal is to devise models that are able to infer both active regulatory regions and the functional sequence patterns in them, while (a) leveraging data from smaller or larger cell neighborhoods as needed; (b) accounting for confounders such as variable dropout and cell type mixtures; and (c) utilizing auxiliary data from other single-cell experiments.

Full-length publications

-

Conference presentations

-

Contact

Supervisors:

Björn Scheuermann (HU)

Winfried Halle (DLR)

A general trend in remote sensing is the simultaneous increase in the number of spectral bands and the geometric resolution. Data rates and data volumes approach the physical limits of onboard memory and downlink data rates to earth. However, the feasibility of much more expensive and complex calculations directly on the satellite has been demonstrated already. Application areas beyond the early detection of fires include, for instance, the situation description after a hurricane or earthquake. For disaster and security research applications, short‐term visual and radar derived information are required to describe the situation for rescue workers and relevant services. Reconfigurable logic on FPGAs is a promising direction for low‐latency, real‐time, high‐volume data processing (also) in space. The goal of the thesis is to bring FPGA‐based in‐satellite data processing solutions to representative real‐time applications.

Full-length publications

1. O. Kondrateva, B. Scheuermann, and S. Dietzel (2022). Scalable Flow Optimization for Small Satellite Networks using Benders Decomposition. In Proceedings of the IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), 221-230. https://doi.org/10.1109/WoWMoM54355.2022.00041

Conference presentations

-

Contact

Supervisors:

Nikolaus Rajewsky (MDC)

Klaus-Robert Müller (TU)

Frederich Klauschen (Charité)

Despite enormous progress in the understanding, early diagnosis and treatment, solid tumors still account for a quarter of deaths. Solid tumors arise from somatic cells through the accumulation of molecular alterations that eventually lead to their uncontrolled proliferation, invasion of healthy tissues and ultimately, distant metastases. In current clinical practice, the best treatment is chosen by assessing clinical presentation, histological tumor type and molecular characteristics, such as oncogenic mutations and, in some cases, gene expression profiles. The evaluation of histomorphological tumor properties in combination with molecular profiles guides risk-adjusted, personalized therapies that aim to optimize outcome for individual patients. However, diagnostic molecular profiling currently mostly relies on techniques performed on bulk tissue without providing any spatial information about the observed molecular tumor properties. While this can already make the interpretation of mutational profiles challenging, spatially resolved profiling becomes indispensable for the molecular analysis of the tumor microenvironment composed of multiple different cell types whose complex interactions influence therapy response. In this context, single-cell sequencing techniques may offer a unique opportunity to offer both high spatial and molecular resolution in the context of complex tumor histology.

Recently, the fruitful interdisciplinary collaboration between clinical, technological, and computational researchers has cultivated rapid progress in the field of digital pathology. AI-based models provide support to diagnostic pathology, for instance by automating tumor identification and tissue classification and cell detection from tumor histology images. Current computational models, however, are limited in their ability to predict tumor molecular characteristics due to the scarcity of paired imaging and molecular training data. Development of spatial transcriptomics assays aim to fill this gap by enabling unbiased, transcriptome-wide profiling of mRNA expression in intact tissue sections. Thus, they represent an ideal source of such paired morphological and molecular data with unprecedented resolution for training the next generation of digital pathology algorithms.

The project aims to push forward the field of digital pathology by predicting gene expression in tissue space from histomorphology alone. To achieve that, we will train deep learning models on the high-resolution gene expression maps provided by spatial transcriptomics co-registered with the corresponding histomorphology images. Advances in explainable AI approaches will be leveraged to reveal which morphological features and areas are exploited by such models to predict gene expression. This would be vital to allow for a transparent decision process, crucial for medical applications, and would also deepen our understanding of tumor biology by correlating tissue and cellular composition/histomorphology of the tumor and its microenvironment with function. We anticipate such a computational approach to have a positive impact on clinical practices by facilitating the prediction of molecular properties from routine diagnostic H&E images and thus to complement or even partially replace molecular testing.

Full-length publications

1. T.M. Pentimalli, S. Schallenberg, D. León-Periñán, ..., F. Klauschen, and N. Rajewsky (2023). High-resolution molecular atlas of a lung tumor in 3D. bioRxiv. https://doi.org/10.1101/2023.05.10.539644

Conference presentations

-

Contact

Supervisors:

Fabrice Cotton (GFZ)

Giuseppe Caire (TU)

In many regions of the world earthquakes pose a persistent threat to the built environment, especially with respect to the civil infrastructures that are now fundamental to our society. In the aftermath of recent earthquakes, such as the 2010‐2011 Christchurch (New Zealand) events, damage to road, railway and utility/communications networks may be the dominant contributor to economic loss, with socio‐economic impacts that can last for a long period after the event and impede the recovery. The importance of analysing the seismic risk and vulnerability of spatially distributed infrastructure networks is becoming widely recognized by engineers, insurers and the scientific community at large. Such analyses present a challenge to scientists and engineers due to the complex interactions between interconnected elements within the infrastructure. The statistical models require a computational complexity so large as to prohibit the real‐time assessment of the post‐event network state. Conversely, simplified models may fail to capture the correlations and dependencies within a system in its entirety. In this project we introduce novel machine learning techniques into this process to provide statistically robust assessments of the performance of a network, in terms of both connectivity and flow, that would allow for rapid evaluation of the impact of an event for use in the immediate aftermath and recovery phase, or as part of a probabilistic assessment of economic loss.

Full-length publications

1. H. Lilienkamp, S. von Specht, G. Weatherill, G. Caire, and F. Cotton (2022). Ground-motion modeling as an image processing task: Introducing a neural network based, fully data-driven, and nonergodic approach. Bull. Seismol. Soc. Am. 112, 1565–1582. https://doi.org/10.1785/0120220008

Conference presentations

1. H. Lilienkamp, G. Weatherill, F. Cotton, and G. Caire. The role of spatial cross-correlation structures of ground motion fields forseismic risk assessment of spatially distributed assets and infrastructurenetworks.  (Poster presentation), EGU General Assembly, Vienna, Austria, 7–12 April 2019.
2. H. Lilienkamp, S. Specht, G. Weatherill, and F. Cotton. Exploring the physics and uncertainties in spatial cross-correlation models for ground motion intensity measures.  (Oral presentation), AGU General Assembly, San Francisco, USA, 9–13 December 2019.
3. H. Lilienkamp, F. Cotton, G. Caire, G. Weatherill, and S. von Specht. Fully data-driven, partially non-ergodic ground motion modeling using convolutional neural networks.  (Poster presentation), Taiwan Earthquake Research Center Annual Meeting, 20-22 October 2020.

Contact

Supervisors:

Jürgen Oberst (DLR)

Odej Kao (TU)

Mars has kilometer‐thick polar ice caps, which represent a unique laboratory to study the dynamics of ice sheets and the possible effects of climate change on a planet other than Earth. Most of the north polar cap margins are characterized by steep scarps, where avalanches and ice‐block falls are frequently observed. These cause a measurable scarp retreat, depending on season and the associated solar heating cycle. Rock falls also represent useful sources for seismic experiment and the exploration of the subsurface of the planet. The High‐Resolution Imaging Science Experiment (HiRISE) on-board the Mars Reconnaissance Orbiter (MRO) has been monitoring the planet for more than a decade, returning images at resolutions up to 25 cm, many of which in stereo. However, the identification of such small “mass wasting” effects, such as block and rock falls, is far from trivial considering the vast number of images in their differing illumination and viewing geometry. The image analysis requires new approaches of automated detection involving machine learning, which go beyond the traditional classification and regression schemes. In this project we apply modern data science techniques, such as deep convolutional networks to identify and measure rock sizes and volumes. The techniques are to be trained on established areas and ultimately be applied to new sets of images to investigate the recent climate evolution and support the seismic exploration of the planet.
 

Full-length publications

-

Conference presentations

-

Contact

Supervisors:

David Berge (DESY)

Matthias Weidlich (HU)

The sensitivity of ground-based gamma-ray telescopes is ultimately limited by their ability to reconstruct the properties of gamma-rays from the particle shower produced when they interact with the atmosphere and reject the much more numerous background of showers from charged cosmic rays. 

At this time, array sensitivity can be considered as “software limited” and as such has continuously improved in the past 20 years by exploiting advanced image reconstruction and classification algorithms [1] and multivariate classification techniques such as boosted decision trees [2] and neural networks. It is clear that using modern machine learning techniques can improve this performance further. Yet, telescope arrays typically make observations under a huge range of context conditions. As such, it is of utmost importance to integrate contextual data on the operation of the telescope as well as on atmospheric conditions into any data analysis pipeline. This is challenging not only because of the heterogeneity of contextual data, but also from a computational point of view. 

This PhD project sets out to develop data processing pipelines that combine deep learning techniques for gamma-ray telescope data with diverse types of contextual data to dramatically improve the telescopes’ sensitivity. To this end, we will draw on initial results on employing state-of-the-art machine learning techniques to gamma-ray telescope data [3] as well as technical insights into integration of static datasets into stream processing pipelines [4].

Full-length publications

-

Conference presentations

1. A. Mehta. Machine Learning for Imaging Atmospheric Cherenkov Telescope (IACT) Background Rejection. H.E.S.S. Collaboration Meeting, Bordeaux, France, 24-29 September 2023.

Contact

Supervisors:

Johann-Christoph Freytag (HU)

Marek Kowalski (DESY)

The Universe holds several avenues for the (catastrophic) end of stars. These include their gravitational collapse to a Neutron star, resulting in a so-called core-collapse Supernova, stars being swallowed by the central Black Hole of a galaxy, as well as Kilonova, the result of two merging Neutron stars recently detected for the first through the electromagnetic follow-up of a Gravitational wave event. The diversity of energetic and explosive events serves as a laboratory for fundamental physics that is explored through increasingly powerful observational facilities. With the start of the Zwicky Transient Facility (ZTF), the detection rate of time-variable phenomena in the Universe will increase by a factor 10 compared to existing surveys, far beyond what can be manually examined by astronomers. This PhD project focuses on developing new data management and machine learning approaches that will allow the scalable analysis of ZTF data through the implementation of flexible/scalable data infrastructure for classifying new transients. As the computing resources needed for this kind of computation will vary, there is also the need to manage them in an elastic manner thus leading to new monitoring and resource management strategies.

Full-length publications

1. J. Nordin, V. Brinnel, J. van Santen, ..., M. Kowalski, A. Mahabal, N. Miranda, ..., and C. Ward (2019). Transient processing and analysis using AMPEL: Alert management, photometry and evaluation of light curves. Astronomy & Astrophysics, 631, A147. https://doi.org/10.1051/0004-6361/201935634
2. Y. Yao, S.R. Kulkarni, K.B. Burdge, …, N. Miranda, ..., and M.T.  Soumagnac (2021). Multi-wavelength observations of AT2019wey: A new candidate black hole low-mass X-ray binary. The Astrophysical Journal, 920(2), 120. https://doi.org/10.3847/1538-4357/ac15f9
3. N. Miranda, J-C. Freytag, J. Nordin, R. Biswas, V. Brinnel, C. Fremling, ..., and J. van Santen (2022). SNGuess: A method for the selection of young extragalactic transients. Astronomy & Astrophysics, 665, A99.  https://doi.org/10.1051/0004-6361/202243668

Conference presentations

-

Contact

Supervisors:

Martin Kircher (Charité)

Dieter Beule (MDC)

Ulf Leser (HU)

More than 15 years after initial sequencing of the human genome, exome and whole genome sequencing are widely performed for research and clinical applications. Despite much progress, pinpointing the few phenotypically causal variants among the millions of variants in our genomes remains a major challenge. To illustrate that problem, the NCBI dbSNP and ClinVar databases report almost 700 million variants discovered in healthy and diseased humans, but only about 500,000 (<0.1%) are clinically or functionally characterized. For diagnostics, it is critical to identify the causal variants among thousands of variants with no physiological effect.
Many algorithms for predicting functional impact of variants were proposed in the past, but are largely limited to highly conserved positions in protein-coding sequences and do not interpret variants genome wide. Kircher et al. previously developed a computational method (Combined Annotation Dependent Depletion, CADD) that combines diverse annotations – from large-scale epigenetic experiments, comparisons of genomes across species, to gene model annotations. Using a linear model, CADD integrates available information in a unified framework and quantifies organismal deleteriousness on a whole-genome and variant-specific scale. While CADD has been successfully applied in thousands of disease studies, its best performance is still observed for the interpretation of variants in and around protein coding genes.
Reasons for that are manifold, ranging from the lack of domain-specific features (e.g. non-coding sequence species, regulatory sequences, 3D genome architecture) to shortcomings in the actual model (e.g. non-linearity, missing feature interactions, mislabeling of the training data). Here, we propose a joint effort, between a group that routinely analyses and interprets genetic data from individual patients, families or large research cohorts and a research group developing computational methods for variant prioritization, to significantly improve the current method and advance the automatic reporting of potentially clinically relevant variants from genetic data. For this purpose, our project has the following aims: (1) Establishing model training for unlabeled or mislabeled data, for example by semi-supervised and iterative learning approaches. (2) Systematic exploration of feature interactions and non-linearity, including but not limited to alternative learning approaches like boosting trees and neural networks, feature transformations, or automated selection of interactions terms. (3) Integration of large sets of correlated annotations (e.g. ENCODE, IHEC) through dimensionality reduction, orthogonalization approaches, parallelization and training via subsampling, or hierarchical integration of models. (4) Using additional and genome-wide available measures of sequence constraint like population variant density and sequence-dependent mutational load, to complement species conservation.

Full-length publications

1. M. Schubach, L. Nazaretyan and M. Kircher (2023). The Regulatory Mendelian Mutation score for GRCh38. GigaScience, 12, 2023, giad024. https://doi.org/10.1093/gigascience/giad024

Conference presentations

1. L. Nazaretyan, M. Schubach, and M. Kircher. The Regulatory Mendelian Mutation (ReMM) score for GRCh38. (Poster presentation), ISMB/ECCB 2021, Online, 25-30 July, 2021.
2. L. Nazaretyan, M. Schubach, and M. Kircher. The Regulatory Mendelian Mutation (ReMM) score for GRCh38. (Poster presentation), ESHG 2021, Online, August 28–31, 2021.
3. L. Nazaretyan, M. Kircher, and U. Leser. Benchmarking machine learning methods for identification of mislabeled data. (Poster presentation), ECCB 2022, Sitges, Spain, September 18-21, 2022.

Contact

Supervisors:

Bernhard Diekmann (AWI)

Boris Biskaborn (AWI)

Johann-Christoph Freytag (HU) 

The goal of this PhD  is to detect the past ecosystem‐climate relationships in Arctic lake settings by big data analytics of a polar proxy dataset. We focus on two topics: Data management and data science ‐ development of a data analytics system for palaeolimnological proxy data designed for multivariate statistics. Geoscience ‐ Past and present environmental dynamics in Arctic landscapes and their impact on polar lake ecosystems. A unique, standardized, data set of proxy data from lake sediment cores in the Eastern Arctic will be compiled using the new PALIM Database. To correlate ecosystem changes with climate changes, multivariate statistics will be performed on quality controlled biotic and abiotic proxy data. The objective of this project is to develop a state‐of‐the‐art data analytics system that allows to detect the main relationships of ecosystem dynamics and climate changes and their spatiotemporal pattern in dependence to lake attributes, i.e. thermokarst or glacial origin, landscape‐type, lake‐ecosystem‐type, lake age, and catchment‐vegetation.

Full-length publications

1. H. Grotheer, V. Meyer, T. Riedel, G. Pfalz, L. Mathieu, J. Hefter, ...,  and M. Fritz (2020). Burial and origin of permafrost‐derived carbon in the nearshore zone of the southern Canadian Beaufort Sea. Geophysical Research Letters, 47, e2019GL085897. https://doi.org/10.1029/2019GL085897
2. G. Pfalz, B. Diekmann, J-C. Freytag, and B.K. Biskaborn (2021). Harmonizing heterogeneous multi-proxy data from lake systems. Computers & Geosciences. https://doi.org/10.1016/j.cageo.2021.104791
3. B.K. Biskaborn, L. Nazarova, T. Kröger, L.A. Pestryakova, L. Syrykh, G. Pfalz, U. Herzschuh, and B. Diekmann (2021). Late quaternary climate reconstruction and lead-lag relationships of biotic and sediment-geochemical indicators at lake Bolshoe Toko, Siberia. Front. Earth Sci., 9, 703. https://doi.org/10.3389/feart.2021.737353
4. S.A. Vyse, U. Herzschuh, G. Pfalz, L.A. Pestryakova, B. Diekmann, N. Nowaczyk, and B.K. Biskaborn (2021). Sediment and carbon accumulation in a glacial lake in Chukotka (Arctic Siberia) during the late Pleistocene and Holocene: Combining hydroacoustic profiling and down-core analyses. Biogeosciences. https://doi.org/10.5194/bg-18-4791-2021
5. L. Hughes-Allen, F. Bouchard, C. Hatté, H. Meyer, L.A. Pestryakova, G. Pfalz, B. Diekmann, D.A. Subetto, and B.K. Biskaborn (2021). 14 000-year carbon accumulation dynamics in a Siberian lake reveal catchment and lake productivity changes. Front. Earth Sci., 9, 1–19. https://doi.org/10.3389/feart.2021.710257
6. G. Pfalz, B. Diekmann, J-C. Freytag, L. Sryrkh, D.A. Subetto, and B.K. Biskaborn (2022). Improving age-depth correlations by using the LANDO model ensemble. Geochronology, 4, 269–295. https://doi.org/10.5194/gchron-4-269-2022
7. G. Pfalz, B. Diekmann, J-C. Freytag, and B.K. Biskaborn (2023). Effect of temperature on carbon accumulation in northern lake systems over the past 21,000 years. Front. Earth Sci., Sec. Quaternary Science, Geomorphology and Paleoenvironment. https://doi.org/10.3389/feart.2023.1233713

Conference presentations

1. G. Pfalz, B. Diekmann, J-C. Freytag, and B.K. Biskaborn. Decipher Arctic lakes ecosystem dynamics. (Poster presentation), YES Congress, Berlin, Germany, September 9-13, 2019.
2. G. Pfalz, B. Diekmann, J-C. Freytag, and B.K. Biskaborn. Harmonizing heterogeneous multi-proxy data from Arctic lake sediment records. (PICO presentation), EGU General Assembly, Online, 19–30 April 2021, EGU21-9401. https://doi.org/10.5194/egusphere-egu21-9401
3. G. Pfalz, B. Diekmann, J-C. Freytag, L. Syrykh, D.A. Subetto, and B.K.  Biskaborn, Using LANDO as a universal wrapper for applying multiple age-depth modeling systems for sediment records from Arctic lake systems. EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022. https://doi.org/10.5194/egusphere-egu22-8743

Twitter

Contact

Supervisors:

Volker Markl (TU)

Uwe Ohler (MDC)

Developing and applying data science methods typically involves specifying and executing complex data processing and analysis pipelines, comprising pre-processing steps, model building, as well as evaluation. Heterogeneous data sources and systems for executing such pipelines can introduce complex dependencies on data or even processing architectures. When training a neural network, for example, sample transformations and preprocessing steps may be carried out with custom scripts, while the actual training may be executed on state of the art systems such as TensorFlow or MXNet, or scalable systems such as Spark and Flink. In order to simplify and automate the data analysis process, including interactive and iterative data selection or hyperparameter tuning, it is imperative to declaratively specify such pipelines and map them to potentially changing target systems and data sets. A declarative specification could enable automation and reproducibility of a data analysis process, and even help with detecting and validating properties of responsible data management, such as fairness, transparency, or the diversity. Current data analysis pipelines lack holistic declarative end-to-end specifications, preventing automatic reproducibility, comparability, re-use of previous results and models, and testing of experiments for properties of responsible data management. Training performance and prediction quality critically depend on configuration- and hyperparameters, but metadata, lineage information, and results of experiments are not systematically tracked and stored in a structured manner. Rather, these parameters are determined ad-hoc, or by using heuristics or explorative grid search for each pipeline anew. In order to overcome these deficiencies and challenges, we propose the introduction of truly declarative specifications of such pipelines and the creation of a repository of declarative descriptions of machine learning experiments and their corresponding evaluation data in an experiment database. We further plan to research and evaluate optimization and automation of the data science process, both in multi-tenant environments and the continuous deployment of machine learning pipelines.

Full-length publications

1. S. Redyuk, Z. Kaoudi, V. Markl, and S. Schelter (2021). Automating data quality validation for dynamic data ingestion. In Proceedings of the International Conference on Extending Database Technology (EDBT). ISBN 978-3-89318-084-4 on OpenProceedings.org.
2. S. Baunsgaard, M. Boehm, ..., V. Markl, ..., S. Redyuk, T. Rieger, A.R. Mahdiraji, S.B. Wrede, and S. Zeuch (2021). ExDRa: Exploratory data science on federated raw data. In Proceedings of the 2021 International Conference on Management of Data, 2450-2463.

Conference presentations

1. S. Redyuk, S. Schelter, T. Rukat, V. Markl, and F. Biessmann. Learning to Validate the Predictions of Black Box Machine Learning Models on Unseen Data. (Workshop paper and presentation), HILDA’19, Amsterdam, Netherlands, 5 July, 2019. doi.org/10.1145/3328519.3329126

2. S. Redyuk. Automated Documentation of End-to-End Experiments in Data Science. (Workshop paper and presentation), ICDE’19, Macau, China, 8-11 April, 2019. 10.1109/ICDE.2019.00243

3. H.J. Meyer, H. Grunert, T. Waizenegger, L. Woltmann, C. Hartmann, W. Lehner, M. Esmailoghli, S. Redyuk, R. Martinez, Z. Abedjan, and A. Ziehn (2019). Particulate Matter Matters - The Data Science Challenge @ BTW 2019. Datenbank-Spektrum, 19(3), pp.165-182.

4. M. Esmailoghli, S. Redyuk, R. Martinez, Z. Abedjan, T. Rabl, and V. Markl (2019). Explanation of air pollution using external data sources. BTW 2019–Workshopband.

5. S. Redyuk, V. Markl, and S. Schelter. Towards Unsupervised Data Quality Validation on Dynamic Data. (Workshop paper and presentation), ETMLP 2020, Copenhagen, Denmark, 30 March 2020. https://www.youtube.com/watch?v=Xhq8X64RA1Q

6. S. Redyuk, Z. Kaoudi, V. Markl, and S. Schelter. Automating data quality validation for Dynamic Data Ingestion.  (Oral presentation), International Conference on Extending Database Technology (EDBT), Nicosia, Cyprus, 23-26 March 2021. https://www.youtube.com/watch?v=v9IR1zjqAek

Contact

Supervisors:

Guido Grosse (AWI)

Johann-Christoph Freytag (HUB)

Temperatures in the Arctic are warming. They’re warming even more rapidly than in other regions of the World. Unfortunately, due to their remoteness, these highly sensitive environments are not fully researched and explored yet, and thus represent a significant unknown in our Earth system models and our general understanding of their importance.

One of the major components of the Arctic is permafrost, which describes any ground that stays at, or below 0 °C for at least two consecutive years. Permafrost covers around one fourth of the landmass in the northern hemisphere and can reach depths of up to 1000 meters. With the warming atmosphere however, these soils are recently also experiencing higher temperatures, and large regions are starting to thaw. In the thawing ground, bacteria are now able to decompose century-old organic carbon, effectively releasing large amounts of carbon dioxide into the atmosphere, thus amplifying the atmospheric greenhouse effect even further. Researchers estimate that this permafrost holds up to 1400 Gt of carbon dioxide, about twice the amount that is currently in our planet’s atmosphere. It is therefore crucial for us to understand the rate of permafrost degradation in order to make the most accurate predictions for our climate and estimate its significance to life on Earth.

One of multiple possibilities for monitoring permafrost regions, is via remote sensing, where we can make use of aerial imagery (i.e., from satellites, airplanes, drones) to observe and analyze features at the ground’s surface that are very specific for permafrost soils. One example of such characteristic landscapes is polygonal thermokarst. These landscapes are characterized by large polygons (approximately 10-50 m across) that, in the undegraded state, show elevated rims at their borders. With an ongoing warming, these rims will however degrade and erode to become lower troughs between these polygons. With ongoing degradation, single troughs will gradually connect and the network of channels between the polygons steadily grows. A higher connectivity of troughs thus advances the drainage of surface water for entire landscapes. By evaluating digital elevation models in combination with multispectral imagery (in the RGB and near-infrared wavelengths), we can observe this process of degradation and permafrost thaw as well as the hydrological state of the landscape from above.

So far, studies on quantifying this trough connectivity are limited to local field studies and regional hydrological models only. My research therefore focuses on the extraction of trough characteristics from high spatial resolution aerial imagery and digital elevation data, to model this network of channels as a graph (a concept for network analysis from discrete mathematics and computer sciences). Graph theory provides a multitude of metrics that allow the analysis of underlying network characteristics such as the connectivity of single channels, the intensity of the connections, but also width and depth of single troughs (which can be stored as edge weights).  The figure below shows some preliminary results of the terrain analysis of a study area in northern Alaska, USA. Further, by evaluating the graphs of channels from multiple steps in time, it is possible to analyze the rate of degradation. This can be inferred when the merging of multiple smaller graphs into fewer larger graphs occurs, when certain connections arise, but also from the increasing values of depth and width of the channels as time advances. With sufficiently available data and dense time series, even the prediction of future network development is possible.

Being able to monitor the hydrological regime of a landscape and quantifying the local thaw rate of permafrost is crucial in order to gain insight into its contribution to greenhouse gases in the atmosphere and therefore its role in climate change. On a methodological level, processing this type of information as graphs, opens up possibilities to consider even small-scale changes on a trough-by-trough level and offers significantly shorter computing times as compared to calculations based on spatial raster-imagery. This allows analysis coverage of much larger regions with available processing capacities, which is an important step towards a pan-Arctic and holistic understanding of the Earth’s permafrost.

Full-length publications

1. T. Rettelbach, M. Langer, I. Nitze, B. Jones. V. Helm, J-C. Freytag, and G. Grosse (2021). A quantitative graph-based approach to monitoring ice-wedge trough dynamics in polygonal permafrost landscapes. Remote Sens. 13, 3098. https://doi.org/10.3390/rs13163098
2. W.J. Foster, G. Ayzel, J. Münchmeyer, T. Rettelbach, N. Kitzmann, T.T. Isson, M. Mutti, and M. Aberhan (2021). Machine learning identifies ecological selectivity patterns across the end-Permian mass extinction. Paleobiology, 1-15.  https://doi.org/10.1017/pab.2022.1
3. T. Rettelbach, M. Langer, I. Nitze, B. Jones, V. Helm, J-C. Freytag, and G. Grosse (2022). From images to hydrologic networks - Understanding the Arctic landscape with graphs. In ACM Proceedings of the 34th International Conference on Scientific and Statistical Database Management (SSDBM 2022). https://doi.org/10.1145/3538712.3538740 
4. N.H. Chan, M. Langer, B. Juhls, T. Rettelbach, P. Overduin, K. Huppert and J. Braun (2023). An Arctic Delta Reduced Complexity Model and its Reproduction of Key Geomorphological Structures. Earth Surface Dynamics, 259–285. https://doi.org/10.5194/esurf-11-259-2023
5. W.J. Foster, B. J. Allen, N.H. Kitzmann, J. Münchmeyer, T. Rettelbach, J.D. Witts, , R.J. Whittle, E. Larina, M.E. Clapham and A.M.  Dunhill (2023). How predictable are mass extinction events? Royal Society Open Science, 10 (3), 221507. https://doi.org/10.1098/rsos.221507
6. B.M. Jones, S. Schaeffer Tessier, T. Tessier, M. Brubaker, M. Brook, J. Schaeffer, M.K. Ward Jones, G. Grosse, I. Nitze, T. Rettelbach, S. Zavoico, J.A. Clark, and K.D. Tape (2023). Integrating local environmental observations and remote sensing to better understand the life cycle of a thermokarst lake in Arctic Alaska. Arctic, Antarctic, and Alpine Research, 55(1).
7. T. Rettelbach, I. Nitze, I. Grünberg, J. Hammar, S. Schäffler, D. Hein, M. Gessner, T. Bucher, J. Brauchle, J. Hartmann, T. Sachs, J. Boike, and G. Grosse (2023). Aerial imagery datasets of permafrost landscapes in Alaska and northwestern Canada acquired by the Modular Aerial Camera System. PANGAEA. https://doi.pangaea.de/10.1594/PANGAEA.961577
8. T. Rettelbach, I. Nitze, I. Grünberg, J. Hammar, S. Schäffler, D. Hein, M. Gessner, T. Bucher, J. Brauchle, J. Hartmann, T. Sachs, J. Boike, and G. Grosse (2023). Super-high-resolution aerial imagery datasets of permafrost landscapes in Alaska and northwestern Canada. Earth System Science Data Discussions. 1-35. https://doi.org/10.5194/essd-2023-193 [Preprint]

Conference presentations

1. T. Rettelbach, M. Langer, I. Nitze, B. Jones. J. Boike, J-C. Freytag, and G. Grosse. Potential von Graphen für die quantitative Analyse von tauenden Eiskeilpolygonnetzwerken. (Oral presentation), 11. Treffen des AK Permafrost der DGP, Online, 11 December 2020.
2. T. Rettelbach, G. Grosse, I. Nitze, J. Brauchle, T. Bucher, M. Gessner, B.M. Jones, J. Boike, M. Langer, and J-C. Freytag. A quantitative graph-based assessment of ice-wedge trough dynamics in polygonal thermokarst landscapes of the Anaktuvuk river fire scar. (Oral presentation), AGU Fall Meeting, Online, 1-17 December 2020.
3. T. Rettelbach, M. Langer, I. Nitze, B. Jones, V. Helm, J-C. Freytag, and G. Grosse. Quantifying erosional dynamics in ice-wedge networks with computer vision and graph theory. (Oral presentation), Regional Conference on Permafrost, Online, 24-29 October 2021.
4. T. Rettelbach, M. Langer, I. Nitze, B. Jones, V. Helm, J-C. Freytag, and G. Grosse. Evaluating the effects of tundra fires on soil microtopography and hydrologic surface networks in polygonal permafrost landscapes. (Oral presentation), AGU Fall Meeting, News Orleans / Online, USA, 13-17 December 2021.
5. T. Rettelbach, I. Nitze, and G. Grosse. Polar-6 airborne expedition Perma-X West Alaska 2021. (Oral presentation), 12. Treffen des AK Permafrost der DGP, Online, 06 May 2022.
6. T. Rettelbach, I. Nitze, S. Schäffler, S. Barth, I. Grünberg, J. Hammar, M. Gessner, T. Bucher, J. Brauchle, T. Sachs, J. Boike, and G. Grosse. Super-high-resolution Earth observation datasets of North American permafrost landscapes. (Oral and poster presentation), 8th NASA ABoVE Science Team Meeting, Fairbanks, Alaska, USA, 9-12 May 2022.
7. T. Rettelbach, C. Witharana, A. Liljedahl, M. Langer, I. Nitze, J-C. Freytag, and G. Grosse. The evolution of ice-wedge polygon networks in tundra fire scars. (Poster presentation), 16th International Circumpolar Remote Sensing Symposium, Fairbanks, Alaska, USA, 16-20 May 2022.
8. T. Rettelbach, M. Langer, I. Nitze, V. Helm, J-C. Freytag, and G. Grosse. Quantifying rapid permafrost thaw with computer vision and graph theory. (Poster presentation), ESA Living Planet Symposium, Bonn, 23-27 May 2022. 
9. T. Rettelbach, M. Langer, I. Nitze, B. Jones, V. Helm, J-C. Freytag, and G. Grosse. From images to hydrologic networks - Understanding the Arctic landscape with graphs. (Oral presentation), 34th International Conference on Scientific and Statistical Database Management, Copenhagen, Denmark, 6-8 July 2022.
10. T. Rettelbach, K. Heidler, N. Lehmann, I. Nitze, M. Langer, X. Zhu, J-C. Freytag, G. Grosse, and D. Kainmüller. Cross-resolution image segmentation for mapping smallest ponds in the Arctic. (Poster presentation), AI4EO Symposium 2023, Munich, Germany, 9-10 October 2023.

Contact

Supervisors:

Janik Wolters (DLR)

Guillermo Gallego (TU)

Classical digital computer architectures are visibly approaching their technological and physical limits. Thus, there is a growing interest in developing post-digital computing approaches to overcome these limitations. Besides quantum computers, approaches that emulate neuromorphic processes represent a promising alternative because they mimic the massively parallel, energy-efficient computations carried out by the human brain. Such computations constitute the building blocks of the pattern recognition algorithms underpinning the success of machine learning (ML). Optically integrated systems promise 2–3 orders of magnitude higher energy efficiency compared to today's electronic approaches. Among others, post-digital computer concepts will enable numerous new applications for ML in places like data centers or security systems, as well as autonomous vehicles, drones and satellites – any area where massive amounts of computations need to be done but is limited by power and time. In this project we will realize ML with optical neural networks. That is, we want to use light to power machine learning, due to the potential advantages that an optical neural network (ONN) has over one that is emulated on conventional GPU chips. Moreover, we will investigate the potential of neuromorphic computing hardware for ML on low-power autonomous systems.

Full-length publications

1. L. Jaurigue, E. Robertson, J. Wolters, and K. Lüdge (2021). Reservoir computing with delayed input for fast and easy optimisation. Entropy, 23, 1560. https://doi.org/10.3390/e23121560
2. M. Yang, E. Robertson, L. Esguerra, K. Busch and J. Wolters (2023). Optical convolutional neural network with atomic nonlinearity. https://doi.org/10.48550/arXiv.2301.09994 [Preprint]
3. L. Meßner, E. Robertson, L. Esguerra, K. Lüdge and J. Wolters (2023). Multiplexed random-access optical memory in warm cesium vapor. https://doi.org/10.48550/arXiv.2301.04885 [Preprint]

Conference presentations

1. E. Robertson, L. Jaruingue, L. Messner, L. Esguerra, G. Gallego, K. Lüdge, and J. Wolters. A scheme for optical reservoir computers with atomic memory. (Poster presentation), Hot Vapor Workshop, Stuttgart / Online, 22-24 March 2021.
2. L. Esguerra, L. Meßner, E. Robertson, N.V. Ewald, M. Gündoğan, and J. Wolters (2022). Optimization and readout-noise analysis of a hot vapor EIT memory on the Cs D1 line. Quantum Physics. https://doi.org/10.48550/arXiv.2203.06151 [Preprint]
3. M. Yang, E. Robertson, L. Esguerra, K. Busch and J. Wolters (2023). Optical convolutional neural network with atomic nonlinearity. https://doi.org/10.48550/arXiv.2301.09994 [Preprint]
4. L. Meßner, E. Robertson, L. Esguerra, K. Lüdge and J. Wolters (2023). Multiplexed random-access optical memory in warm cesium vapor. https://doi.org/10.48550/arXiv.2301.04885 [Preprint]

Contact

Supervisors:

Christiane Becker (HZB)

Christof Schütte (FU)

Sven Burger (ZIB)

With the growing share of photovoltaic (PV) solar energy in the global energy generation capacity, building-integrated photovoltaics (BIPV) gains increasing importance. For BIPV, aesthetical factors, such as color, play a larger role than for large industrial PV fields. The structural color of PV modules and surface texturing for minimizing reflective losses and for self-cleaning have to be considered for accurately estimating the optical performance and, subsequently, the annual energy yield. This raises several issues which are hardly considered in state-of-the-art numerical methods, which are used for planning PV systems and for estimating their performance: Which specific module surface treatments ensure maximum energy yield? How does the module appear visually to an observer throughout the day and year? How do the spectral albedo of the surroundings, local weather conditions and local shadowing affect the result? How can the complex environments in building-integrated and bifacial solar modules be simulated as efficiently as possible?

This project aims to develop an all-encompassing optical modeling and optimization toolbox for individual PV modules considering the full optical circumstances from specifically textured module surfaces to local solar irradiance conditions. Together with urban planners and architects, desirable color effects will be identified. Metrics will be developed in order to quantify the aesthetical effect of color appearance of BIPV modules. The PhD student will model PV modules with different textured surfaces and coloring techniques, and account for shadowing and spectral reflection of the surroundings, e.g. from overgrown ground and trees.

Full-length publications

-

Conference presentations

1. J. Schaible, B. Nouri, T. Kotzab, M. Loevenich, N. Blum, A. Hammer, K. Jäger, C. Becker and S. Wilbert. Application of Nowcasting to Reduce the Impact of Irradiance Ramps on PV Power Plants. (Oral presentation), EU PVSEC, Lisbon, Portugal, 18-22 September 2023.

Contact

Supervisors:

Annika Bande (HZB)

Ulf Leser (HU)

In recent years data science more and more finds its way into the field of materials sciences, usually in the form of regression or classification approaches which are trained on known properties of certain materials and applied to predict these properties for less well characterized yet physically and chemically similar other materials. However, a pertinent problem in this field is the search for materials which are optimal with regard to some property, a problem which only partly matches to the above described scenario, since it is impossible to compute properties of the typically exponentially many candidate structures. For such a setting, the methods for calculating the properties of given structures must be complemented by intelligent algorithms which decide which structures are assessed in which order, and use the results of previous assessments to guide this search process through the space of all candidates. In this project, we research such methods and apply them to a particular important problem in the development of Solar Fuels: The search for an optimal material for photocatalysis, namely water oxidation or even water splitting. The searched material class will be sustainable grapheneoxide quantum dots. The desired properties are: (1) a band edge of more than 1.23 eV that is to be centered around the energies for the water-to-oxygen oxidation and the water-to-hydrogen reduction, (2) a good representation of its diverse spectra (X-ray absorption and emission (XAS and XES), UV/vis, and infrared), (3) sizes of the quantum dots, and (4) their oxygen and eventually dopand content. Many of these properties can be computed numerically with standard quantum chemistry software as e.g. the ORCA program, but only at the cost of extremely resource-intensive calculations which currently limits the number of materials that can be characterized. We study methods that allow a goal-driven search through the space of all candidate materials to find those that maximize a cost function derived from the materials properties. To this end, we plan to apply methods for reinforcement learning and heuristic search procedures, possibly combined with Deep Learning network to model correlations between the target properties and other, material-derived features. Particular challenges in this project are the inhomogeneity and incompleteness of data with respect to the target properties. The expected impact is high, because the project will create a complete workflow from high-throughput data production, database generation, and learning of structureproperty relationships that can as well be applied to many other interesting materials and classes of properties in the material sciences.

Full-length publications

1. K. Singh, J. Münchmeyer, L. Weber, U. Leser, and A. Bande (2022). Graph Neural Networks for Learning Molecular Excitation Spectra. J. Chem. Theory Comp., 18, 7, 4408-4417. DOI: 10.1021/acs.jctc.2c00255
2. A. Kotobi, K. Singh, D. Höche, S. Bari, R. Meißner, and A. Bande (2023). Integrating Explainability into Graph Neural Network Models for the Prediction X-ray Absorption Spectra. J. Am. Chem. Soc., 145, 22584. https://doi.org/10.1021/jacs.3c07513

Conference presentations

1. K. Singh. Machine Learning for Quantum Dynamics. (Oral presentation), Asia Pacific Conference of Theoretical and Computational Chemistry, Quy Nhon, Vietnam, 19-23 February 2023.

Contact

Supervisors:

Matthias Weidlich (HU)

Elisa Pueschel (DESY)

The most exciting recent astrophysical events have involved transient phenomena. Notable examples are the possible correlation of an astrophysical neutrino with a flaring gamma-ray source, and the observation of gamma rays and gravitational waves produced by a Kilonova. The potential very-high-energy gamma-ray emission from such events is of particular scientific interest, tracking extreme acceleration processes. However, the occurrence of such transients can often only be anticipated on a very short term and, hence, is not considered in traditional approaches to scheduling observations.
This project sets out to increase the coverage of transient and variable gamma-ray sources through dynamic scheduling of observations. To this end, time series data representing light curves of gamma-ray and further energy bands will be used for short-term forecasting of flares, e.g., based on outlier detection or recurrent neural networks. Combining these forecasts with contextual information, such as monitored weather conditions, source positions in the sky, and telescope permutations in the participating array, the observing schedule is optimised.
Any realisation of the above idea, however, has to cope with scalability and responsiveness challenges: Time series data are collected at high rates (several hundred Hertz for current telescope arrays) and some gamma-ray sources are known to be variable at minute timescales. Hence, low-latency data processing and efficient online decision making are of crucial importance for dynamic observation scheduling. This project aims at providing the respective conceptual and technological foundations, by answering the following research questions:
• What are models for online decision making that combine approaches for flare forecasting with contextual information for optimal observation scheduling? This includes questions related to the expressiveness needed for the decision mechanism as well as the temporal and spatial granularity considered in these models.
• How can the online decision making be expressed in a computational model that is based on streaming data? Here, important aspects are the required operator algebra and data correlation mechanisms. Also, the notions of state to be maintained during processing is to be clarified.
• How to optimise the latency of stream processing for online decision making? Directions to answer this question are (i) prefetching of contextual information for low-latency assessment of flare forecasts, (ii) state management for streaming operators, and (iii) approximate stream processing using data sketches.
In sum, the results of this project will be a grounding of dynamic observation scheduling in models for data stream processing along with algorithm for their efficient realisation.

Full-length publications

1. F. Schintke, N.D. Mecquenem, D. Frantz, V.E. Guarino, M. Hilbrich, F. Lehmann, R. Sattler, J.A. Sparka, D. Speckhard, H. Stolte, A.D. Vu, and U. Leser (2023). Validity constraints for data analysis workflows. CoRR. https://arxiv.org/abs/2305.08409 [Preprint]

Conference presentations

1. H. Stolte. Checking plausibility in exploratory data analysis.  (Oral presentation), 47th International Conference on Very Large Databases, Copenhagen, Denmark, 16-20 August 2021. http://ceur-ws.org/Vol-2971/paper08.pdf
2. H. Stolte, J. Sinapius, I. Sadeh, E. Pueschel, D. Berge, and M. Weidlich. Detecting VHE blazar flares with deep learning. (Oral presentation), International Conference on Machine Learning for Astrophysics - ML4Astro, Catania, Italy, 30 May - 02 June 2022.

Contact

Supervisors:

Odej Kao (TU)

Claudia Stolle (GFZ)

Over the past few decades, high-precision magnetic satellite missions have been steadily providing new insights into the Earth’s magnetic field and the processes that underlie it. Yet we do not have a full picture of the geophysical mechanisms by which the geomagnetic field is created. Electric currents flowing along polar geomagnetic field lines are an important mechanism by which energy is transferred between the magnetosphere at several Earth radii and the ionosphere, the ionised part of the upper atmosphere, which lies at an altitude of 100-300 km altitude. These field-aligned currents (FAC) cannot be detected on ground, but they produce significant signatures in observations of the magnetic field made by Low-Earth-Orbiting satellites, sometimes even becoming visible as auroral lights. These currents are highly fluctuating and high-precision magnetic field missions such as CHAMP and Swarm have been used to characterize them.

An international consortium has started an initiative to make us of data from other sources. A number of satellites carry magnetometers that were not originally designed for scientific applications, but rather for the purposes of navigation – they include Cryosat, GOCE, GRACE-FO, and many others. A careful calibration of data from these magnetometers permits extractions of the signature of currents from the Earth's magnetic field. Combining such data from several satellites will provide an unprecedented level of global coverage and should strongly enhance our ability to particularly monitor the polar ionosphere and its interaction with the magnetosphere. This data and a higher level of coverage can play an important role in understanding short-lived magnetic storms, which have a large impact on a variety of domains.

To achieve this, we can draw on a global dataset that has been collected continuously from diverse satellite missions since the year 2000. Combining all of this data into a common mapping procedure is challenging: it comprises different sampling rates, signal amplitudes, noise levels, and latency. Combining data of different sampling rate, signal amplitude, noise level, and latency in one mapping procedure requires special care for data handling and inter-calibration to achieve an unbiased result uniformly valid over the globe. The amplitude of the signal detected is subject to variation and noise due to descending satellite orbits, the architectural settings of missions and naturally varying levels of solar flux. This means that particular attention must be paid to data handling and inter-calibration to achieve an unbiased result that is uniformly valid across the globe. Our initial results, based on data collected from for the GRACE-FO1 satellite, show that it is indeed possible to detect FACs by calibrating platform magnetometer data, as shown in the attached figures.

Full-length publications

1. K. Styp-Rekowski, C. Stolle, I. Michaelis, and O. Kao (2021). Calibration of the GRACE-FO satellite platform magnetometers and co-estimation of intrinsic time shift in data. IEEE International Conference on Big Data, 5283-5290. https://doi.org/10.1109/BigData52589.2021.9671977

2. C. Stolle, I. Michaelis, C. Xiong, M. Rother, T. Usbeck, Y. Yamazaki, J.  Rauberg, and K. Styp-Rekowski (2021). Observing Earth’s magnetic environment with the GRACE-FO mission. Earth, Planets and Space, 73, 51. https://doi.org/10.1186/s40623-021-01364-w

3. I. Michaelis, K. Styp-Rekowski, J. Rauberg, C. Stolle, and M. Korte (2022). Geomagnetic data from the GOCE satellite mission. Earth, Planets, and Space, 74, 135. https://doi.org/10.1186/s40623-022-01691-6

4. K. Styp-Rekowski, I. Michaelis, C. Stolle, J. Baerenzung, M. Korte, and O. Kao (2022). Machine Learning-based Calibration of the GOCE Satellite Platform Magnetometers. Earth, Planets, and Space, 74, 138. https://doi.org/10.1186/s40623-022-01695-2

Conference presentations

1. K. Styp-Rekowski, C. Stolle, O. Kao, and I. Michaelis. Satellite Platform Magnetometer Calibration Using Machine Learning. (Oral Presentation), Joint Scientific Assembly IAGA-IASPEI, Online, 21-27 August 2021.
2. K. Styp-Rekowski, C. Stolle, O. Kao, and I. Michaelis. Automatic Calibration of Satellite Platform Magnetometers with Neural Network-based Time Shift Approximation. (Oral Presentation), Joint Scientific Assembly IAGA-IASPEI, Online, 21-27 August 2021.
3. K. Styp-Rekowski, C. Stolle, I. Michaelis, and O. Kao. Machine Learning-based Information Extraction from Non-dedicated Sensors. (Oral Presentation), Photonics Days Berlin Brandenburg, Berlin, Germany, 4-7 October 2021.
4. K. Styp-Rekowski, C. Stolle, I. Michaelis, and O. Kao. Calibration of GRACE-FO and GOCE Platform Magnetometers Using Machine Learning. (Oral Presentation), Swarm Data Quality Workshop, Athens, Greece, 11-16 October 2021.
5. K. Styp-Rekowski, C. Stolle, I. Michaelis, and O. Kao. Calibration of the GRACE-FO Satellite Platform Magnetometers and Co-Estimation of Intrinsic Time Shift in Data. (Oral Presentation), IEEE Big Data 2021, Online, 15-18 December 2021.
6. K. Styp-Rekowski, I. Michaelis, C. Stolle, and O. Kao. Magnetic Datasets from Non-dedicated Satellites. (Oral Presentation), Living Planet Symposium, Bonn, Germany, 23-27 May 2022.
7. K. Styp-Rekowski, I. Michaelis, C. Stolle, and O. Kao. Physics-informed Neural Network for Platform Magnetometer Calibration. (Oral Presentation), Swarm Data Quality Workshop, Uppsala, Sweden, 10-14 October 2022.

Contact

Supervisors:

Chistian Meske (FU)

Rutger Schlatmann (HZB)

Today, because of the complexity of underlying machine learning models, AI appears as a ‘black box’, since the internal learning and optimization processes are often not completely comprehensible. To tackle the trade-off problem, methods of “Explainable Artificial Intelligence” (XAI) were developed to increase the transparency of underlying models without decreasing their performance. In this context, the PhD project aims to apply XAI methods for machine learning models in two use cases in the field of energy, esp. photovoltaics, to increase the models’ explainability and hence trustworthiness.

The first use case is linked to the existing HEIBRiDS project “Optimization of solar energy yield..” The control strategies in this project incorporate predictive and prescriptive data analytics based on machine learning approaches, which however represent black boxes. The XAI research proposed here makes it possible both to improve the trust and acceptability of AI-based solutions, and to generate findings that sustainably improve product design or system configurations. The second use case relates to the “Combinatorial materials discovery” pursued in the HZB research groups Unold/Schorr and Abdi/van de Krol, which focuses on the exploration of light-absorbing semiconductor sand catalysts for solar energy conversion devices by combinatorial high throughput methods. This generates high-dimensional sets of data, which have to be searched and analyzed for structure-property-function relationships and from which guidance for further experiments is sought. For this purpose, it is aimed to use machine learning approaches to automate time-consuming procedures regarding the analysis of the multidimensional datasets. The introduction of machine learning in combination with XAI methods is hence expected to accelerate development processes, to provide new physical understandings and eventually support the efficient development of materials for solar conversion devices.

The results of the PhD project will be synthesized to contribute knowledge regarding a) the validity and reliability of XAI methods in the field of photovoltaic, b) new opportunities for data pre-processing and data analytics based on XAI outcomes, c) consequent effects on the development of (and trust towards) energy systems, and d) the findings’ transferability to other fields in the energy sector.

Full-length publications

1. C. Utama, C. Meske, J. Schneider, and C. Ulbrich (2022). Reactive power control in photovoltaic systems through (explainable) artificial intelligence. Applied Energy, 328. https://doi.org/10.1016/j.apenergy.2022.120004
2. C. Utama, B. Karg, C. Meske, and S. Lucia (2022). Explainable artificial intelligence for deep learning-based model predictive controllers. In Proceedings of the 26th International Conference on System Theory, Control and Computing (ICSTCC), 464-471. https://doi.org/10.1109/ICSTCC55426.2022.9931794
3. C. Utama, C. Meske, J. Schneider, R. Schlatmann, and C. Ulbrich (2023). Explainable artificial intelligence for photovoltaic fault detection: A comparison of instruments. Solar Energy, 249, 139–151. https://doi.org/10.1016/j.solener.2022.11.018

Conference presentations

1. C. Utama, C. Meske, J. Schneider, and C. Ulbrich. Reactive power control in photovoltaic systems through (explainable) artificial intelligence. (Poster presentation), 8th World Conference on Photovoltaic Energy Conversion (WCPEC-8), Milan, Italy, 26-30 September 2022.
2. C. Utama, B. Karg, C. Meske, and S. Lucia. Explainable artificial intelligence for deep learning-based model predictive controllers. (Oral presentation), 26th International Conference on System Theory, Control and Computing (ICSTCC), Online, 19-21 October 2022.

Contact

Supervisors:

Ulrike Herzschuh (AWI)

Begüm Demir (TU)

The Polar Terrestrial Environmental Systems Research Group at the Alfred-Wegener-Institut has multiple field trips a year that, among other goals, aim to collect data to monitor the landcover and vegetation dynamics in the Arctic region. Vegetation dynamics can provide an insight into the effects of global warming on the environment. The aim of the current project is to employ Machine Learning and Deep learning methods to analyse this data to gain better insights into the dynamics of the vegetation species and how these are changing over time. In order to accomplish this goal, various types of Remote Sensing data is used such as Sentinel-2, Landsat 7/8 as well as drone data collected in the field. The ultimate goal is to develop a fusion method that can utilise the available data to create a comprehensive overview of vegetation dynamics of the past, present and future.  

Full-length publications

1. F. van Geffen, B. Heim, F. Brieger, ..., U. Herzschuh, and S. Kruse. SiDroForest: A comprehensive forest inventory of Siberian boreal forest investigations including drone-based point clouds, individually labelled trees, synthetically generated tree crowns and Sentinel-2 labelled image patches. Earth Syst. Sci. Data, 14, 4967–4994, 2022. https://doi.org/10.5194/essd-14-4967-2022

Conference presentations

1. F. van Geffen, B. Heim, U. Herzschuh, L. Pestryakova, E. Zakharov, R. Hänsch, B. Demir, B. Kleinschmit, M.Förster, and S. Kruse. SiDro Forest: Siberian drone-mapped forest inventory. (Oral presentation), Arctic Science Summit Week, Lisbon, Portugal / Online,19-26 March 2021.
2. F. van Geffen, B. Heim, U. Herzschuh, L. Pestryakova, E. Zakharov, R. Hänsch, B. Demir, B. Kleinschmit, M.Förster, and S. Kruse. SiDro Forest: Siberian drone-mapped forest inventory. (PICO presentation), EGU General Assembly, Online, 19-30 April 2021. https://doi.org/10.5194/egusphere-egu21-15106 

Contact

Supervisors:

Andrea Cominola (TU)

Heidi Kreibich (GFZ)

Flooding events in urban and non-urban settings represent a major cause of insured losses, with costs of several billion US$/year globally (e.g., US$ 60 billion in 2016). Both people and infrastructural assets are vulnerable to flood events, which are becoming increasingly frequent and severe as a consequence of climate change, extreme rainfall events, and the increasing number of people living in flood-prone areas. Several quantitative approaches for flood risk assessment exist in the literature. They rely on statistical methods and hydrological models to quantify the expected risk as a function of hazard (i.e., flood extent depth), exposure (of people/assets), and vulnerability (damage). Yet, there is limited integration of dynamic human behaviors in such methods. Human behavior dynamics play a key role in affecting the impact and recovery time of floods in coupled human-environment systems. The perception of risk, as well as adaptive behaviors for prevention, preparation, response, and recovery during flood events (e.g., accessing weather warnings, donating money for recovery) depend on several individual and collective socio-psychographic determinants. One of the key challenges in quantitative risk assessment at present is how to integrate information on dynamic human behaviors in risk assessment models. These should then inform precautionary and emergency measures and risk management approaches which mitigate flood risk.
This project addresses the above challenge as articulated in these three specific questions: (i) which data and approaches can be utilized to better understand and model relevant human behaviors before, during, and after flood events? (ii) how can relevant human behaviors be learned from the above data and integrated in dynamic flood risk assessments to support decision-making in risk management and climate adaptation? (iii) which environmental and economic benefits can be achieved by embedding human behaviors in flood risk models?

Full-length publications

1. N. Veigel, H. Kreibich, and A. Cominola (2022). A gradient boosting approach to identify behavioral and policy determinants of flood resilience in the continental US. IFAC-PapersOnLine, 55, 33, 85–91. https://doi.org/10.1016/j.ifacol.2022.11.014
2. N. Veigel, H. Kreibich, and A. Cominola (2023). Interpretable machine learning reveals potential to overcome reactive flood adaptation in the continental US. Earth's Future, 11, e2023EF003571. https://doi.org/10.1029/2023EF003571

Conference presentations

1. N. Veigel, H. Kreibich, and A. Cominola. Mining flood insurance big data to reveal the determinants of humans' flood resilience. (Oral presentation), EGU General Assembly, Online, 19–30 Apr 2021. https://doi.org/10.5194/egusphere-egu21-3042
2. N. Veigel, H. Kreibich, and A. Cominola. Mining flood insurance big data to incorporate behavioural and social aspects in flood risk modelling (Poster presentation), AGU Fall Meeting, Online & New Orleans, USA, 13–17 Dec 2021, SY55D-0390.
3. N. Veigel, H. Kreibich, and A. Cominola. Exploring Behavioral Determinants of Flood Insurance Adoption with Explainable Machine Learning in the Continental US. EGU General Assembly, Vienna, Austria & Online, 23-27 May 2022. https://doi.org/10.5194/egusphere-egu22-5839

Contact

Supervisors:

Dagmar Kainmueller (MDC)

Andreas Hocke (Charité)

Marina Höhne (Uni Potsdam)

The field of digital pathology is growing and people have higher and higher expectations of artificial intelligence to realize the automatic diagnosis. Before achieving automatic medicine diagnosis or other specific clinic tasks, realizing medical images semantic instance segmentation, which is pixel-wise recognition of objects and their classification into meaningful categories, is a key step. And semantic instance segmentation is also a classical task in the computer vision area. The great success of semantic instance segmentation by applying neural networks has shown on nature images with the help of a large amount of annotated dataset. While this can be achieved for microscope images, there are problems: new types of samples by means of microscopy are continually being entered into medical image datasets and getting annotated samples of each type for training neural networks would not be feasible. Secondly, diverse microscopy modality including upcoming techniques is being developed which makes this task even more complex. To date, there are no automated methods that are able to perform desired image analyses at the necessary level of accuracy without extensive manual input. This particularly holds for microscopy data of cells in heterogeneous tissue, where the cost for accurate outlining of cell boundaries, be it as part of a manual analysis or as part of generating training data for deep learning methods, restricts the feasibility of high-content studies.

In this project, we aim at overcoming this restriction by leveraging “sparse” annotations for training deep neural networks for semantic instance segmentation, which amounts to the weak supervision. We will develop a model for learning pixel-accurate instance segmentation purely from center point annotations, which is an unsolved problem for clusters of densely packed objects, like cells in tissue. Beyond leveraging center point annotations, we will investigate alternative sparse annotations, like image-level labels, in terms of their potential to be generated by crowd workers. In the extreme case, we want to explore the possibility of solving the problem in an unsupervised manner. We would like to investigate on a deep learning method applied to medical images doing semantic instance segmentation task without any labels and its limit.

Full-length publications

1. J.L. Rumberger, X. Yu, P. Hirsch, M. Dohmen, V.E. Guarino, A. Mokarian, L. Mais, J. Funke, and D. Kainmueller (2021). How shift equivariance impacts metric learning for instance segmentation.  Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).

Conference presentations

-