The field of digital pathology is growing and people have higher and higher expectations of artificial intelligence to realize the automatic diagnosis. Before achieving automatic medicine diagnosis or other specific clinic tasks, realizing medical images semantic instance segmentation, which is pixel-wise recognition of objects and their classification into meaningful categories, is a key step. And semantic instance segmentation is also a classical task in the computer vision area. The great success of semantic instance segmentation by applying neural networks has shown on nature images with the help of a large amount of annotated dataset. While this can be achieved for microscope images, there are problems: new types of samples by means of microscopy are continually being entered into medical image datasets and getting annotated samples of each type for training neural networks would not be feasible. Secondly, diverse microscopy modality including upcoming techniques is being developed which makes this task even more complex. To date, there are no automated methods that are able to perform desired image analyses at the necessary level of accuracy without extensive manual input. This particularly holds for microscopy data of cells in heterogeneous tissue, where the cost for accurate outlining of cell boundaries, be it as part of a manual analysis or as part of generating training data for deep learning methods, restricts the feasibility of high-content studies.

In this project, we aim at overcoming this restriction by leveraging “sparse” annotations for training deep neural networks for semantic instance segmentation, which amounts to the weak supervision. We will develop a model for learning pixel-accurate instance segmentation purely from center point annotations, which is an unsolved problem for clusters of densely packed objects, like cells in tissue. Beyond leveraging center point annotations, we will investigate alternative sparse annotations, like image-level labels, in terms of their potential to be generated by crowd workers. In the extreme case, we want to explore the possibility of solving the problem in an unsupervised manner. We would like to investigate on a deep learning method applied to medical images doing semantic instance segmentation task without any labels and its limit.