3D structure information of biological entities has a strong impact on drug screening and clinical experiments. Microscopy serves as a reliable tool for imaging the 3D structures - both electron microscopy (EM) and light microscopy (LM). On a nanoscale in EM, the Cryo-ET is advancing as a method to determine the biological structure within the entities’ native environment. However, higher time consumption and constraints on electron dose limit the potential of Cryo-ET. On a macro scale in LM, confocal fluorescence microscopy (CFM) obtains axial optical sections by filtering out out-focus light using a pinhole or a slit in the optical path of the microscope. This allows stacking the thin slices into a 3D volume. Yet, CFM comes with drawbacks such as high equipment costs and higher skill requirements in microscopy. In contrast, widefield microscopy is simple and ubiquitous in biomedical laboratories.

Machine learning (ML) serves as a promising end-to-end solution. For 3D model reconstruction in CV, the ML solutions show advantages by restoring 3D information based on limited 2D input images (e.g., single-images and multi-images). In the biology domain, scholars proposed the potential to enhance 3D microscopy performance with ML technology at an early stage. However, since then only few contributions have been made to 3D biological model reconstruction with newly composed ML theories (e.g., GAN, VAE, etc.).

In this work, we will focus on the 3D reconstructions from the focal series of LM and EM using deep neural networks (DNNs). Specifically in Cryo-EM, through electron-optical defocusing we could obtain 3D information of given molecules on the 2D focal planes. We hypothesize it is possible to restore 3D information of pleomorphic objects from 2D images. For LM, instead of the expensive and skills-taxing CFM, we will adopt the images from cheap widefield microscopes. By filtering out the out-focus pixels of images in focal planes through DNNs, We will explore the possibilities to recover 3D information from out-of-focus planes of non-confocal microscopic 3D stacks.