Deep learning facilitated filament extraction towards quantitative analysis of filamentous structures
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Filamentous objects are ubiquitous in biological images, and segmenting filament instances is fundamental for quantitative biological research. Deep learning-based methods have shown remarkable performance in recent years on various instance segmentation tasks. However, existing approaches are not applicable for filament extraction due to the unique properties of filaments. Unlike objects with well-defined boundaries and centers, filaments are extremely thin, non-rigid, varying in shape, sharing similar local patterns, and often spanning widely over the image with numerous crossovers. These properties significantly impede the ability of current deep learning approaches to learn their shape and disentangle individual filaments. ☐ In this dissertation, I aim to develop deep learning facilitated methods for filament extraction. I first introduce a U-shape-based neural network for filament segmentation. The proposed method outperforms existing segmentation methods on actin filament and microtubule datasets. I then propose a framework combining a keypoint detection neural network and a fast-marching algorithm to extract and quantify the segments of filament networks. My proposed method does not skew the original layout of filamentous structures and achieves higher accuracy than traditional methods. ☐ Furthermore, I have developed three deep learning facilitated methods to extract complete filament instances for different use cases. I first propose an orientation-aware neural network to disentangle filaments by sorting them into six different layers by their orientation angles. A terminus pairing algorithm is also proposed for post-process and forming complete instances. The proposed method works best for extracting long and thin filaments. Then I propose a novel approach for filament extraction by transforming the instance segmentation problem into a sequence modeling problem. The proposed method segments each instance by tracing them from their tips with a sequential encoder-decoder framework, simulating the process of humans extracting filaments and achieving the best performance on thicker filaments. The last approach I introduce is filament disentangler, a bottom-up method for filament extraction. The proposed method dissects the filament network into segments with predicted junctions and outlines. Dissected segments are then grouped into full filament instances utilizing novel connection-aware embeddings and orientation-aware threads. The proposed methods achieve the lowest processing time and competitive performance. All methods are evaluated on synthetic filament datasets and various filament datasets, including microtubule, Planktothrix rubescens (P. rubescens), and Caenorhabditis elegans (C. elegans). ☐ This dissertation also emphasizes alleviating the data shortage problem. I implemented methods to generate synthetic filaments for training and evaluation. I also collected several real filament datasets, including a microtubule and actin filament dataset with semantic labels and a microtubule dataset with instance labels. In the experiments, I show that the proposed filament extraction methods can be trained on synthetic datasets and evaluated on real filaments, reducing the annotation workload and making it feasible to use deep learning with less manual annotated data.
Description
Keywords
Deep learning, Filament extraction, Image segmentation, Instance segmentation, Microscopic images, Microtubules