Topology-informed robust optimization for out-of-distribution generalization
Date
2025
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Machine learning models have demonstrated remarkable success across numerous domains, but their performance often deteriorates significantly when deployed on data distributions that differ from those encountered during training. This dissertation addresses the critical challenge of out-of-distribution (OoD) generalization through four complementary approaches. First, we introduce Topology-informed Robust Optimization (TRO) for single expert models, a method that leverages the topological structure of data distributions to construct realistic uncertainty sets and improve OoD resilience. Second, we extend this approach to mixture-of-experts models by developing a novel ensemble pruning technique that adaptively selects complementary models based on a learned ensemble topology graph. Third, for the challenging scenario of single domain generalization, we propose an adversarial domain augmentation framework that relaxes worst-case constraints to generate diverse "fictitious" domains while maintaining semantic consistency. Finally, we address the crucial need for uncertainty quantification in OoD scenarios by developing a Bayesian meta-learning approach that applies uncertainty-guided perturbations to both feature and label spaces. Through extensive theoretical analysis and empirical validation across diverse tasks including classification, regression, semantic segmentation, and speech recognition, we demonstrate that incorporating structural information about distribution relationships substantially enhances generalization to unseen distributions. This research contributes novel optimization methods, theoretical insights, and practical techniques to improve the robustness and reliability of machine learning models deployed in real-world environments where distribution shifts are inevitable.
Description
Keywords
Machine learning, Semantic segmentation, Fictitious, Structural information