Optimal transport meets information science: from measure concentration, to information theory, to machine learning

Abstract

Optimal transport (OT) studies how to transport one distribution to another one in the most cost-effective way. It has many known connections with, and applications to areas such as economics, geometry, quantum mechanics, etc and has received renewed interest more recently due to its increasingly many applications in imaging sciences, computer vision, and statistical learning. In this thesis, we study an information constrained variation of optimal transport, and explore its interplay with three particular areas in information science, namely concentration of measure, information theory, and machine learning. ☐ We first investigate the relationship between OT inequalities and the measure concentration results in Gaussian space and on the sphere. Our study yields strengthening and generalization of Talagrand's celebrated transportation cost inequality. Following Marton's approach, we show the new transportation inequality can be used to recover old and new concentration of measure results. We then provide an application of the new transportation inequality in information theory. We show that it can be used to recover a recent solution to a long-standing open problem posed by Cover in 1987 regarding the capacity of the relay channel. Finally, we discuss the recent applications of OT and information constrained OT in machine learning, particularly in generative models such as Generative Adversarial Networks (GANs).