Seminar by Vicky Kouni: "Model-based (unfolding) networks and where to find them: the saga"

Vicky Kouni, Postdoctoral Fellow at the Isaac Newton Institute for Mathematical Sciences of Cambridge University

"Model-based (unfolding) networks and where to find them: the saga"

Abstract

In recent years, a new class of deep neural networks – dubbed deep unfolding networks (DUNs) – has emerged, finding its roots at model-based iterative algorithms typically solving inverse problems. The term is coined due to DUNs’ formulation: the iterations of optimization algorithms are “unfolded” as layers of neural networks, which solve the problem at hand. In this talk, we restrict ourselves to the compressed sensing (CS) problem, which pertains to reconstructing data from incomplete observations, and highlight DUNs as state-of-the-art CS-solvers. We present recent trends regarding DUNs’ broader family and dive into their theory, which mainly revolves around their generalization performance; the latter is important, because it informs about a network’s behavior on examples it has never been trained before. Particularly, we show that highly overparameterized DUNs exhibit remarkable reconstruction and generalization performance, which we further improve, by employing continuation. The latter is an optimization technique, typically used to increase the performance of model-based solvers. Empirically, we observe that continuation smooths out the DUN loss landscape. Our overall theoretical and empirical findings highlight the dependence of the generalization performance of overparameterized DUNs on their architectural properties. Our analysis sets a solid mathematical ground for developing more stable, robust, and efficient DUNs, boosting their real-world performance.

This seminar will be held in English.