An End-to-end Framework For Unified Compression And Learning To Optimize The Minimax Problem
Abstract
Deep neural networks (DNNs) are resource-intensive and call for efficient compression methods to reduce the resource cost. For a composite DNN with various modules, the optimal resource allocation among these modules remains a question. To address this issue, we propose a novel unified compression framework that compress the whole network in an end-to-end way, without any multi-stage heuristics nor expensive hyper-parameters tuning. We demonstrate the generality of this framework by showing its superior performance in compressing a recommendation system and a vision transformer. Furthermore, we discuss to what extent the optimizers learnt by learning to optimize(L2O) technique can be adapted to a special class of functions and outperform general-purpose optimizers for the minimax objective.
Citation
Shen, Jiayi (2022). An End-to-end Framework For Unified Compression And Learning To Optimize The Minimax Problem. Doctoral dissertation, Texas A&M University. Available electronically from https : / /hdl .handle .net /1969 .1 /198649.