“…From the pruning algorithm aspect, heuristic-based pruning was first proposed in [23] and gets improvements with more sophisticated designed heuristics [19,27,36,49,74,87]. Regularizationbased pruning [21,26,39,41,43,55,56,62,69,76,77,81], on the other hand, are more mathematicsoriented. Recent works [39,51,62,81,82] achieve substantial weight reduction without hurting the accuracy by leveraging Alternating Direction Methods of Multipliers (ADMM) with dynamic regularization penalties, but these methods require the manual setting of the compression rate for each layer.…”