Celebratio Mathematica

W. Zhu, Q. Qiu, J. Huang, R. Calderbank, G. Sapiro, and I. Daubechies: “LDMNet: Low dimensional manifold regularized neural networks,” pp. 2743–2751 in 2018 IEEE/CVF conference on computer vision and pattern recognition (Salt Lake City, UT, 18–23 June 2018). IEEE (Piscataway, NJ), 2018. ArXiv 1711.06246 incollection

Deep neural networks have proved very successful on archetypal tasks for which large training sets are available, but when the training data are scarce, their performance suffers from overfitting. Many existing methods of reducing overfitting are data-independent. Data-dependent regularizations are mostly motivated by the observation that data of interest lie close to a manifold, which is typically hard to parametrize explicitly. These methods usually only focus on the geometry of the input data, and do not necessarily encourage the networks to produce geometrically meaningful features. To resolve this, we propose the Low-Dimensional-Manifold-regularized neural Network (LDMNet), which incorporates a feature regularization method that focuses on the geometry of both the input data and the output features. In LDMNet, we regularize the network by encouraging the combination of the input data and the output features to sample a collection of low dimensional manifolds, which are searched efficiently without explicit parametrization. To achieve this, we directly use the manifold dimension as a regularization term in a variational functional. The resulting Euler–Lagrange equation is a Laplace–Beltrami equation over a point cloud, which is solved by the point integral method without increasing the computational complexity. In the experiments, we show that LDMNet significantly outperforms widely-used regularizers. Moreover, LDMNet can extract common features of an object imaged via different modalities, which is very useful in real-world applications such as cross-spectral face recognition.

Guillermo R. Sapiro	Related
A. Robert Calderbank	Related
Qiang Qiu	Related
Wei Zhu	Related
Jiaji Huang	Related

W. Zhu, Q. Qiu, B. Wang, J. Lu, G. Sapiro, and I. Daubechies: “Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning,” Siam J. Math. Data Sci. 1 : 3 (2019), pp. 479–496. MR 3996914 ArXiv 1805.07291 article

Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed. We find out that the reason for this is the inconsistency between the enforced geometry and the standard softmax cross entropy loss. To resolve this, we propose a new framework for data-dependent DNN regularization, the Geometrically-Regularized-Self-Validating neural Networks (GRSVNet). During training, the geometry enforced on one batch of features is simultaneously validated on a separate batch using a validation loss consistent with the geometry. We study a particular case of GRSVNet, the Orthogonal-Low-rank Embedding (OLE)-GRSVNet, which is capable of producing highly discriminative features residing in orthogonal low-rank subspaces. Numerical experiments show that OLE-GRSVNet outperforms DNNs with conventional regularization when trained on real data. More importantly, unlike conventional DNNs, OLE-GRSVNet refuses to memorize random data or random labels, suggesting it only learns intrinsic patterns by reducing the memorizing capacity of the baseline DNN.

Guillermo R. Sapiro	Related
Jianfeng Lu	Related
Qiang Qiu	Related
Wei Zhu	Related
Bao Wang	Related

W. Zhu and I. Daubechies: Constructing curvelet-like bases and low-redundancy frames. Preprint, October 2019. ArXiv 1910.06418 techreport

We provide a detailed analysis of the obstruction (studied first by S. Durand and later by R. Yin and one of us) in the construction of multidirectional wavelet orthonormal bases corresponding to any admissible frequency partition in the framework of subband filtering with non-uniform subsampling. To contextualize our analysis, we build, in particular, multidirectional alias-free hexagonal wavelet bases and low-redundancy frames with optimal spatial decay. In addition, we show that a 2D cutting lemma can be used to subdivide the obtained wavelet systems in higher frequency rings so as to generate bases or frames that satisfy the “parabolic scaling” law enjoyed by curvelets and shearlets. Numerical experiments on high bit-rate image compression are conducted to illustrate the potential of the proposed systems.

year	title	people

			clear

Ingrid Daubechies

Complete Bibliography

Works connected to Wei Zhu

Filter the Bibliography List