Sparse Estimation for High-Dimensional Statistical Problems

Abstract

This dissertation focuses on the sparse estimation problems in high-dimensional statistics. We first study the recovery of a near low-rank matrix coefficient with generalized trace regression, which extends the notion of sparsity for a vector coefficient. This model accommodates various types of responses and embraces many important problem setups such as reduced-rank regression, matrix regression that accommodates a panel of regressors, matrix completion, among others. Next, we adopt the diffusion approximation techniques to study sparse online regression. We propose a two-step algorithm for sparse online regression: a burn-in step using offline learning and a refinement step using a variant of truncated stochastic gradient descent. Under appropriate assumptions, we show the proposed algorithm produces near-optimal sparse estimators. We also apply the spirit of the two-stage algorithm to the sparse signal recovery problem in high-dimensional Cox’s model. By taking the statistical model structures into account, we identify local convexity near the global optima, motivated by which we propose to use two convex programs to optimize the nonconvex penalized Cox’s proportional hazard regression. This approach is named as TLAMM and it could be easily extended to other common models that share similar geometry around the true signal. Finally, we work on a case study with a large housing mortgage dataset. We use a parametric local smoothing method to study the dataset with time-varying Cox’s model.