Estimation and Inference in Modern Nonparametric Statistics

Abstract

Nonparametric methods are central to modern statistics, enabling data analysiswith minimal assumptions in a wide range of scenarios. While contemporary procedures such as random forests and kernel methods are popular due to their performance and flexibility, their statistical properties are often less well understood. The availability of sound inferential techniques is vital in the sciences, allowing researchers to quantify uncertainty in their models. We develop methodology for robust and practical statistical estimation and inference in some modern nonparametric settings involving complex estimators and nontraditional data. We begin in the regression setting by studying the Mondrian random forest, avariant in which the partitions are drawn from a Mondrian process. We present a comprehensive analysis of the statistical properties of Mondrian random forests, including a central limit theorem for the estimated regression function and a characterization of the bias. We show how to conduct feasible and valid nonparametric inference by constructing confidence intervals, and further provide a debiasing procedure that enables minimax-optimal estimation rates for smooth function classes in arbitrary dimension. Next, we turn our attention to nonparametric kernel density estimation withdependent dyadic network data. We present results for minimax-optimal estimation, including a novel lower bound for the dyadic uniform convergence rate, and develop methodology for uniform inference via confidence bands and counterfactual analysis. Our methods are based on strong approximations and are designed to be adaptive to potential dyadic degeneracy. We give empirical results with simulated and real-world economic trade data. Finally, we develop some new probabilistic results with applications tononparametric statistics. Coupling has become a popular approach for distributional analysis in recent years, and Yurinskii's method stands out for its wide applicability and explicit formulation. We present a generalization of Yurinskii's coupling, treating approximate martingale data under weaker conditions than previously imposed. We allow for Gaussian mixture coupling distributions, and a third-order method permits faster rates in certain situations. We showcase our results with applications to factor models and martingale empirical processes, as well as nonparametric partitioning-based and local polynomial regression procedures.