Asymptotic Normality of the Nadaraya-Watson Estimator

Assume the univariate random design model and some regularity conditions, discussing the asymptotic normality of the Nadaraya-Watson estimator ˆmh(x)

The Nadaraya-Watson estimator is a popular nonparametric method for estimating the conditional expectation of a random variable. In the context of a univariate random design model, the Nadaraya-Watson estimator for the conditional mean ( m(x) = E[Y | X = x] ) is defined as:

[ \hat{m}h(x) = \frac{\sum{i=1}^n Y_i K\left(\frac{X_i - x}{h}\right)}{\sum_{i=1}^n K\left(\frac{X_i - x}{h}\right)} ]

where:

• ( K(\cdot) ) is a kernel function (typically a symmetric and non-negative function that integrates to one),
• ( h ) is the bandwidth parameter,
• ( (X_i, Y_i) ) are the observed data points.

Asymptotic Normality

To discuss the asymptotic normality of the Nadaraya-Watson estimator ( \hat{m}_h(x) ), we need to consider the following regularity conditions:

1. Kernel Function: The kernel ( K(\cdot) ) is a bounded, symmetric function with ( \int K(u) du = 1 ) and ( \int u K(u) du = 0 ). Additionally, ( K(u) ) should have a finite second moment.

2. Bandwidth Condition: The bandwidth ( h ) should satisfy ( h \to 0 ) as ( n \to \infty ) and ( nh \to \infty ). This ensures that the estimator becomes more localized around the point ( x ) as the sample size increases.

3. Smoothness of the True Function: The true regression function ( m(x) ) should be sufficiently smooth (e.g., Lipschitz continuous) in a neighborhood of ( x ).

4. Independence and Identically Distributed (i.i.d.) Samples: The observations ( (X_i, Y_i) ) are assumed to be i.i.d. from some joint distribution.

Asymptotic Distribution

Under these conditions, the asymptotic distribution of the Nadaraya-Watson estimator can be derived. Specifically, as ( n \to \infty ), the estimator ( \hat{m}_h(x) ) converges in distribution to a normal distribution:

[ \sqrt{nh} \left( \hat{m}_h(x) - m(x) \right) \xrightarrow{d} N(0, \sigma^2(x)) ]

where ( \sigma^2(x) ) is the asymptotic variance of the estimator, which can be expressed as:

[ \sigma^2(x) = \frac{1}{\left( \int K(u) du \right)^2} \cdot \left( \int K^2(u) du \right) \cdot m'(x)^2 + \frac{1}{\int K(u) du} \cdot \text{Var}(Y | X = x) ]

Interpretation

• The term ( \sqrt{nh} ) indicates that the estimator's convergence rate is influenced by both the sample size ( n ) and the bandwidth ( h ).
• The variance ( \sigma^2(x) ) captures the variability of the estimator around the true mean function ( m(x) ) and depends on the choice of the kernel and the underlying distribution of the data.

Conclusion

In summary, under appropriate regularity conditions, the Nadaraya-Watson estimator exhibits asymptotic normality, allowing for inference about the conditional mean function ( m(x) ). This property is particularly useful in nonparametric statistics, as it provides a foundation for constructing confidence intervals and hypothesis tests based on the estimator.