Asymptotic Normality of Maximum Likelihood Estimators (statistics tutoring)

This means the MLE becomes approximately normal for large samples, with variance equal to the inverse Fisher information — the best possible asymptotic variance among regular estimators.

Why Asymptotic Normality Matters

Asymptotic normality is essential because it:

• provides approximate distributions for estimators
• justifies standard errors and confidence intervals
• enables Wald, score, and likelihood ratio tests
• forms the backbone of modern statistical inference

How Asymptotic Normality of MLE Works

1. Start with the log‑likelihood:
   ℓ(θ) = Σ log f(Xᵢ | θ).
2. First‑order condition:
   ∂ℓ/∂θ | θ̂ = 0.
3. Apply a Taylor expansion around θ₀:
   0 = ℓ′(θ₀) + (θ̂ − θ₀)ℓ″(θ̃).
4. Use the Law of Large Numbers:
   ℓ′(θ₀)/n → 0.
5. Use the Central Limit Theorem:
   √n ℓ′(θ₀) → 𝒩(0, I(θ₀)).
6. Combine results:
   √n(θ̂ − θ₀) → 𝒩(0, I(θ₀)⁻¹).

Numerical Example

Suppose X₁,…,Xₙ are i.i.d. 𝒩(μ, σ²) with known σ².

MLE for μ: μ̂ = x̄.

Fisher information: I(μ) = 1/σ².

Asymptotic distribution: √n(μ̂ − μ) → 𝒩(0, σ²).

This matches the exact finite‑sample distribution of the sample mean — a perfect illustration of asymptotic normality.

Common Mistakes

• Assuming asymptotic normality holds without checking regularity conditions
• Confusing asymptotic variance with finite‑sample variance
• Using MLE normality for very small samples
• Ignoring model misspecification, which breaks the result

Why This Matters in Statistics

Asymptotic normality is central to:

• maximum likelihood inference
• large‑sample confidence intervals
• Wald, score, and likelihood ratio tests
• econometrics, biostatistics, and machine learning

Related Topics

• Statistical Inference
• Mathematical Statistics
• Statistics Blog
• Question Hub (WHY Hub)