Introduction

Statistics can be roughly broken down into two problems: hypothesis testing and point estimation. Each type of problem has:

1. Some data \(X_{1}, X_{2} , \dots, X_{n}\) sampled (usually) i.i.d. from some distribution \(P_{\theta}\) with unknown parameter \(\theta\).
2. We make a decision given this data. The decision comes in the form of a function \(f(X_{1}, X_{2}, \dots , X_{n})\) that eats the data and returns the decision

In hypothesis testing, our decision is to determine whether we reject or fail to reject a hypothesis regard \(P_{\theta}\).

In point estimation, our decision is an estimate of some quantity. For example, we may want to estimate the average (\(\mu\)) grade in the class, which we use the sample mean \(\bar{X}\).

Asymptotics play a large role in this course. They concern the behaviour of \(X_{n}\) as \(n\) increases, where \(n\) is the sample size. \(\text{E.g}\) consider the behaviour of an estimator \(\hat{\theta}(X_{1}, X_{2}, \dots X_{n})\) as \(n\) increases. We expect (or hope) that \(\hat{\theta}\) converges to the parameter \(\theta\) we are trying to estimate.

• Note that asymptotics results will not provide much information when the sample size is small.

---

Outline of course topics

1. Notions of convergence of random variables and asymptotics
   • Convergence in Probability
   • Convergence in Distribution
   • Asymptotic Unbiasedness
   • MGFs and the Central Limit Theorem
   • Delta Method
2. Making optimal decisions in point estimation and hypothesis testing problems
   • Maximum Likelihood Estimation
   • Jensen's Inequality
   • Score function, Fisher Information, Cramer-Rao Lower Bound and Efficiency
3. Non-parametric and robust statistical procedures
4. Bayesian statistics