Maximum likelihood Shota Gugushvili Multivariate normal MLE Maximum likelihood estimation: multidimensional case Shota Gugushvili Leiden University Leiden, 20 October 2014 Outline Maximum likelihood Shota Gugushvili Multivariate normal MLE 1 Multivariate normal 2 MLE Multivariate normal: particular case Maximum likelihood Shota Gugushvili Multivariate normal MLE Definition Let Z1 Z = ... Zk where Z1 , . . . , Zk ∼ N(0, 1) are independent. Then Z is said to have multivariate normal distribution with mean 0 (k × 1 vector of zeros) and covariance matrix I (k × k identity matrix). The PDF of Z is given by k X 1 1 1 1 2 T f (z) = exp − zj = exp − z z . 2 2 (2π)k/2 (2π)k/2 j=1 Multivariate normal: general case Maximum likelihood Shota Gugushvili Multivariate normal MLE Definition A vector X1 X = ... Xk is said to have multivariate normal distribution with mean µ (vector of length k) and covariance matrix Σ (k × k symmetric, positive definite matrix), which we denote by X ∼ N(µ, Σ), if its PDF is given by 1 1 T −1 f (x; µ, Σ) = exp − (x − µ) Σ (x − µ) . 2 (2π)k/2 |Σ|1/2 Properties of a normal vector Maximum likelihood Shota Gugushvili Multivariate normal Theorem If Z ∼ N(0, I ) and X = µ + Σ1/2 Z , then X ∼ N(µ, Σ). Conversely, if X ∼ N(µ, Σ), then Σ−1/2 (X − µ) ∼ N(0, I ). MLE Theorem Let X ∼ N(µ, Σ). Then Xj ∼ N(µj , Σjj ). Furthermore, aT X ∼ N(aT µ, aT Σa), and V = (X − µ)T Σ−1 (X − µ) ∼ χ2k . Asymptotic normality Maximum likelihood Theorem Shota Gugushvili Let X1 , . . . , Xn be IID random vectors, where X1i .. X = . Xki Multivariate normal MLE with mean µ1 E[X1i ] µ = ... = ... µk E[Xki ] and covariance P matrix Σ. Let X be a vector with components X j = n−1 ni=1 Xji . Then √ n(X − µ) N(0, Σ). MLE and Fisher information Maximum likelihood Shota Gugushvili Multivariate normal MLE Let θ = (θ1 , . . . , θk ), and let θˆ = (θˆ1 , . . . , θˆk ) be the MLE. P Let `n = ni=1 log f (Xi ; θ) and Hjk = ∂ 2 `n . ∂θj ∂θk The k × k matrix In (θ) = −(E[Hjk ]) is called the Fisher information matrix. Let Jn (θ) = In−1 (θ) be the inverse of In (θ). Asymptotic normality Maximum likelihood Shota Gugushvili Multivariate normal MLE Theorem Under mild assumptions, √ n(θˆ − θ) ≈ N(0, Jn ). Also, θˆj − θj se ˆj N(0, 1), where se ˆ 2j is the jth diagonal element of Jn . The approximate covariance of θˆj and θˆk is Jn (j, k). Multiparameter delta method Maximum likelihood Shota Gugushvili Theorem Let τ = g (θ1 , . . . , θk ) and let Multivariate normal ∂g ∂θ1 ∇g = ... MLE ∂g ∂θ1 ˆ Then . Let τˆ = g (θ). τˆ − τ se(ˆ ˆ τ) where se(ˆ ˆ τ) = ˆ ˆ = ∇g (θ). ∇g N(0, 1), q ˆ and ˆ )T Jˆn (∇g ˆ ), with Jˆn = Jn (θ) (∇g Example Maximum likelihood Shota Gugushvili Multivariate normal MLE Example Let X1 , . . . , Xn be IID N(µ, σ 2 ). Suppose we want to estimate τ = g (µ, σ) = σ/µ. It can be shown that 1 σ2 0 Jn (θ) = 2 n 0 σ2 and ∇g = (−σ/µ2 , 1/µ)T . Thus q ˆ )T Jˆn (∇g ˆ ) = √1 se ˆ = (∇g n s 1 σ ˆ2 + . µ ˆ4 2ˆ µ2
© Copyright 2024 ExpyDoc