ST5215: Advanced Statistical Theory

ST5215: Advanced Statistical Theory
Chen Zehua
Department of Statistics & Applied Probability
Tuesday, October 14, 2014
Chen Zehua
ST5215: Advanced Statistical Theory
Lecture 17: Methods for the derivation of UMVUE
The 1st method for deriving UMVUE: solving for h
I
Find a sufficient and complete statistic T and its distribution.
I
Try some function h to see if E [h(T )] is related to ϑ.
I
Solve for h such that E [h(T )] = ϑ for all P.
Example 3.1 (cont.)
Let X1 , ..., Xn be i.i.d. from the uniform distribution on (0, θ),
θ > 0. The order statistic X (n) is sufficient and complete with
Lebesgue p.d.f. nθ−n x n−1 I(0,θ) (x).
Consider ϑ = g (θ), where g is a differentiable function on (0, ∞).
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.1 (cont.)
I
An unbiased estimator h(X(n) ) of ϑ must satisfy
n
Z
θ g (θ) = n
θ
h(x)x n−1 dx
for all θ > 0.
0
I
Differentiating both sizes of the previous equation and
applying the result of differentiation of an integral lead to
nθn−1 g (θ) + θn g 0 (θ) = nh(θ)θn−1 .
I
Hence, the UMVUE of ϑ is
h(X(n) ) = g (X(n) ) + n−1 X(n) g 0 (X(n) ).
I
In particular, if ϑ = θ, then the UMVUE of θ is (1 + n−1 )X(n) .
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.2
Let X1 , ..., Xn be i.i.d. from
Pthe Poisson distribution P(θ) with an
unknown θ > 0. T (X ) = ni=1 Xi is sufficient and complete for
θ > 0 and has the Poisson distribution P(nθ).
SupposeP
that ϑ = g (θ), where g is a smooth function such that
j
g (x) = ∞
j=0 aj x , x > 0. An unbiased estimator h(T ) of ϑ must
satisfy (for any θ > 0):
∞
X
h(t)nt
t=0
t!
θt = e nθ g (θ)
∞
X
nk
∞
X
θ
aj θj
k!
j=0
k=0


∞
X
X n k aj
 θt .

=
k!
=
t=0
Chen Zehua
k
j,k:j+k=t
ST5215: Advanced Statistical Theory
Example 3.2 (continued)
Thus, a comparison of coefficients in front of θt leads to
h(t) =
t!
nt
X
j,k:j+k=t
nk aj
,
k!
i.e., h(T ) is the UMVUE of ϑ.
In particular, if ϑ = θr for some fixed integer r ≥ 1, then ar = 1
and ak = 0 if k 6= r and
(
0
t<r
h(t) =
t!
t≥r
nr (t−r )!
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.5
Let X1 , ..., Xn be i.i.d. from a power series distribution, i.e.,
P(Xi = x) = γ(x)θx /c(θ),
x = 0, 1, 2, ...,
with a known function γ(x) ≥ 0 and an unknown parameter θ > 0.
It turns out that the joint distribution of X = (X1 , ..., Xn ) is in an
exponential
Pnfamily with a sufficient and complete statistic
T (X ) = i=1 Xi .
Furthermore, the distribution of T is also in a power series family,
i.e.,
P(T = t) = γn (t)θt /[c(θ)]n ,
t = 0, 1, 2, ...,
where γn (t) is the coefficient of θt in the power series expansion of
[c(θ)]n .
This result can help us to find the UMVUE of ϑ = g (θ).
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.5 (continued)
For example, by comparing both sides of
∞
X
h(t)γn (t)θt = [c(θ)]n−p θr ,
t=0
we conclude that the UMVUE of θr /[c(θ)]p is
(
0
T <r
h(T ) =
γn−p (T −r )
T ≥ r,
γn (T )
where r and p are nonnegative integers.
In particular, the case of p = 1 produces the UMVUE γ(r )h(T ) of
the probability P(X1 = r ) = γ(r )θr /c(θ) for any nonnegative
integer r .
Chen Zehua
ST5215: Advanced Statistical Theory
The 2nd method for deriving UMVUE: conditioning
I
Find an unbiased estimator of ϑ, say U(X ).
I
Conditioning on a sufficient and complete statistic T (X ):
E [U(X )|T ] is the UMVUE of ϑ.
I
The distribution of T is not needed. We only need to work
out the conditional expectation E [U(X )|T ].
I
From the uniqueness of the UMVUE, it does not matter
which U(X ) is used. Thus, U(X ) should be chosen so as to
make the calculation of E [U(X )|T ] as easy as possible.
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.3
Let X1 , ..., Xn be i.i.d. from the exponential distribution E (0, θ)
with p.d.f. fθ (x) = 1θ e −x/θ I(0,∞) (x).
Consider the estimation of ϑ = 1 − Fθ (t).
X¯ is sufficient and complete for θ > 0.
I(t,∞) (X1 ) is unbiased for ϑ,
E [I(t,∞) (X1 )] = P(X1 > t) = ϑ.
Hence
T (X ) = E [I(t,∞) (X1 )|X¯ ] = P(X1 > t|X¯ )
is the UMVUE of ϑ.
If the conditional distribution of X1 given X¯ is available, then we
can calculate P(X1 > t|X¯ ) directly.
By Basu’s theorem (Theorem 2.4), X1 /X¯ and X¯ are independent.
By Proposition 1.10(vii),
P(X1 > t|X¯ = x¯) = P(X1 /X¯ > t/X¯ |X¯ = x¯) = P(X1 /X¯ > t/¯
x ).
Chen Zehua
ST5215: Advanced Statistical Theory
To compute this unconditional probability, we need the distribution
!
of
X
n
n
X
X1
Xi = X1
X1 +
Xi .
i=1
i=2
Using theP
transformation technique discussed in §1.3.1 and the
fact that ni=2 Xi is independentPof X1 and has a gamma
distribution, we obtain that X1 / ni=1 Xi has the Lebesgue p.d.f.
(n − 1)(1 − x)n−2 I(0,1) (x). Hence
Z 1
t n−1
P(X1 > t|X¯ = x¯) = (n − 1)
(1 − x)n−2 dx = 1 −
n¯
x
t/(n¯
x)
and the UMVUE of ϑ is
t n−1
T (X ) = 1 − ¯
.
nX
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.4 (cont.)
Let X1 , ..., Xn be i.i.d. from N(µ, σ 2 ) with unknown µ ∈ R and
σ 2 > 0. Let c be a fixed constant and
c −µ
.
ϑ = P(X1 ≤ c) = Φ
σ
Consider the UMVUE of ϑ.
I
I
Since I(−∞,c) (X1 ) is an unbiased estimator of ϑ, the UMVUE
of ϑ is
E [I(−∞,c) (X1 )|T ] = P(X1 ≤ c|T ).
By Basu’s theorem, the ancillary statistic Z (X ) = (X1 − X¯ )/S
is independent of T = (X¯ , S 2 ).
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.4 (continued)
I
Then, by Proposition 1.10(vii),
c − X¯ 2
T = (¯
x, s )
P X1 ≤ c|T = (¯
x, s ) = P Z ≤
S c − x¯
=P Z ≤
.
s
2
I
It can be shown that Z has the Lebesgue p.d.f.
√
f (z) = √
I
nΓ
n−1
2
π(n − 1)Γ
1−
n−2
2
nz 2
(n − 1)2
(n/2)−2
I(0,(n−1)/√n) (|z|)
Hence the UMVUE of ϑ is
Z
P(X1 ≤ c|T ) =
Chen Zehua
(c−X¯ )/S
√
−(n−1)/ n
f (z)dz
ST5215: Advanced Statistical Theory
Example 3.4 (continued)
Now consider the estimation of
1 0 c −µ
ϑ= Φ
,
σ
σ
the Lebesgue p.d.f. of X1 evaluated at a fixed c, where Φ0 is the
first-order derivative of Φ.
I
I
I
By the previous result, the conditional
p.d.f. of X1 given
x
.
X¯ = x¯ and S 2 = s 2 is s −1 f x−¯
s
Let fT be the joint p.d.f. of T = (X¯ , S 2 ). Then
Z Z
1
c − x¯
c − X¯
1
ϑ=
f
f
fT (t)dt = E
.
s
s
S
S
Hence the UMVUE of ϑ is
1
c − X¯
f
.
S
S
Chen Zehua
ST5215: Advanced Statistical Theory
Example
Let X1 , ..., Xn be i.i.d. with Lebesgue p.d.f. fθ (x) = θx −2 I(θ,∞) (x),
where θ > 0 is unknown.
Suppose that ϑ = P(X1 > t) for a constant t > 0.
The smallest order statistic X(1) is sufficient and complete for θ.
Hence, the UMVUE of ϑ is
P(X1 > t|X(1) ) = P(X1 > t|X(1) = x(1) )
X1
t X
=
x
=P
>
(1)
X(1)
X(1) (1)
X1
t =P
>
X = x(1)
X(1)
x(1) (1)
X1
=P
>s
X(1)
(Basu’s theorem), where s = t/x(1) .
If s ≤ 1, this probability is 1.
Chen Zehua
ST5215: Advanced Statistical Theory
Consider s > 1 and assume θ = 1 in the calculation:
X
n
X1
X1
P
>s =
P
> s, X(1) = Xi
X(1)
X(1)
i=1
n
X
X1
> s, X(1) = Xi
=
P
X(1)
i=2
X1
= (n − 1)P
> s, X(1) = Xn
X(1)
= (n − 1)P (X1 > sXn , X2 > Xn , ..., Xn−1 > Xn )
Z
n
Y
1
dx1 · · · dxn
= (n − 1)
2
x1 >sxn ,x2 >xn ,...,xn−1 >xn i=1 xi
#
Z ∞ "Z ∞ n−1
Y Z ∞ 1
1
1
= (n − 1)
dxi
dx1 2 dxn
2
2
xn
x1
1
sxn i=2
xn xi
Z ∞
(n − 1)x(1)
1
= (n − 1)
dxn =
n+1
nt
sxn
1
Chen Zehua
ST5215: Advanced Statistical Theory
Example (continued)
This shows that the UMVUE of P(X1 > t) is
(
(n−1)X(1)
X(1) < t
nt
h(X(1) ) =
1
X(1) ≥ t
Use the method of finding h:
n
The Lebesgue p.d.f. of X(1) is xnθ
n+1 I(θ,∞) (x).
Let the UMVUE be h(X(1) ) where, for all θ > 0, h satisfies
Z ∞
nθn
Pθ (X1 > t) =
h(x)dx.
x n+1
θ
If θ ≥ t, then Pθ (X1 > t) = 1. Hence , for all θ ≥ t,
Z ∞
nθn
h(x)dx,
1=
x n+1
θ
which implies that h(x) = 1 when x ≥ t.
Chen Zehua
ST5215: Advanced Statistical Theory
If θ < t,
∞
nθn
dx
x n+1
θ
Z t
Z ∞
Z t
nθn
nθn
nθn
θn
=
h(x) n+1 dx +
dx
=
h(x)
dx
+
x
x n+1
x n+1
tn
θ
t
θ
Since P(X1 > t) = θ/t, we have
Z t
Z t
θ
nθn
θn
1
n
1
=
h(x) n+1 dx + n i.e. n−1 =
h(x) n+1 dx + n
t
x
t
tθ
x
t
θ
θ
Differentiating both sizes w.r.t. θ leads to
n
n−1
− n = −h(θ) n+1
tθ
θ
Hence, for any X(1) < t,
Z
E [h(X(1) )] =
h(X(1) ) =
Chen Zehua
h(x)
(n − 1)X(1)
.
nt
ST5215: Advanced Statistical Theory
A necessary and sufficient condition for UMVUE
Theorem 3.2
Let U be the set of all unbiased estimators of 0 with finite
variances and T be an unbiased estimator of ϑ with E (T 2 ) < ∞.
(i) A necessary and sufficient condition for T (X ) to be a UMVUE
of ϑ is that E [T (X )U(X )] = 0 for any U ∈ U and any P ∈ P.
(ii) Suppose that T = h(T˜ ), where T˜ is a sufficient statistic for
P ∈ P and h is a Borel function.
Let UT˜ be the subset of U consisting of Borel functions of T˜ .
Then a necessary and sufficient condition for T to be a
UMVUE of ϑ is that E [T (X )U(X )] = 0 for any U ∈ UT˜ and
any P ∈ P.
Chen Zehua
ST5215: Advanced Statistical Theory
Proof of Theorem 3.2(i)
Suppose that T is a UMVUE of ϑ. Then Tc = T + cU, where
U ∈ U and c is a fixed constant, is also unbiased for ϑ and, thus,
Var(Tc ) ≥ Var(T )
c ∈ R, P ∈ P,
which is the same as
c 2 Var(U) + 2cCov(T , U) ≥ 0
c ∈ R, P ∈ P.
This is impossible unless Cov(T , U) = E (TU) = 0 for any P ∈ P.
Suppose now E (TU) = 0 for any U ∈ U and P ∈ P.
Let T0 be another unbiased estimator of ϑ with Var(T0 ) < ∞.
Then T − T0 ∈ U and, hence,
E [T (T − T0 )] = 0
P ∈ P,
which with the fact that ET = ET0 implies that
Var(T ) = Cov(T , T0 )
P ∈ P.
Note that [Cov(T , T0 )]2 ≤ Var(T )Var(T0 ).
Hence Var(T ) ≤ Var(T0 ) for any P ∈ P.
Chen Zehua
ST5215: Advanced Statistical Theory
Proof of Theorem 3.2(ii)
It suffices to show that E (TU) = 0 for any U ∈ UT˜ and P ∈ P
implies that E (TU) = 0 for any U ∈ U and P ∈ P
Let U ∈ U.
Then E (U|T˜ ) ∈ UT˜ and the result follows from the fact that
T = h(T˜ ) and
E (TU) = E [E (TU|T˜ )] = E [E (h(T˜ )U|T˜ )] = E [h(T˜ )E (U|T˜ )].
Theorem 3.2 can be used
I
to find a UMVUE,
I
to check whether a particular estimator is a UMVUE, and
I
to show the nonexistence of any UMVUE.
If there is a sufficient statistic, then by Rao-Blackwell’s theorem,
we only need to focus on functions of the sufficient statistic and,
hence, Theorem 3.2(ii) is more convenient to use.
Chen Zehua
ST5215: Advanced Statistical Theory
Corollary 3.1
(i) Let Tj be a UMVUE of ϑj , j = 1, ..., k, where k is a fixed
positive
Pinteger.
P
Then kj=1 cj Tj is a UMVUE of ϑ = kj=1 cj ϑj for any
constants c1 , ..., ck .
(ii) Let T1 and T2 be two UMVUE’s of ϑ.
Then T1 = T2 a.s. P for any P ∈ P.
Example 3.7
Let X1 , ..., Xn be i.i.d. from the uniform distribution on the interval
(0, θ).
In Example 3.1, (1 + n−1 )X(n) is shown to be the UMVUE for θ
when the parameter space is Θ = (0, ∞).
Suppose now that Θ = [1, ∞).
Then X(n) is not complete, although it is still sufficient for θ.
Thus, Theorem 3.1 does not apply to X(n) .
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.7 (continued)
We now use Theorem 3.2(ii) to find a UMVUE of θ.
Let U(X(n) ) be an unbiased estimator of 0. Since X(n) has the
Lebesgue p.d.f. nθ−n x n−1 I(0,θ) (x),
Z 1
Z θ
0=
U(x)x n−1 dx +
U(x)x n−1 dx for all θ ≥ 1.
0
1
This implies that U(x) = 0 a.e. Lebesgue measure on [1, ∞) and
Z 1
U(x)x n−1 dx = 0.
0
Consider T = h(X(n) ). To have E (TU) = 0, we must have
Z 1
h(x)U(x)x n−1 dx = 0.
0
Thus, we may consider the following function:
c
0≤x ≤1
h(x) =
bx
x > 1,
where c and b are some constants.
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.7 (continued)
From the previous discussion,
E [h(X(n) )U(X(n) )] = 0,
θ ≥ 1.
Since E [h(X(n) )] = θ, we obtain that
θ = cP(X(n) ≤ 1) + bE [X(n) I(1,∞) (X(n) )]
= cθ−n + [bn/(n + 1)](θ − θ−n ).
Thus, c = 1 and b = (n + 1)/n. The UMVUE of θ is then
1
0 ≤ X(n) ≤ 1
h(X(n) ) =
−1
(1 + n )X(n)
X(n) > 1.
This estimator is better than (1 + n−1 )X(n) , which is the UMVUE
when Θ = (0, ∞) and does not make use of the information about
θ ≥ 1. When Θ = (0, ∞), this estimator is not unbiased.
In fact, h(X(n) ) is complete and sufficient for θ ∈ [1, ∞).
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.7 (continued)
It suffices to show that
1
0 ≤ X(n) ≤ 1
X(n)
X(n) > 1.
is complete and sufficient for θ ∈ [1, ∞).
The sufficiency follows from the fact that the joint p.d.f. of
X1 , ..., Xn is
1
1
I(0,θ) (X(n) ) = n I(0,θ) (g (X(n) )).
n
θ
θ
If E [f (g (X(n) )] = 0 for all θ > 1, then
Z θ
Z 1
Z θ
n−1
n−1
0=
f (g (x))x
dx =
f (1)x
dx +
f (x)x n−1 dx
g (X(n) ) =
0
0
1
for all θ > 1.
Letting θ → 1 we obtain that f (1) = 0.
Then
Z θ
0=
f (x)x n−1 dx
1
for all θ > 1, which implies f (x) = 0 a.e. for x > 1.
Hence, g (X(n) ) is complete.
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.8
Let X be a sample (of size 1) from the uniform distribution
U(θ − 12 , θ + 21 ), θ ∈ R. There is no UMVUE of ϑ = g (θ) for any
nonconstant function g .
Note that an unbiased estimator U(X ) of 0 must satisfy
Z θ+ 1
2
U(x)dx = 0
for all θ ∈ R.
θ− 21
Differentiating both sides of the previous equation and applying
the result of differentiation of an integral lead to
U(x) = U(x + 1)
a.e. m,
where m is the Lebesgue measure on R.
If T is a UMVUE of g (θ), then T (X )U(X ) is unbiased for 0 and,
hence,
T (x)U(x) = T (x + 1)U(x + 1) a.e. m,
where U(X ) is any unbiased estimator of 0.
Chen Zehua
ST5215: Advanced Statistical Theory
Example 3.8 (continued)
Since this is true for all U,
T (x) = T (x + 1)
Since T is unbiased for g (θ),
Z θ+ 1
2
g (θ) =
T (x)dx
a.e. m.
for all θ ∈ R.
θ− 12
Differentiating both sides of the previous equation and applying
the result of differentiation of an integral, we obtain that
g 0 (θ) = T θ + 12 − T θ − 12 = 0 a.e. m.
Chen Zehua
ST5215: Advanced Statistical Theory

Download Report