(Kronecker summation lemma) Let
,
be sequences of real numbers,
.
If the series
converge then
,
.

Proof

We
set
Observe
that
We make a change
in the second
term:
Since
we can use the mean value
theorem
where the
is a "middle" value with the
property
Note
that
Hence,

Proposition

(Strong law of large
numbers for mean zero). Assume that the function
satisfies the following criteria

1.
,

2.
,
,

3.
is increasing,

4.
is decreasing.

Let
be a sequence of independent r.v. with
for every
.
Let
be a sequence of real numbers,
,
.
If
then
and

Proof

We denote
the d.f. of the r.v.
.

We aim to apply the proposition
(
Kolmogorov three series
theorem
) to the series
.
Hence, we introduce the
r.v.
and proceed to verify that the series

(a).
According to the condition 3 the function
is increasing, hence
.
We
continue

(b).
Note that
.
Hence,
.
According to the condition 3 the function
is increasing and according to 1
.
Hence
.
We
continue

(c).
According to the condition 4 the function
is decreasing. Hence
.
We
continue

Proposition

(Strong law of large
numbers for iid r.v.) Let
be a sequence of iid r.v.
Then

Proof

We
define
We
have
According to the proposition
(
Estimate of mean by
probability series
) the last series converge. Hence,
and
are equivalent sequences and, according to the proposition
(
Property of equivalent
sequences of r.v.
), it suffices to prove the first part of the proposition
for
.
To accomplish it we apply the proposition
(
Strong law of large
numbers for mean zero
) to the sequence
and
.
We
estimate
Here the
is the d.f. of
.
The term
may be estimated using
.
The result is
for some constant
.
In addition,
on the set
.