Math 311

Class 19

Sum of Exponential Variables

Let \(X_1, X_2, \dots, X_n\) are independent indentically distributed exponential random variables with rate \(\lambda\).

\[X_j \stackrel{\text{iid}}{\sim} \operatorname{Exp}(\lambda) \text{ for } j = 1, 2, \dots, n\]

Let \(X = X_1 + X_2 + \cdots + X_n\)

Then \(X \sim \operatorname{\Gamma}(n,\lambda)\), or \(\displaystyle f(t) = \lambda e^{-\lambda t}\frac{(\lambda t)^{n-1}}{(n-1)!}\)

Comparison of Exponential Variables

Let \(X_1\) and \(X_2\) be independent exponentially distributed with rates \(\lambda_1\) and \(\lambda_2\).

Then \(\displaystyle\operatorname{P}(X_1 < X_2) = \frac{\lambda_1}{\lambda_1 + \lambda_2}\).

Minimum of Exponential Variables

Suppose \(X_i\) is exponential with rate \(\lambda_i\) for \(i = 1, 2, \ldots, n\) and \(X_i\)’s are independent. Then

\[ \begin{aligned} \operatorname{P}(\min(X_1, X_2, \ldots, X_n) > x) &= \operatorname{P}(X_i > x \text{ for each } i = 1, 2, \ldots, n) \\ &= \prod_{i=1}^n \operatorname{P}(X_i > x)\\ &= \prod_{i=1}^n e^{-\lambda_i} x\\ &= \exp\left(-\left(\sum_{i=1}^n \lambda_i\right) x\right) \end{aligned} \]

In other words, \(\min(X_1, X_2, \ldots, X_n)\) has exponential distribution with the rate \(\lambda = \lambda_1 + \lambda_2 + \cdots + \lambda_n\).

Example:

Suppose you arrive at a post office where there are two clerks that are both busy at the time, but there is no one waiting in line in front of you. At the moment one of the two clerks is free, it will be your turn.

Assume that the service time for clerk \(i\) is exponential with rate \(\lambda_i\), for \(i = 1, 2\). Let \(T\) be the total time you spend at the post office. Find \(\operatorname{E} T\).

Counting Processes

A stochastic process \(\{N(t), t \ge 0\}\) is called a counting process if \(N(t)\) represents a number of occurrences of certain event by the time \(t\).

\(N(t) \ge 0\)
\(N(t)\) is an integer for each \(t\)
If \(s < t\) then \(N(s) \le N(t)\)
If \(s < t\) then \(N(t) - N(s)\) is the number of occurrences during the time interval \((s, t]\)

Examples

\(N(t) = {}\) the number of people born in certain town since 0 am, January 1, 1950.
\(N(t) = {}\) the number of people that passed through an entrance of an office building since the time it opened in the morning.
\(N(t) = {}\) the number of time a particle collided with a sensor since the start of an experiment.
Not a counting process: \(N(t) = {}\) the number of people in an office building at the time \(t\).
Not a counting process: \(N(t) = {}\) the total weight of all people that passed through an entrance of an office building since the time it opened in the morning.

Increments

We say that a counting process \(\{N(t), t\ge 0\}\) has independent increments if for any two disjoint intervals \((s_1, t_1)\) and \((s_2, t_2)\) the random variables \(N(t_1) - N(s_1)\) and \(N(t_2) - N(s_2)\) are independent.
We say that a counting process \(\{N(t), t\ge 0\}\) has stationary increments if for any interval \((s, t)\), the distribution of the random variable \(N(t) - N(s)\) depends _only on the length of the interval \((s,t)\).

\(\mathcal{O}\) and \(\mathcal{o}\)

Given two functions \(f\) and \(g\), both defined on the interval \((0,\infty)\):

We say that \(f(x) = \mathcal{O}(g(x))\) as \(x \to \infty\) if there is \(M > 0\) and \(x_0 > 0\) such that \(\left\lvert f(x)\right\rvert \le M \left\lvert g(x)\right\rvert\) for every \(x > x_0\).
We say that \(f(x) = \mathcal{O}(g(x))\) as \(x \to 0\) if there is \(M > 0\) and \(\delta > 0\) such that \(\left\lvert f(x)\right\rvert \le M \left\lvert g(x)\right\rvert\) for every \(0 < x < \delta\).
We say that \(f(x) = \mathcal{o}(g(x))\) as \(x \to \infty\) if _for each \(\varepsilon > 0\) there is \(x_0 > 0\) such that \(\left\lvert f(x)\right\rvert \le \varepsilon \left\lvert g(x)\right\rvert\) for every \(x > x_0\).
- If \(g(x) \neq 0\) for all \(x\), the \(f(x) = \mathcal{o}(g(x))\) as \(x \to \infty\) if and only if \(\displaystyle \lim_{x \to \infty}\frac{f(x)}{g(x)} = 0\).
We say that \(f(x) = \mathcal{o}(g(x))\) as \(x \to 0\) if for each \(\varepsilon > 0\) there is \(\delta > 0\) such that \(\left\lvert f(x)\right\rvert \le \varepsilon \left\lvert g(x)\right\rvert\) for every \(0 < x < \delta\).
- If \(g(x) \neq 0\) for all \(x\), the \(f(x) = \mathcal{o}(g(x))\) as \(x \to 0\) if and only if \(\displaystyle \lim_{x \to 0^+}\frac{f(x)}{g(x)} = 0\).

Poisson Process

A counting process \(\{N(t), t \ge 0\}\) is called a Poisson process with rate \(\lambda\) if all of the following are true:

\(N(0) = 0\)
\(\{N(t), t \ge 0\}\) has independent increments
\(\operatorname{P}(N(t + h) - N(t) = 1) = \lambda h + \mathcal{o}(h)\) as \(h \to 0\)
\(\operatorname{P}(N(t + h) - N(t) \ge 2) = \mathcal{o}(h)\) as \(h \to 0\)

If \(\{N(t), t \ge 0\}\) is a Poisson process with rate \(\lambda\) then for each \(s \ge 0\), \(t > 0\), \[N(s+t) - N(s) \sim \operatorname{Pois}(\lambda t).\]

Each Poisson process has stationary increments