Probability Spaces & Random Variables
Probability Spaces & Random Variables
A rigorous foundation for stochastic processes begins with a probability space $( \Omega, \mathcal{F}, P )$. The sample space $\Omega$ collects all possible outcomes; the $\sigma$-algebra $\mathcal{F}$ specifies which subsets are measurable events; and the probability measure $P$ assigns numbers in $[0,1]$ to those events satisfying Kolmogorov's axioms. Random variables are then measurable functions $X:\Omega\to\mathbb{R}$, and stochastic processes are indexed families $\{X_t\}_{t\in T}$ of random variables on a common probability space.
A filtration $\{\mathcal{F}_t\}$ is an increasing family of sub-$\sigma$-algebras representing the information accumulated up to time $t$. A process is adapted if $X_t$ is $\mathcal{F}_t$-measurable for each $t$.
Probability Space
A triple $(\Omega, \mathcal{F}, P)$ where $\Omega$ is the sample space, $\mathcal{F}\subseteq 2^\Omega$ is a $\sigma$-algebra (closed under complements and countable unions), and $P:\mathcal{F}\to[0,1]$ satisfies $P(\Omega)=1$ and countable additivity: $$P\!\left(\bigcup_{n=1}^\infty A_n\right)=\sum_{n=1}^\infty P(A_n)$$ for pairwise disjoint $\{A_n\}\subseteq\mathcal{F}$.
Existence of Conditional Expectation
Let $X\in L^1(\Omega,\mathcal{F},P)$ and $\mathcal{G}\subseteq\mathcal{F}$ a sub-$\sigma$-algebra. There exists a $\mathcal{G}$-measurable random variable $E[X|\mathcal{G}]$, unique a.s., satisfying $$\int_G E[X|\mathcal{G}]\,dP = \int_G X\,dP \quad \forall G\in\mathcal{G}.$$
Example 1 — Discrete Probability Space
Roll a fair die: $\Omega=\{1,2,3,4,5,6\}$, $\mathcal{F}=2^\Omega$, $P(\{k\})=1/6$. Define $X(\omega)=\omega^2$. Then $E[X]=\frac{1}{6}(1+4+9+16+25+36)=\frac{91}{6}\approx 15.17$.
Example 2 — Conditional Expectation
Let $\Omega=\{1,\ldots,6\}$, $\mathcal{G}=\sigma(\{\{1,2,3\},\{4,5,6\}\})$, $X=\omega$. Then $E[X|\mathcal{G}](\omega)=2$ if $\omega\le 3$ and $5$ if $\omega\ge 4$, since $E[X|\omega\le 3]=(1+2+3)/3=2$.
Practice
- Verify that $P(A^c)=1-P(A)$ follows from the Kolmogorov axioms.
- If $X\sim\text{Uniform}[0,1]$, compute $E[X^2]$ and $\text{Var}(X)$.
- Show $E[E[X|\mathcal{G}]]=E[X]$ (tower property with $\mathcal{G}_0=\{\emptyset,\Omega\}$).
Show Answer Key
1. By axiom 3 (countable additivity) on the disjoint union $A\cup A^c=\Omega$: $P(A)+P(A^c)=P(\Omega)=1$, so $P(A^c)=1-P(A)$.
2. $E[X^2]=\int_0^1 x^2\,dx=\tfrac{1}{3}$. $\text{Var}(X)=E[X^2]-(E[X])^2=\tfrac{1}{3}-\tfrac{1}{4}=\tfrac{1}{12}$.
3. With $\mathcal{G}_0=\{\emptyset,\Omega\}$, $E[X|\mathcal{G}_0]=E[X]$ (a constant). Then $E[E[X|\mathcal{G}_0]]=E[E[X]]=E[X]$. The general case follows by the same tower-property argument applied to any sub-$\sigma$-algebra.