Families of Parametrized Distributions
Each of the following defines a
- A sample space \(X\) + sigma algebra \(\mathcal{S}\)
- A parameter space \(P\)
- A mapping from the parameter space \(P\) to a subset
of the set of measures defined on \((X, \mathcal{S})\)
- This mapping is typically defined by a probability density together with a reference distribution.
- The notation \(F(P, Q, ..)\) of application of a function \(F\) to probability distributions \(P\), \(Q\) is defined to be the pushforward by F of the cartesian product of \(P\), \(Q\), ..
Bernoulli
Parameters :
- p \(\in\) \([0,1] \subset \mathbb{R}\)
Sample Space : \(\{0, 1\}\)
Sigma Algebra : Discrete
Measure : \(P(1) = p\) and \(P(0) = 1-p\)
Arithmetic:
-
XOR : \(B(p) \wedge B(q) = B(p (1 - q) + q (1 - p)) = B(q + p - 2qp)\)
-
OR : \(B(p) \vert B(q) = B(p + q - pq)\)
-
AND : \(B(p) \& B(q) = B(pq)\)
-
NOT : \(!B(p) = B(1-p)\)
-
Addition : \(\sum_{i=1}^N B(p) = B(N, p)\)
Links
- Wikipedia : Bernoulli
Beta
Parameters :
-
\(\alpha\) \(\in\) \(\mathbb{R^+}\)
-
\(\beta\) \(\in\) \(\mathbb{R^+}\)
Sample Space : \([0,1] \subset \mathbb{R}\)
Sigma Algebra : Lebesgue
Density : \(\frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha)\Gamma(\beta)}x^{\alpha-1} (1-x)^{\beta -1}\)
Arithmetic:
Links
- Wikipedia : Beta
Binomial
Parameters :
-
N \(\in\) \(\mathbb{N}\)
-
p \(\in\) \([0,1] \subset \mathbb{R}\)
Sample Space : \([0,N] \subset \mathbb{Z}\)
Sigma Algebra : Discrete
Density : \(\rho(n) = {N\choose n} p^n (1-p)^{N-n}\)
Arithmetic:
- Addition (same p) : \(B(N, p) + B(M, p) = B(N+M, p)\)
Links
Cauchy
Parameters :
Sample Space :
Sigma Algebra :
Arithmetic:
Links
Gamma
Parameters :
-
\(\alpha\) \(\in\) \(\mathbb{R}^+\)
-
\(\beta\) \(\in\) \(\mathbb{R}^+\)
Sample Space : \(\mathbb{R}^+\)
Sigma Algebra : Lebesgue
Density : \(\frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\beta x}\)
Arithmetic:
-
Addition : \(Gamma(\alpha, \beta) + Gamma(\alpha', \beta) = Gamma(\alpha + \alpha', \beta)\)
-
Scaling : \(\eta \times Gamma(\alpha, \beta) = G(\alpha, \beta / \eta)\)
Links
- Wikipedia : Gamma
Multinomial
Parameters :
-
\(N\) \(\in\) \(\mathbb{N}\)
-
\(p\) \(\in\) \([0,1]^d \subset \mathbb{R}^d\) with \(\sum_i p_i = 1\)
Sample Space :
Sigma Algebra :
Density : \(\rho(n_1,n_2,n_3,..,n_m) = choose(N,n_1,n_2,..,n_m) p_1^{n_1} p_2^{n_2} .. p_m^{n_m}\)
Arithmetic:
Links
- Wikipedia : Multinomial
Multi-Normal
Parameters :
Sample Space : \(\mathbb{R}^d\)
Sigma Algebra : Legesgue
Density : \(N(\mu, \Sigma) = \frac{1}\{(2\pi)^{d/2} \sqrt{\Sigma} \} \exp[-1/2 (x-\mu)^T\Sigma^{-1}(x-\mu)]\)
Arithmetic:
-
Addition : \(N(\mu, \Sigma) + N(\mu', \Sigma') = N(\mu+\mu', \Sigma+\Sigma')\)
-
Linear Map, with $A$ invertible : \(A \cdot N(\mu, \Sigma) = N(A \mu, A \Sigma A^T)\)
Links
Normal
Parameters :
-
\(\mu\) \(\in\) \(\mathbb{R}\)
-
\(\sigma\) \(\in\) \(\mathbb{R}^+\)
Sample Space : \(\mathbb{R}\)
Sigma Algebra : Lebesgue
Arithmetic:
-
Addition : \(N(\mu, \sigma) + N(\mu', \sigma') = N(\mu+\mu', \sqrt{\sigma^2 + \sigma'^2})\)
-
Scalar Multiplication : \(\lambda N(\mu, \sigma) = N(\lambda \mu, \lambda \sigma)\)
Links
Poisson
Parameters :
- \(\lambda\) \(\in\) \(\mathbb{R}^+\)
Sample Space : \(\mathbb{N}\)
Sigma Algebra : Discrete
Density : \(\rho(n) = \frac{\lambda^n e^{-n}}{n!}\)
Arithmetic:
- Sum : \(Pois(\lambda_1)+Pois(\lambda_2) = Pois(\lambda_1 + \lambda_2)\)
Links
- Wikipedia : Poisson