- Definition, density, and moment-generating functions
- Linear combinations
- Quadratic forms
- Marginal and conditional distributions
- Conditional independence and the precision matrix
Definition, density, and moment-generating functions
Let $x$ be a $d \times 1$ random vector with mean vector $\bm$ and covariance matrix $\bS$, where $\textrm{rank}(\bS) = k > 0$. Let $\bG$ be a $k \times d$ matrix such that $\bS = \bG\Tr\bG$. Then $x$ is said to have a d-variate normal distribution of rank $k$ if its distribution is the same as that of the random vector $\bm + \bG \Tr z$, where $z$ is a $k \times 1$ vector of independent standard normal random variables. This is typically denoted $\x \sim \Norm_d(\bm, \bS)$.
The pdf of $\Norm_d(\bm, \bS)$ is
\[(2\pi)^{-d/2} \abs{\bS}^{-1/2} \exp\{ -\tfrac{1}{2} (\x - \bm) \Tr \bS^{-1} (\x - \bm) \}.\]The MGF of $\Norm_d(\bm, \bS)$ is
\[m(\t) = \exp\{ \t \Tr \bm + \tfrac{1}{2} \t \Tr \bS \t \}.\]The characteristic function of $\Norm_d(\bm, \bS)$ is
\[\phi(\t) = \exp\{ i \t \Tr \bm - \tfrac{1}{2} \t \Tr \bS \t \}.\]Linear combinations
Let $\b$ be a $q \times 1$ vector of constants, $\B$ a $q \times d$ matrix of constants, and $\x \sim \Norm_d(\bm, \bS)$. Then
\[\b + \B\x \sim \Norm_q(\B\bm + \b, \B\bS\B \Tr).\]As a corollary, note that
\[\bS^{-1/2} (\x - \bm) \sim \Norm_p(\zero, \I),\]or more generally, if $\bS$ is not full rank,
\[(\bG \bG \Tr)^{-1}\bG (\x - \bm) \sim \Norm_k(\zero, \I).\]Note that $\bS^{-1/2} = \Q\bL^{-1/2}\Q\Tr$, where $\Q\bL\Q\Tr$ is the eigendecomposition of $\bS$, while $(\bG \bG \Tr)^{-1}\bG = \Q_k\bL_k^{-1/2}\Q_k\Tr$, the decomposition including only the $k$ nonzero eigenvalues.
Quadratic forms
If $\z \sim N(0, \bS)$, where $\bS$ is full rank, then $\z\Tr\A\z \sim \chi^2_r$, where $r$ is the rank of $\A$.
If $\z \sim N(\bm, \bS)$, where $\bS$ is full rank, then $\z\Tr\A\z \sim \chi^2_r(\tfrac{1}{2}\mu\Tr\A\mu)$, where $r$ is the rank of $\A$.
Marginal and conditional distributions
In this section, assume $\x \sim \Norm_d(\bm, \bS)$ can be partitioned as follows:
\[\x = \left[ \begin{array}{c} \x_1 \\\\ \x_2 \end{array} \right], \qquad \bm = \left[ \begin{array}{c} \bm_1 \\\\ \bm_2 \end{array} \right], \qquad \bS = \left[ \begin{array}{cc} \bS_{11} & \bS_{12} \\\\ \bS_{21} & \bS_{22} \end{array} \right]\]The marginal distribution of $\x_1$ is
\[\x_1 \sim \Norm(\bm_1, \bS_{11}).\]The conditional distribution of $\x_1$ given $\x_2$ is
\[\x_1 \given \x_2 \sim \Norm \left( \bm_1 + \bS_{12}\bS_{22}^{-1}(\x_2 - \bm_2), \bS_{11} - \bS_{12} \bS_{22}^{-1} \bS_{21} \right).\]Finally, $\x_1$ and $\x_2$ are independent iff $\bS_{12}=\zero$.
Conditional independence and the precision matrix
Suppose we partition $\x$ into $\x_1$, containing two variables of interest, and $\x_2$ containing the remaining variables. Then by the above results and using the inverse of a partitioned matrix, we have that $\x_1 \vert \x_2$ is multivariate normal with covariance matrix $\Theta_{11}^{-1}$, where $\Theta=\Sigma^{-1}$ is the precision matrix.
Thus, if any off-diagonal element of $\Theta$ is zero, then the corresponding variables are conditionally independent given the remaining variables.