If \(\u = (U_1 \,\, U_2 \,\, \cdots U_n)\) is a vector of random variables, then \(\Ex(\u) = \left(\Ex(U_1) \,\, \Ex(U_2) \,\, \cdots \Ex (U_n)\right)\). Meanwhile, \(\Var \u\) is an \(n \times n\) matrix with elements
\[\left(\Var(\u)\right)_{ij} = \Ex\left\{(U_i - \mu_i)(U_j - \mu_j)\right\}\]where \(\mu_i=\Ex(U_i)\). The matrix \(\Var(\u)\) is referred to as the variance-covariance matrix of \(\u\).
With scalars, we know that \(x^2\) is always positive (or at least, non-negative). The equivalent notion for matrices is that \(\X \Tr\X\) is said to be a positive definite matrix. More precisely, If $\X$ is full rank, then $\X \Tr\X$ is positive definite; if $\X$ is singular (not of full rank), then $\X \Tr\X$ is said to be positive semidefinite.
Letting \(\A\) and \(\B\) denote matrices of constants and \(\x\) a random vector with mean \(\bm\) and variance \(\bS\),
\[\as{\Ex(\A \Tr\x) &= \A \Tr\bm \\ \Var(\A \Tr\x) &= \A \Tr\bS\A \\ \Ex(\x \Tr\A\x) &= \bm \Tr\A\bm + \trace(\A\bS) \\ \cov(\A \Tr \x, \B \Tr \x) &= \A \Tr \bS \B}\]Where the operator \(\trace\) (defined for any square matrix) refers to the trace of a matrix, defined as the sum of its diagonal elements:
\[\trace(\A) = \sum_i A_{ii}\]Some basic facts about traces:
\[\as{\trace(\A\B) &= \trace(\B\A)\\ \trace(\A+\B) &= \trace(\A) + \trace(\B)\\ \trace(c\A) &= c\,\trace(\A)}\]A further fact about traces that is not at all obvious but nonetheless useful is that if a matrix \(\A\) is idempotent, then \(\trace(\A)=\textrm{rank}(\A)\).
Finally, assuming that \(\A \Tr \x\) and \(\B \Tr \y\) are the same dimension, we have
\[\Var(\A \Tr \x + \B \Tr \y) = \A \Tr \bS_A \A + \B \Tr \bS_B \B + \A \Tr \cov(\x, \y) \B + \B \Tr \cov(\y, \x) \A.\]