NationMaster - Encyclopedia: Covariance

For the physics topics, see covariant transformation; about the mathematics example for groupoids, see covariance in special relativity; for the computer science topic see covariance and contravariance (computer science), in mathematics and theoretical physics see covariance and contravariance of vectors, in category theory see covariance and contravariance of functors.

In probability theory and statistics, covariance is the measure of how much two random variables vary together (as distinct from variance, which measures how much a single variable varies). If two variables tend to vary together (that is, when one of them is above its expected value, then the other variable tends to be above its expected value too), then the covariance between the two variables will be positive. It has been suggested that this article or section be merged with Covariant. ... In mathematics, especially in category theory and homotopy theory, a groupoid is a concept (first developed by Heinrich Brandt in 1926) that simultaneously generalises groups, equivalence relations on sets, and actions of groups on sets. ... A covariant type operator in a type system preserves the ordering â‰¤ of types. ... It has been suggested that Covariant transformation be merged into this article or section. ... In category theory, a branch of mathematics, a functor is a special type of mapping between categories. ... Probability theory is the branch of mathematics concerned with analysis of random phenomena. ... This article is about the field of statistics. ... This article is about mathematics. ...

On the other hand, if one of them is above its expected value and the other variable tends to be below its expected value, then the covariance between the two variables will be negative.

The covariance between two real-valued random variables X and Y, with expected values $scriptstyle E(X),=,mu$ and $scriptstyle E(Y),=,nu$ is defined as In mathematics, the real numbers may be described informally as numbers that can be given by an infinite decimal representation, such as 2. ... In probability theory, a random variable is a quantity whose values are random and to which a probability distribution is assigned. ... In probability theory the expected value (or mathematical expectation) of a random variable is the sum of the probability of each possible outcome of the experiment multiplied by its payoff (value). Thus, it represents the average amount one expects as the outcome of the random trial when identical odds are...

$operatorname{Cov}(X, Y) = operatorname{E}((X - mu) (Y - nu)), ,$

where E is the expected value operator. This can also be written: In probability theory the expected value (or mathematical expectation) of a random variable is the sum of the probability of each possible outcome of the experiment multiplied by its payoff (value). Thus, it represents the average amount one expects as the outcome of the random trial when identical odds are...

$operatorname{Cov}(X, Y) = operatorname{E}(X cdot Y) - mu nu. ,$

If X and Y are independent, then their covariance is zero. This follows because under independence,

$E(X cdot Y)=E(X) cdot E(Y)=munu.$

Recalling the second form of the covariance given above, and substituting, we get

$operatorname{Cov}(X, Y) = mu nu - mu nu = 0.$

The converse, however, is not true: if X and Y have covariance zero, they need not be independent.

The units of measurement of the covariance Cov(X, Y) are those of X times those of Y. By contrast, correlation, which depends on the covariance, is a dimensionless measure of linear dependence. Measurement is the determination of the size or magnitude of something. ... Positive linear correlations between 1000 pairs of numbers. ... In dimensional analysis, a dimensionless number (or more precisely, a number with the dimensions of 1) is a pure number without any physical units. ...

Random variables whose covariance is zero are called uncorrelated. In probability theory and statistics, to call two real-valued random variables X and Y uncorrelated means that their correlation is zero, or, equivalently, their covariance is zero. ...

1 Properties
- 1.1 Relationship to inner products
2 Covariance matrices and operators
3 See also

Properties

If X, Y are real-valued random variables and a, b, c, d are constant ("constant" in this context means non-random), then the following facts are a consequence of the definition of covariance:

$operatorname{Cov}(X, X) = operatorname{Var}(X),$

$operatorname{Cov}(X, Y) = operatorname{Cov}(Y, X),$

$operatorname{Cov}(aX, bY) = ab, operatorname{Cov}(X, Y),$

$operatorname{Cov}(X+a, Y+b) = operatorname{Cov}(X, Y),$

$operatorname{Cov}(aX+bY, cW+dV) = ac,operatorname{Cov}(X,W)+ad,operatorname{Cov}(X,V)+bc,operatorname{Cov}(Y,W)+bd,operatorname{Cov}(Y,V),$

For sequences X₁, ..., X_n and Y₁, ..., Y_m of random variables, we have

$operatorname{Cov}left(sum_{i=1}^n {X_i}, sum_{j=1}^m{Y_j}right) = sum_{i=1}^n{sum_{j=1}^m{operatorname{Cov}left(X_i, Y_jright)}}.,$

For a sequence X₁, ..., X_n of random variables, we have

$operatorname{Var}left(sum_{i=1}^n X_i right) = sum_{i=1}^n operatorname{Var}(X_i) + 2sum_{i,j,:,i<j} operatorname{Cov}(X_i,X_j).$

Relationship to inner products

Many of the properties of covariance can be extracted elegantly by observing that it satisfies similar properties to those of an inner product:
In mathematics, an inner product space is a vector space with additional structure, an inner product (also called a scalar product), which allows us to introduce geometrical notions such as angles and lengths of vectors. ...

(1) bilinear: for constants a and b and random variables X, Y, and U, Cov(aX + bY, U) = a Cov(X, U) + bCov(Y, U)

(2) symmetric: Cov(X, Y) = Cov(Y, X)

(3) positive definite: Var(X) = Cov(X, X) ≥ 0, and Cov(X, X) = 0 implies that X is a constant random variable (K).

It can be shown that the covariance is an inner product over some subspace of the vector space of random variables with finite second moment. In mathematics, a bilinear operator is a generalized multiplication which satisfies the distributive law. ... In mathematics, a definite bilinear form B is one for which B(v,v) has a fixed sign (positive or negative) when it is not 0. ...

Covariance matrices and operators

For column-vector valued random variables X and Y with respective expected values μ and ν, and respective scalar components m and n, the covariance is defined to be the m×n matrix called the covariance matrix: In mathematics, a vector space (or linear space) is a collection of objects (called vectors) that, informally speaking, may be scaled and added. ... In linear algebra, real numbers are called scalars and relate to vectors in a vector space through the operation of scalar multiplication, in which a vector can be multiplied by a number to produce another vector. ... In mathematics, a matrix (plural matrices) is a rectangular table of elements (or entries), which may be numbers or, more generally, any abstract quantities that can be added and multiplied. ... In statistics and probability theory, the covariance matrix is a matrix of covariances between elements of a vector. ...

$operatorname{Cov}(X, Y) = operatorname{E}((X-mu)(Y-nu)^top).,$

For vector-valued random variables, Cov(X, Y) and Cov(Y, X) are each other's transposes. In linear algebra, the transpose of a matrix A is another matrix AT (also written Atr, tA, or Aâ€²) created by any one of the following equivalent actions: write the rows of A as the columns of AT write the columns of A as the rows of AT reflect A...

Even more generally, for a probability measure P on a Hilbert space H with inner product $langle cdot,cdotrangle$ , the covariance operator of P is the operator Cov : H → H given by In mathematics, a probability space is a set S, together with a σ-algebra X on S and a measure P on that σ-algebra such that P(S) = 1. ... The mathematical concept of a Hilbert space (named after the German mathematician David Hilbert) generalizes the notion of Euclidean space in a way that extends methods of vector algebra from the plane and three-dimensional space to spaces of functions. ... In mathematics, an inner product space is a vector space with additional structure, an inner product (also called a scalar product), which allows us to introduce geometrical notions such as angles and lengths of vectors. ...

$mathrm{Cov}(x, y) = int_{H} langle x, z rangle langle y, z rangle , mathrm{d} mathbf{P} (z)$

for all x and y in H. Cov is a self-adjoint operator (the infinite-dimensional analogy of the transposition symmetry in the finite-dimensional case); when P is a centred Gaussian measure, Cov is also a nuclear operator. On a finite-dimensional inner product space, a self-adjoint operator is one that is its own adjoint, or, equivalently, one whose matrix is Hermitian, where a Hermitian matrix is one which is equal to its own conjugate transpose. ... In mathematics, Gaussian measure is a Borel measure on finite-dimensional Euclidean space , closely related to the normal distribution in statistics. ... In mathematics, a nuclear operator or a trace-class operator is a compact operator for which a trace may be defined, such that the trace is finite and independent of the choice of basis. ...

The covariance is sometimes called a measure of "linear dependence" between the two random variables. That does not mean the same thing as in the context of linear algebra (see linear dependence). When the covariance is normalized, one obtains the correlation matrix. From it, one can obtain the Pearson coefficient, which gives us the goodness of the fit for the best possible linear function describing the relation between the variables. In this sense covariance is a linear gauge of dependence. Linear algebra is the branch of mathematics concerned with the study of vectors, vector spaces (also called linear spaces), linear maps (also called linear transformations), and systems of linear equations. ... In linear algebra, a set of elements of a vector space is linearly independent if none of the vectors in the set can be written as a linear combination of finitely many other vectors in the set. ... In probability theory and statistics, correlation, also called correlation coefficient, indicates the strength and direction of a linear relationship between two random variables. ...

Contents

Properties GA_googleFillSlot("encyclopedia_square");

Relationship to inner products

Covariance matrices and operators

See also

Properties