Linear Algebra with R – Part II – Vector and Matrix Operations


Transpose

The transpose of a matrix means turning rows (columns) into columns (rows).

For instance, if the matrix is a vector,

Screen Shot 2015-02-25 at 20.28.49

Generalizing, if A is of order m x n its transpose, A’ (or A⊤ ) is a matrix of order n x m.

Transpose

To obtain the transpose of a matrix, we use the transpose operator t()

For instance:

Tip: Transposition is reflexive: (A’)’ = A

Trace

The trace of a square matrix is defined as the sum of the diagonal elements.

For a matrix A of order m x m:

Screen Shot 2015-02-25 at 21.59.30

For example, for

the trace is given by:

In R, we use a combination of the operators sum() and diag().

Note that the command will work even on rectangular matrices. For a m × n matrix, where n < m, the command will sum the diagonal elements of the first n rows in the matrix.

Addition and Subtraction

Any two matrices (or vectors) can be added or subtracted if they are of the same order m x n. In this case, we say that the matrices are conformable for addition.

Because of the matrices are of the same order, the resulting matrix is also of the order m x n.

The elements of the new matrix are given by the sum (+) or difference (-) of the corresponding elements in the original matrices:

Let’s start with an example with vectors.

Let v1 be a column vector (this is the vector “by default”, unless otherwise stated) with values: 1,3,2, and v2 another column vector with values: 2,8,9

A formal representation of a sum of matrices could be this:

MatrixAddition

In R, to add two conformable matrices, A and B, we do:

For instance, let’s create the following matrices:

And now, if we sum both matrices,

ExampleMatrixAdd

We will get;

To subtract B from A:

ExampleMatrixSub

Matrix addition and subtraction have several important properties.

  • Matrix addition complies with the commutative law: A+B = B+A.
  • Matrix subtraction does not comply with the commutative law: A − B <> B − A.
  • Matrix addition complies with the associative law: (A + B) + C = A + (B + C) = A + B + C.
  • The transpose of a sum of matrices is equal to the sum of the transposes: (A + B)’ = A’ + B’.
  • The transpose of a difference between matrices is equal to the difference between the transposes: (A − B)’ = A’ − B’.
  • The trace of a matrix sum is equal to the sum of the traces: tr(A + B) = tr(A) + tr(B). Similarly, tr(A − B) = tr(A) − tr(B).

Multiplication

R supports all of the major forms of matrix multiplication.

Scalar Multiplication

For a number k and a matrix A, the product kA is the matrix obtained by multiplying each element in A by k.

Thus, the result of this operation is a new matrix of the same order as the original matrix.

Formally:

Let k denote a scalar and let B=kA, where A is of order m x n. Then

  1. B is of order m x n.
  2. b[i,j] =k x a[i,j] for all i,j.

ScalarMultiplication

For example,

A

If k=5, we should have the following result:

Scalar multiplication has several important properties:

k(A + B) = kA + kB

(kA)’ = kA’

Inner Products

Inner or dot products involve the multiplication of two vectors.

Let u a row vector of order 1 x n, and v a column vector of order n x 1, then the inner product uv sometimes also written as < u, v >, is defined by:

Screen Shot 2015-02-26 at 12.13.55

In other words, the inner product is the sum of the products between corresponding elements in the vectors:

The first element in the row vector is multiplied by the first element of the column vector, the second element of the row vector is multiplied by the second element of the column vector, etc. All of these product terms are summed so that a single value is returned by the inner product.

Note: It is required that both vectors have the same number of elements.

For example:

Consider the row vector

and the column vector

The inner product < u, v >, exists because both have the same number of elements (u has 3 columns and v has 3 rows). This inner product is given by:

That is:

Matrix multiplication

Two matrices, A and B, can be multiplied if A is of the order m x n and B is of the order n x p.

In other words, matrix multiplication is only possible when the second dimension of A equals the first dimension of B (n). Thus, the resulting matrix, C, is of the order m x p.

Formally:

Screen Shot 2015-02-26 at 11.17.54

We can say that c[i,j] is equal to the inner product of the i-th row in A and the j-th column in B.

Screen Shot 2015-02-26 at 11.29.15

In R, a matrix product can be performed using

Let’s see an example. Consider the following matrices:

First, we verify that A and B are conformable for multiplication.

The matrix A is of order 3 x 2, while the matrix B is of order 2 x 2. Since A has as many columns as B has rows, the matrix product is defined. Now, let’s multiply both matrices:

What happened here:

Screen Shot 2015-02-26 at 11.43.37

Matrix multiplication has several important properties:

  • The product of a matrix, A, and a conformable identity matrix is the matrix itself: IA = AI = A.
  • Matrix multiplication follows the associative law: A(BC) = (AB)C = ABC.
  • Matrix multiplication follows the distributive law: A(B+C) = AB+AC.
  • The transpose of a product is equal to the product of the transposes in reversed order: (AB)’ = B’A’.
  • If the matrix products AB and BA are both defined, then the trace of these products is identical: tr(AB) = tr(BA). If B = A’, this implies tr(AA’) = tr(A’A).
  • If the product of a matrix with itself is equal to the matrix, we call this matrix an idempotent matrix: AA = A.

Kronecker Products

Kronecker products or direct products are another variation of matrix multiplication with important statistical applications.

A ⊗ B, takes each element of the matrix A to create scalar products with the second matrix B.

For example, if A is of order 3×2 and B is of order 2×3, then A⊗B is given by

Kronecker Products

The dimension of the new matrix is (assuming A=mxn and B=pxq):

To obtain Kronecker products in R:

For instance:

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS