iambaggy

... exploration and discovery

Of vector and matrix products

by · August 21, 2011

Listed below are definitions of some vector or matrix products. I will be needing these definitions in the succeeding posts. Thanks to Jetpack‘s Latex support, I can now include equations in my post.

Outer Product

Given a row vector \mathbf{u} = \begin{bmatrix} u_1 & u_2 & \ldots & u_n\end{bmatrix}  with n elements and another vector \mathbf{v} = \begin{bmatrix} v_1 & v_2 & \ldots & v_m\end{bmatrix}  with m elements, the outer product of u and v is given by

\mathbf{u} \otimes \mathbf{v} =\begin{bmatrix}u_1v_1 & u_1v_2 & \ldots & u_1v_m \\ u_2v_1 & u_2v_2 & \ldots & u_2v_m \\ \vdots & \vdots & \ddots & \vdots \\ u_nv_1 & u_nv_2 & \ldots & u_nv_m \end{bmatrix}.

The outer product as defined above can also be written as a matrix multiplication:

\mathbf{u} \otimes \mathbf{v} = \mathbf{u}^T \mathbf{v} =\begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} \begin{bmatrix} v_1 & v_2 & \ldots & v_m \end{bmatrix} = \begin{bmatrix} u_1 v_1 & u_1 v_2 & \ldots & u_1 v_m \\ u_2 v_1 & u_2 v_2 & \ldots & u_2 v_m \\ \vdots & \vdots & \ddots & \vdots \\ u_nv_1 & u_nv_2 & \ldots & u_n v_m \end{bmatrix}.

Kronecker Product

Given two matrices A which is m-by-n and B, which is q-by-p, the Kronecker product C of A and B is given by:

\mathbf{C} = \mathbf{A} \otimes \mathbf{B} = \begin{bmatrix} a_{11}\mathbf{B} & a_{12}\mathbf{B} & \ldots & a_{1n}\mathbf{B} \\ a_{21}\mathbf{B} & a_{22}\mathbf{B} & \ldots & a_{2n}\mathbf{B} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1}\mathbf{B} & a_{m2}\mathbf{B} & \ldots & a_{mn} \mathbf{B} \end{bmatrix}

where C is an mp-by-nq matrix.

Khatri-Rao Product

Using the above definition, we can now define the Khatri-Rao product C of two matrices A and B as a column-wise Kronecker product of the two matrices. First, rewrite the matrices A and B as follows:

\mathbf{A} = \left[\; \mathbf{a}_1 \mid \mathbf{a}_2 \mid \ldots \mid \mathbf{a}_n \; \right]

and

\mathbf{B} = \left[\; \mathbf{b}_1 \mid \mathbf{b}_2 \mid \ldots \mid \mathbf{b}_n \;\right]

where \mathbf{a}_i, \mathbf{b}_i represent the columns of A and B, respectively. The Khatri-Rao product is given by

\mathbf{C} = \mathbf{A} \odot \mathbf{B} = \left[\; \mathbf{a}_1 \otimes \mathbf{b}_1 \mid \mathbf{a}_2 \otimes \mathbf{b}_2 \mid \ldots \mid \mathbf{a}_n \otimes \mathbf{b}_n \right]

Whew! I thought including equations was easy. Just hover your mouse over the equations above and you’ll see how complicated the code is to write such simple equations. There should be a better way.  Anyone?

Tags:

You may also like...

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.