Unless otherwise specified, we only consider the Euclidean space $\mathbb{R}^n$ with orthonormal basis and its dual space. The tensor is in bold and the scalar is in standard Roman font.

The elements in $\mathbb{R}^m$ are column vectors $\mathbf{a}^i=[a^1,a^2,\cdots,a^m]^T$. The elements in $(\mathbb{R}^n)^*$ are row vectors $ \mathbf{b}_j=[b_1,b_2,\cdots,b_n]$. The orthonormal basis of $\mathbb{R}^m$ can be written as $\boldsymbol{ \delta } ^1,\cdots \boldsymbol{ \delta } ^m $, where the $i$-th component of $\boldsymbol{ \delta } ^i$ is $ 1 $ while other components equal $0$. The orthonormal basis of $(\mathbb{R}^n)^*$ can be written as row vectors $\boldsymbol{ \delta }_1,\cdots, \boldsymbol{ \delta }_n$, where the $i$-th component of $ \boldsymbol{ \delta }_j$ is $1$ while other components equal $0$.

Tensor Product

Define that $(\mathbb{R}^n)^*\otimes \mathbb{R}^m $ is a free vector space with basis $\boldsymbol{ \delta }_j \otimes \boldsymbol{ \delta } ^i$. It is easy to check

$$\begin{align} \otimes : (\mathbb{R}^n)^*\times \mathbb{R}^m &\longrightarrow (\mathbb{R}^n)^*\otimes \mathbb{R}^m \\ \mathbf{b} _j \times \mathbf{a} ^i&\longmapsto \sum_{i}\sum_{j} b_j a^i \boldsymbol{ \delta } _j \otimes \boldsymbol{ \delta } ^i \end{align}$$

is a bilinear mapping. That is to say,

$$\begin{align} (\lambda\mathbf{b}_j+ \mathbf{c}_j )\otimes \mathbf{a}^i&= \lambda\mathbf{b}_j\otimes \mathbf{a}^i +\mathbf{c}_j \otimes \mathbf{a}^i ,\quad&\forall \lambda\in\mathbb{R}, \mathbf{a}^i \in\mathbb{R}^m,\mathbf{b}_j , \mathbf{c}_j\in (\mathbb{R}^n)^* ,\\ \mathbf{b} _j \otimes (\lambda\mathbf{a} ^i + \mathbf{d} ^i ) &= \lambda \mathbf{b}_j\otimes \mathbf{a}^i + \mathbf{b}_j \otimes \mathbf{d}^i ,\quad&\forall \lambda\in\mathbb{R}, \mathbf{a}^i , \mathbf{d}^i \in\mathbb{R}^m,\mathbf{b}_j \in (\mathbb{R}^n)^*. \end{align}$$

According to the isomorphism $\mathcal{L}(\mathbb{R}^n, \mathbb{R}^m )\cong \mathrm{M}_{m\times n}( \mathbb{R} )$, a linear map $f:\mathbb{R}^n\to\mathbb{R}^m\in\mathcal{L}(\mathbb{R}^n, \mathbb{R}^m ) $ can be represented by a matrix

$$\mathbf{A}^i {}_j= \begin{bmatrix} A^1 {}_1 & A^1 {} _2& \cdots & A^1 {}_n \\ A^2 {}_1 & A^2 {}_2& \cdots & A^2 {}_n \\ \vdots& \vdots& & \vdots \\ A^m {}_1 & A^m {}_2& \cdots & A^m {}_n \end{bmatrix}.$$

We can verify that the mapping

$$\begin{align} \varphi: \mathrm{M}_{m\times n}( \mathbb{R} ) &\longrightarrow (\mathbb{R}^n)^*\otimes \mathbb{R}^m\\ \mathbf{A}^i {}_j&\longmapsto \sum_{i}\sum_{j}A^i {}_j \boldsymbol{ \delta } _j \otimes \boldsymbol{ \delta } ^i \end{align}$$

is a linear isomorphism. Therefore, we have the following isomorphic relationship,

$$\mathrm{M}_{m\times n}( \mathbb{R} ) \cong \mathcal{L}(\mathbb{R}^n, \mathbb{R}^m ) \cong (\mathbb{R}^n)^*\otimes \mathbb{R}^m,$$

and the elements in $ (\mathbb{R}^n)^*\otimes \mathbb{R}^m $ can be written as matrices. A simple example is

$$\begin{bmatrix} 1&2 \end{bmatrix} \otimes \begin{bmatrix} 3\\ 4 \end{bmatrix} = \begin{bmatrix} 3\times1&3 \times 2 \\ 4\times1&4 \times 2 \end{bmatrix} .$$

Tensor

Let $V=\mathbb{R}^n$. In differential geometry, we often consider the tensor product between a vector space $V$ and its dual space $V^*$. The tensor product

has elements

$$\mathbf{A}^{i_1,\cdots,i_{r}} {}_{ j_1,\cdots,j_{s} } =\sum\limits_{i_1,\cdots,i_{r},\\ j_1,\cdots,j_{s} } A^{ i_1 \cdots i_{r}} {}_{ j_1 \cdots j_{s} } \boldsymbol{ \delta } ^{i_1} \otimes \cdots \otimes \boldsymbol{ \delta } ^{i_{r}} \otimes \boldsymbol{ \delta } _{j_1} \otimes \cdots \otimes \boldsymbol{ \delta } _{j_{s}}$$

called $(r,s)$ type mixed tensors, or simply $(r,s)$ tensors. Denote

$$\boldsymbol{ \delta } ^ {i_1,\cdots,i_{r}} {}_{ j_1,\cdots,j_{s} }:= \boldsymbol{ \delta } ^{i_1} \otimes \cdots \otimes \boldsymbol{ \delta } ^{i_{r}} \otimes \boldsymbol{ \delta } _{j_1} \otimes \cdots \otimes \boldsymbol{ \delta } _{j_{s}} ,$$

then we have

$$\mathbf{A}^{i_1,\cdots,i_{r}} {}_{ j_1,\cdots,j_{s} } =\sum\limits_{i_1,\cdots,i_{r},\\ j_1,\cdots,j_{s} } A^{ i_1 \cdots i_{r}} {}_{ j_1 \cdots j_{s} } \boldsymbol{ \delta } ^ {i_1\cdots i_{r}} {}_{ j_1\cdots j_{s} }$$

If we introduce the shorthand notation $\alpha= (i_1,\cdots,i_{r})$, $\beta= (j_1,\cdots,j_{s})$, the above equation can be further simplified to

$$\mathbf{A}^{ \alpha } {}_{ \beta } =\sum\limits_{ \alpha , \beta } A^{ \alpha } {}_{ \beta } \boldsymbol{ \delta } ^ { \alpha } {}_{ \beta }.$$

Tensor Operations

Tensors have two operations: multiplication and contraction. The multiplication of an $(r_1,s_1)$ tensor with an $(r_2,s_2)$ tensor is simply the tensor product

$$\begin{align} \otimes : V_{s_1}^{r_1} \times V_{s_2}^{r_2} &\longrightarrow V_{s_1+s_2}^{r_1+r_2} \\ \mathbf{A}^{ \alpha_1 } {}_{ \beta_1} \times \mathbf{B}^{ \alpha_2} {}_{ \beta_2} &\longmapsto \sum\limits_{ \alpha_1 , \beta_1, \alpha_2 , \beta_2} A^{ \alpha_1}{}_{ \beta_1 } B^{ \alpha_2} {}_{ \beta_2 }\boldsymbol{\delta }^{ \alpha_1 }{}_{ \beta_1} {}^{\alpha_ 2 } {}_{\beta_ 2 } \end{align}\ .$$

The trace is defined as the following mapping,

$$\begin{align} \mathrm{Tr}: V^*\otimes V &\longrightarrow \mathbb{R}\\ \sum_{i,j} A^i {}_j \boldsymbol{ \delta } _j \otimes \boldsymbol{ \delta } ^i &\longmapsto \sum_{i}A^i {}_i \end{align}\ ,$$

It is easy to verify that this is a linear map. From this, we can define the contraction operation. Let $1 \le p \le r$, $1 \le q \le s$, the contraction of an $(r,s)$ tensor is defined as

$$\begin{align} \mathrm{Tr_{pq}}: V_{s}^{r} &\longrightarrow V_{s-1}^{r-1} \\ \mathbf{A}^{i_1,\cdots,i_{r}} {}_{ j_1,\cdots,j_{s} }&\longmapsto \sum\limits_{i_1, \cdots, \widehat{i_p}, \cdots,i_{r},\\ j_1, \cdots ,\widehat{j_q}, \cdots,j_{s} } \mathrm{Tr} \left( \sum_{i_p,j_q} A^{ i_1 \cdots i_p \cdots i_{r}} {}_{ j_1 \cdots j_q \cdots j_{s} } \boldsymbol{ \delta } ^{ i_p }{}_{ j_q } \right) \boldsymbol{ \delta } ^ {i_1\cdots \widehat{i_p}\cdots i_{r}} {}_{ j_1 \cdots \widehat{j_q} \cdots j_{s} } \end{align},$$

where the indices marked with ${}^\widehat{\hspace{0.8em}}$ have been omitted.