Tensors

(Spivac, p. 75)

Definition 36 (Tensors)

Let \(V\) be a vector space, often \(V = \mathbb{R}^n\). A k-tensor \(T\) is a multilinear mapping from \(V^k\) to \(\mathbb{R}\):

\[\begin{split}T : \left\{ \begin{array}{lr} V^k \to \mathbb{R} \\ (x_1, x_2, \ldots, x_k) \mapsto T(x_1, x_2, \ldots, x_k) \end{array} \right .\end{split}\]

The set of all tensors over \(V^k\) is called \(\mathcal{I}^k(V)\). This is again a vector space. The set \(\mathcal{I}^1(V)\) is just \(V^*\), the dual space of \(V\). A famous example of an n-tensor is the determinant \(\det \in \mathcal{I}^n(\mathbb{R}^n)\).

Definition 37 (Tensor Product)

For \(S \in \mathcal{I}^k(V)\) and \(T \in \mathcal{I}^l(V)\) we define the tensor product of two tensors \(S, T\) as:

\[(S \otimes T)(v_1, \ldots, v_k, w_1, \ldots, w_l) = S(v_1, \ldots, v_k) \, T(w_1, \ldots, w_l)\]

Some formulae:

\[\begin{split}&(S_1 + S_2) \otimes T = S_1 \otimes T + S_2 \otimes T \\ &S \otimes (T_1 + T_2) = S \otimes T_1 + S \otimes T_2 \\ &(aS) \otimes T = S \otimes (aT) = a(S \otimes T) \\ &(S \otimes T) \otimes U = S \otimes (T \otimes U)\end{split}\]

Theorem 44 (Basis for k-Tensors)

Let \(\{v_1, \ldots, v_n\}\) be a basis for \(V\) and \(\{\phi_1, \ldots, \phi_n\}\) a basis for \(V^*\), so:

\[\phi_i(v_j) = \delta_{ij}\]

Then, the set

\[\{\phi_{i_1} \otimes \ldots \otimes \phi_{i_k} \mid 1 \le i_1, \ldots, i_k \le n \}\]

is a basis for \(\mathcal{I}^l(V)\) and we have:

\[\text{dim} \, \mathcal{I}^k(V) = n^k\]

Proof. TODO

Each basis vector \(\phi_{i_1} \otimes \ldots \otimes \phi_{i_k}\) can be thought of as a k-dimensional matrix with exactly one entry equal to one at position \(i_1, \ldots, i_k\) and all others equal to zero. Every k-tensor \(T\) is a sum of these:

\[T = \sum_{i_1, \ldots, i_k = 1, \ldots, n} T_{i_1, \ldots, i_k}\, \phi_{i_1} \otimes \ldots \otimes \phi_{i_k}\]

and we can think of \(T\) as a k-dimensional matrix:

\[\begin{split}T = \begin{bmatrix} &\vdots \\ \cdots &T_{i_1, \ldots, i_k} &\cdots \\ &\vdots \end{bmatrix}_{i_1, \ldots, i_k = 1, \ldots, n}\end{split}\]

This is, in terms of Pytorch or Numpy, a tensor of shape \((n, \ldots, n)\) (\(k\) times). For \(k = 1\) we get a vector and for \(k = 2\) a matrix. In Einstein notation, we can express \(T\) as:

\[T(x_1,\ldots, x_k) = T_{i_1, \ldots, i_k} x_1^{i_1} \cdots x_k^{i_k}\]

For \(S \in \mathcal{I}^k(V)\) and \(T \in \mathcal{I}^l(V)\), we get:

\[S \otimes T = S_{i_1, \ldots, i_k} T_{i_{k+1}, \ldots, i_{k+l}}\]

Definition 38 (Dual Functions)

Let

\[f: V \to W\]

be a linear mapping from \(V\) into some other vector space \(W\). Then the dual of f is a linear transformation \(f^*\) defined by:

\[\begin{split}f^* : \left\{ \begin{array}{lr} \mathcal{I}^k(W) \to \mathcal{I}^k(V) \\ T \mapsto f^*T \end{array} \right .\end{split}\]

where:

\[f^*T(v_1, \ldots, v_k) = T(f(v_1), \ldots, f(v_k))\]

It holds that:

\[f^*(S \otimes T) = f^*S \otimes f^*T\]

Definition 39 (Inner Product)

The inner product on a vector space \(V\) is a 2-tensor, denoted by \(\langle \cdot, \cdot \rangle\), required to be symmetric and positive-definite:

\[\begin{split}&\langle x, y \rangle = \langle y, x \rangle \\ &\langle x, x \rangle > 0 \text{ if } x \ne 0\end{split}\]

The matrix form of the inner product is simply the identity matrix:

\[\begin{split}\langle \cdot, \cdot \rangle = \begin{bmatrix} 1 & &0 \\ &\ddots & \\ 0 & &1 \end{bmatrix}\end{split}\]

\[\langle x, y \rangle = \sum_{i = 1}^n x_i y_i\]

Definition 40 (Alternating Tensors, Alt-Operator)

(a) A k-tensor \(\omega\) is called alternating, if the sign of \(\omega\) is changed by swapping any two variables.

\[\omega(v_1, \ldots, v_i, \ldots, v_j, \ldots v_n) = -\omega(v_1, \ldots, v_j, \ldots, v_i, \ldots v_n)\]

The determinant \(\det\) is famously alternating, the inner product is not.

(b) For \(T \in \mathcal{I}^k(V)\), we define \(\text{Alt}(T) \in \mathcal{I}^k(V)\) through:

\[\text{Alt}(T)(v_1, \ldots, v_k) = \frac{1}{k!} \sum_{\sigma \in S_k} \text{sgn}(\sigma) \, T(v_{\sigma_1}, \ldots, v_{\sigma_k})\]

where \(S_k\) is the set of all permutations of the numbers \(1\) to \(k\).

(c) The set of all alternating tensors in \(\mathcal{I}^k(V)\) is denoted by \(\Lambda^k(V)\). It is a subspace of \(\mathcal{I}^k(V)\).

Theorem 45 (Properties of Alternating Tensors)

a) If \(\omega \in \Lambda^k(V)\), then \(\text{Alt}(\omega) = \omega\)

b) If \(T \in \mathcal{I}^k(V)\), then \(\text{Alt}(T) \in \Lambda^k(V)\)

c) If \(T \in \mathcal{I}^k(V)\), then \(\text{Alt}(T) = \text{Alt}(\text{Alt}(T))\)

Proof. TODO

Definition 41 (Wedge Product)

Let \(\omega \in \Lambda^k(V)\) and \(\eta \in \Lambda^l(V)\). Then, in general, \(\omega \otimes \eta \notin \Lambda^{k+l}(V)\). But the wedge product

\[\omega \wedge \eta = \frac{(k + l)!}{k! \, l!} \text{Alt}(\omega \otimes \eta)\]

is clearly alternating.

Theorem 46 (Properties of the Wedge Product)

Proof. TODO

Remark 13 (Wedge Product of 1-Forms)

Let \(\alpha, \beta \in \Lambda^1(V)\). Then \(\alpha \wedge \beta \in \Lambda^2(V)\) and:

\[\begin{split}\alpha \wedge \beta &= 2 \, \text{Alt}(\alpha \otimes \beta) \\ &= \frac{2}{2}(\alpha \beta - \beta \alpha)\end{split}\]

so:

\[(\alpha \wedge \beta)(u, v) = \alpha(u) \beta(v) - \beta(u) \alpha(v)\]

Theorem 47 (Basis for Alternating Tensors)

Let \(\{v_1, \ldots, v_n\}\) be a basis for \(V\) and \(\{\phi_1, \ldots, \phi_n\}\) a basis for \(V^*\), so:

\[\phi_i(v_j) = \delta_{ij}\]

Then, the set

\[\{\phi_{i_1} \wedge \ldots \wedge \phi_{i_k}\ \mid 1 \le i_1 < \ldots < i_k \le n \}\]

is a basis for \(\Lambda^k(V)\) and we have:

\[\text{dim}\, \Lambda^k(V) = \binom{n}{k}\]

In particular,

\[\text{dim}\, \Lambda^n(\mathbb{R}^n) = 1\]

So, all alternating n-tensors on \(V\) are multiples of any non-zero one, e.g. \(\det\).

Proof. TODO

Theorem 48 (Basis Transformation)

Let \(\{v_1, \ldots, v_n\}\) be a basis for \(V\), \(\omega \in \Lambda^n(V)\), and \(A \in \text{GL}_n(\mathbb{R})\). Then:

\[\omega(A v_1, \ldots, A v_n) = \det A \, \omega(v_1, \ldots, v_n)\]

Proof. TODO

Definition 42 (Basis Orientation)

Let \(\{v_1, \ldots, v_n\}\) be a basis for \(V\), \(A \in \text{GL}_n(\mathbb{R})\), and \(w_i = Av_i\). If \(\det A > 0\), \(v\) and \(w\) are said to have the same orientation, and we clearly have:

\[\omega(v_1, \ldots, v_n) \, \omega(w_1, \ldots, w_n) > 0\]