Chapter 8 Dual spaces and tensors

Let

$$\mathcal{A}=\{\mathbf{a}_{1},\mathbf{a}_{2},\ldots,\mathbf{a}_{n}\},\qquad\mathcal{B}=\{\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{n}\}$$

be two bases in $V$, and let $[L]_{\mathcal{A}}=[L]_{\mathcal{S},\mathcal{A}}$ and $[L]_{\mathcal{B}}=[L]_{\mathcal{S},\mathcal{B}}$ be the matrices of $L$ in the bases $\mathcal{A}$ and $\mathcal{B}$ respectively (we suppose that the basis in the target space of scalars is always the standard one, so we can skip the subscript $\mathcal{S}$ in the notation). Then recalling the change of coordinate rule from Section 2.8.4 in Chapter 2 we get that

$$[L]_{\mathcal{B}}=[L]_{\mathcal{A}}[I]_{\mathcal{A},\mathcal{B}}.$$

Recall that for a vector $\mathbf{v}\in V$ its coordinates in different bases are related by the formula

$$[\mathbf{v}]_{\mathcal{B}}=[I]_{\mathcal{B},\mathcal{A}}[\mathbf{v}]_{\mathcal{A}},$$

and that

$$[I]_{\mathcal{A},\mathcal{B}}=[I]_{\mathcal{B},\mathcal{A}}^{-1}.$$

If we denote $S:=[I]_{\mathcal{B},\mathcal{A}}$, so $[\mathbf{v}]_{\mathcal{B}}=S[\mathbf{v}]_{\mathcal{A}}$, the entries of the vectors $[L]_{\mathcal{B}}^{T}$ and $[L]_{\mathcal{A}}^{T}$ are related by the formula

(8.1.1)

$$[L]_{\mathcal{B}}^{T}=(S^{-1})^{T}[L]_{\mathcal{A}}^{T}$$

(since we usually represent a vector as a column of its coordinates, we use $[L]_{\mathcal{A}}^{T}$ and $[L]_{\mathcal{B}}^{T}$ instead of $[L]_{\mathcal{A}}$ and $[L]_{\mathcal{B}}$)

Saying it in words

If $S$ is the change of coordinate matrix (from old coordinates to the new ones) in $X$, then the change of coordinate matrix in the dual space $X^{\prime}$ is $(S^{-1})^{T}$.

So, the dual space $V^{\prime}$ of $V$ while isomorphic to $V$ is indeed a different object: the difference is in how the coordinates in $V$ and $V^{\prime}$ change when one changes the basis in $V$.

The dual system can be used for computing the coordinates in the basis $\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{n}$. Let $\mathbf{b}^{\prime}_{1},\mathbf{b}^{\prime}_{2},\ldots,\mathbf{b}^{\prime}_{n}$ be the biorthogonal system to $\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{n}$, and let $\mathbf{v}=\sum_{k}\alpha_{k}\mathbf{b}_{k}$. Then, as it was shown before

$$\mathbf{b}^{\prime}_{j}(\mathbf{v})=\mathbf{b}_{j}\left(\sum_{k}\alpha_{k}\mathbf{b}_{k}\right)=\sum_{k}\alpha_{k}\mathbf{b}_{j}(\mathbf{b}_{k})=\alpha_{j}\mathbf{b}_{j}^{\prime}(\mathbf{b}_{j})=\alpha_{j},$$

so $\alpha_{k}=\mathbf{b}_{k}^{\prime}(\mathbf{v})$. Then we can write

(8.1.3)

$$\mathbf{v}=\sum_{k}\mathbf{b}_{k}^{\prime}(\mathbf{v})\mathbf{b}_{k}.$$

In other words,

The $k$th coordinate of a vector $\mathbf{v}$ in a basis $\mathcal{B}=\{\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{n}\}$ is $\mathbf{b}_{k}^{\prime}(\mathbf{v})\}$, where $\mathcal{B}^{\prime}=\{\mathbf{b}^{\prime}_{1},\mathbf{b}^{\prime}_{2},\ldots,\mathbf{b}^{\prime}_{n}\}$ is the dual basis.

This formula is called (a baby version of) the abstract non-orthogonal Fourier decomposition of $\mathbf{v}$ (in the basis $\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{n}$). The reason for this name will be clear later in Section 8.2.3.

The next example is more interesting. Let us consider the space $\mathbb{P}_{n}$ of polynomials of degree at most $n$. As we know, the powers $\{\mathbf{e}_{k}\}_{k=0}^{n}$, $\mathbf{e}(t)=t^{n}$ form the standard basis in this space. What is the dual to this basis?

The answer might be tricky to guess, but it is very easy to check when you know it. Namely, consider the linear functionals $\mathbf{e}^{\prime}_{k}\in(\mathbb{P}_{n})^{\prime}$, $k=0,1,\ldots,n$, acting on polynomials as follows:

$$\mathbf{e}^{\prime}_{k}(p):=\frac{1}{k!}\frac{d^{k}}{dt^{k}}p(t)\bigm{|}_{t=0}=\frac{1}{k!}p^{(k)}(0);$$

here we use the usual agreement that $0!=1$ and $d^{0}f/dt^{0}=f$.

Since

$$\frac{d^{k}}{dt^{k}}t^{j}=\left\{\begin{array}[]{ll}j(j-1)\ldots(j-k+1)t^{j-k},&k\leq j\\ 0&k>j\end{array}\right.$$

we can easily see that the system $\{\mathbf{e}^{\prime}_{k}\}_{k=0}^{n}$ is the dual to the system of powers $\{\mathbf{e}_{k}\}_{k=0}^{n}$.

Applying (8.1.3) to the above system $\{\mathbf{e}_{k}\}_{k=0}^{n}$ and its dual we get that any polynomial $p$ of degree at most $n$ can be represented as

(8.1.4)

$$p(t)=\sum_{k=0}^{n}\frac{p^{(k)}(0)}{k!}t^{k}$$

This formula is well-known in Calculus as the Taylor formula for polynomials. More precisely, this is a particular case of the Taylor formula, the so-called Maclaurin formula. The general Taylor formula

$$p(t)=\sum_{k=0}^{n}\frac{p^{(k)}(a)}{k!}(t-a)^{k}$$

can be obtained from (8.1.4) by applying it to the polynomial $p(\tau-a)$ and then denoting $t:=\tau-a$. It also can be obtained by considering powers $(t-a)^{k}$, $k=0,1,\ldots,n$ and finding the dual system the same way we did it for $t^{k}$.²²2 Note, that the general Taylor formula says more than the formula for polynomials obtained here: it says that any $n$ times differentiable function can be approximated near the point $a$ by its Taylor polynomial. Moreover, if the function is $n+1$ times differentiable, it allows us to estimate the error. The above formula for polynomials serves as a motivation and a starting point for the general case

Our next example deals with the so-called Lagrange interpolating formula. Let $a_{1},a_{2},\ldots,a_{n+1}$ be distinct points (in $\mathbb{R}$ or $\mathbb{C}$), and let $\mathbb{P}_{n}$ be the space of polynomials of degree at most $n$. Define functionals $\mathbf{f}_{k}\in\mathbb{P}_{n}^{\prime}$ by

$$\mathbf{f}_{k}(p)=p(a_{k})\qquad\forall p\in\mathbb{P}_{n}.$$

What is the dual of this system of functionals? Note, that while it is not hard to show that the functionals $\mathbf{f}_{k}$ are linearly independent, and so (since $\dim(\mathbb{P}_{n})^{\prime}=\dim\mathbb{P}_{n}=n+1$) form a basis in $(\mathbb{P}_{n})^{\prime}$, we do not need that. We will construct the dual system directly, and then will be able to see that the system $\mathbf{f}_{1},\mathbf{f}_{2},\ldots,\mathbf{f}_{n+1}$ is indeed a basis.

Namely, let us define the polynomials $p_{k}$, $k=1,2,\ldots,n+1$ as

$$p_{k}(t)={\prod_{j:j\neq k}(t-a_{j})}\Bigm{/}{\prod_{j:j\neq k}(a_{k}-a_{j})}$$

where $j$ in the products runs from $1$ to $n+1$. Clearly $p_{k}(a_{k})=1$ and $p_{k}(a_{j})=0$ if $j\neq k$, so indeed the system $p_{1},p_{2},\ldots,p_{n+1}$ is dual to the system $\mathbf{f}_{1},\mathbf{f}_{2},\ldots,\mathbf{f}_{n+1}$.

There is a little detail here, since the notion of a dual system was defined only for a basis, and we did not prove that either of the systems is one. But one can immediately see that the system $p_{1},p_{2},\ldots,p_{n+1}$ is linearly independent (can you explain why?), and since it contains $n+1=\dim\mathbb{P}_{n}$ vectors, it is a basis. Therefore, the system of functionals $\mathbf{f}_{1},\mathbf{f}_{2},\ldots,\mathbf{f}_{n+1}$ is also a basis in the dual space $(\mathbb{P}_{n})^{\prime}$.

Applying formula (8.1.3) to the above example one can see that the unique polynomial $p$, $\deg p\leq n$ satisfying

(8.1.5)

$$p(a_{k})=y_{k},\qquad k=1,2,\ldots,n+1$$

can be reconstructed by the formula

(8.1.6)

$$p(t)=\sum_{k=1}^{n+1}y_{k}p_{k}(t).$$

This formula is well-known in mathematics as the “Lagrange interpolation formula”.

Let us first consider the case when $X=\mathbb{F}^{n}$, $Y=\mathbb{F}^{m}$ ($\mathbb{F}$ here is, as usual, either $\mathbb{R}$ or $\mathbb{C}$, but everything works for the case of arbitrary fields)

As usual, we identify a vector $\mathbf{v}$ in $\mathbb{F}^{n}$ with the column of its coordinates, and a linear transformation with its matrix (in the standard basis).

The dual of $\mathbb{F}^{n}$ is, as it was discussed above, the space of rows of size $n$, so we can identify its with $\mathbb{F}^{n}$. Again, we will treat an element of $(\mathbb{F}^{n})^{\prime}$ as a column vector of its coordinates.

Under these agreements we have for $\mathbf{x}\in\mathbb{F}^{n}$ and $\mathbf{x}^{\prime}\in(\mathbb{F}^{n})^{\prime}$

$$\mathbf{x}^{\prime}(\mathbf{x})=\langle\mathbf{x},\mathbf{x}^{\prime}\rangle=(\mathbf{x}^{\prime})^{T}\mathbf{x}$$

where the right side is the product of matrices (or a row and a column). Then, for arbitrary $\mathbf{x}\in X=\mathbb{F}^{n}$ and $\mathbf{y}^{\prime}\in Y^{\prime}=(\mathbb{F}^{m})^{\prime}$

$$\langle A\mathbf{x},\mathbf{y}^{\prime}\rangle=(\mathbf{y}^{\prime})^{T}A\mathbf{x}=(A^{T}\mathbf{y}^{\prime})^{T}\mathbf{x}=\langle\mathbf{x},A^{T}\mathbf{y}^{\prime}\rangle$$

(the expressions in the middle are products of matrices).

So we have proved that the adjoint transformation exists. let us show that it is unique. Assume that for some transformation $B$

$$\langle A\mathbf{x},\mathbf{y}^{\prime}\rangle=\langle\mathbf{x},B\mathbf{y}^{\prime}\rangle\qquad\forall\mathbf{x}\in\mathbb{F}^{n},\forall\mathbf{y}^{\prime}\in(\mathbb{F}^{m})^{\prime}.$$

That means that for arbitrary

$$\langle\mathbf{x},(A^{T}-B)\mathbf{y}^{\prime}\rangle=0,\qquad\forall\mathbf{x}\in\mathbb{F}^{n},\forall\mathbf{y}^{\prime}\in(\mathbb{F}^{m})^{\prime}$$

By taking for $\mathbf{x}$ and $\mathbf{y}^{\prime}$ the vectors from the standard bases in $\mathbb{F}^{n}$ and $(\mathbb{F}^{m})^{\prime}\cong\mathbb{F}^{m}$ respectively we get that the matrices $B$ and $A^{T}$ coincide. ∎

So, for $X=\mathbb{F}^{n}$, $Y=\mathbb{F}^{m}$

The dual transformation $A^{\prime}$ exists, and is unique. Moreover, its matrix (in the standard bases) equals $A^{T}$ (the transpose of the matrix of $A$)

Now, let us consider the general case. In fact, we do not need to do much, since everything can be reduced to the case of spaces $\mathbb{F}^{n}$.

Namely, let us fix bases $\mathcal{A}=\mathbf{a}_{1},\mathbf{a}_{2},\ldots,\mathbf{a}_{n}$ in $X$, and $\mathcal{B}=\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{m}$ in $Y$, and let $\mathcal{A}^{\prime}=\mathbf{a}^{\prime}_{1},\mathbf{a}^{\prime}_{2},\ldots,\mathbf{a}^{\prime}_{n}$ and $\mathcal{B}=\mathbf{b}^{\prime}_{1},\mathbf{b}^{\prime}_{2},\ldots,\mathbf{b}^{\prime}_{m}$ be their dual bases (in $X^{\prime}$ and $Y^{\prime}$ respectively). For a vector $\mathbf{v}$ (from a space or its dual) we as usual denote by $[\mathbf{v}]_{\mathcal{B}}$ the column of its coordinates in the basis $\mathcal{B}$. Then

$$\langle\mathbf{x},\mathbf{x}^{\prime}\rangle=([\mathbf{x}^{\prime}]_{\mathcal{A}^{\prime}})^{T}[\mathbf{x}]_{\mathcal{A}},\qquad\forall\mathbf{x}\in X\quad\forall\mathbf{x}^{\prime}\in X^{\prime},$$

i.e. instead of working with $\mathbf{x}\in X$ and $\mathbf{x}^{\prime}\in X^{\prime}$ we can work with columns their coordinates (in the dual bases $\mathcal{A}$ and $\mathcal{A}^{\prime}$ respectively) absolutely the same way we do in in the case of $\mathbb{F}^{n}$. Of course, the same works for $Y$, so working with columns of coordinates and then translating everything back to the abstract setting we get that the dual transformation exists in unique in this case as well.

Moreover, using the fact (which we just proved) that for $A:\mathbb{F}^{n}\to\mathbb{F}^{m}$ the matrix of $A^{\prime}$ is $A^{T}$ we get

(8.3.1)

$$[A^{\prime}]_{\mathcal{A}^{\prime},\mathcal{B}^{\prime}}=([A]_{\mathcal{B},\mathcal{A}})^{T},$$

or in plain English

The matrix of the dual transformation in the dual basis is the transpose of the matrix of the transformation in the original bases.

Let us now present another, more “high brow” way of defining the dual of a linear transformation. Namely, for $\mathbf{x}\in X$, $\mathbf{y}^{\prime}\in Y$ let us fix for a moment $\mathbf{y}^{\prime}$ and treat the expression $\langle A\mathbf{x},\mathbf{y}^{\prime}\rangle=\mathbf{y}^{\prime}(A\mathbf{x})$ as a function of $\mathbf{x}$. It is easy to see that this is a composition of two linear transformations (which ones?) and so it is a linear function of $\mathbf{x}$, i.e. a linear functional on $X$, i.e. an element of $X^{\prime}$.

Let us call this linear functional $B(\mathbf{y}^{\prime})$ to emphasize the fact that it depends on $\mathbf{y}^{\prime}$. Since we can do this for every $\mathbf{y}^{\prime}\in Y^{\prime}$, we can define the transformation $B:Y^{\prime}\to X^{\prime}$ such that

$$\langle A\mathbf{x},\mathbf{y}^{\prime}\rangle=\langle\mathbf{x},B(\mathbf{y}^{\prime})\rangle$$

Our next step is to show that $B$ is a linear transformation. Note, that since the transformation $B$ was defined in rather indirect way, we cannot see immediately from the definition that it is linear. To show the linearity of $B$ let us take $\mathbf{y}^{\prime}_{1},\mathbf{y}^{\prime}_{2}\in Y^{\prime}$. For $\mathbf{x}\in X$

	$\displaystyle\langle\mathbf{x},B(\alpha\mathbf{y}^{\prime}_{1}+\beta\mathbf{y}^{\prime}_{2})\rangle$	$\displaystyle=\langle A\mathbf{x},\alpha\mathbf{y}^{\prime}_{1}+\beta\mathbf{y}^{\prime}_{2}\rangle\qquad\text{by the definition of }B$
		$\displaystyle=\alpha\langle A\mathbf{x},\mathbf{y}^{\prime}_{1}\rangle+\beta\langle A\mathbf{x},\mathbf{y}^{\prime}_{2}\rangle\qquad\text{by linearity}$
		$\displaystyle=\alpha\langle\mathbf{x},B(\mathbf{y}^{\prime}_{1})\rangle+\beta\langle\mathbf{x},B(\mathbf{y}^{\prime}_{2})\rangle\qquad\text{by the definition of }B$
		$\displaystyle=\langle\mathbf{x},\alpha B(\mathbf{y}^{\prime}_{1})+\beta B(\mathbf{y}^{\prime}_{2})\rangle\qquad\text{by linearity}$

Since this identity is true for all $\mathbf{x}$, we conclude that $B(\alpha\mathbf{y}^{\prime}_{1}+\beta\mathbf{y}^{\prime}_{2})=\alpha B(\mathbf{y}^{\prime}_{1})+\beta B(\mathbf{y}^{\prime}_{2})$, i.e. that $B$ is linear.

The main advantage of this approach that it does not require a basis, so it can be (and is) used in the infinite-dimensional situation. However, the proof that we presented above in Sections 8.3.1, 8.3.1 gives a constructive way to compute the dual transformation, so we used that proof instead of more general coordinate-free one.