Home page for accesible maths 3 Inner products 3 Inner products 3.B Positive definiteness

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

3.A Bilinear forms

First we recall how to express the usual notions of length and angle in ${\mathbb{R}}^{n}$ , which is our easiest vector space. They both use the following function $\langle\cdot,\cdot\rangle:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n}\to{\mathbb{R}}$ :

\langle\vec{x},\vec{y}\rangle:=\sum_{i=1}^{n}x_{i}y_{i}.

This is the scalar product that you will have seen in previous modules; it is also called the standard inner product on ${\mathbb{R}}^{n}$ . Instead of writing this product in the way you have seen, $\vec{x}\cdot\vec{y}$ , we have written it as $\langle\vec{x},\vec{y}\rangle$ . So the function $\langle\cdot,\cdot\rangle:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n}\to{\mathbb{R}}$ sends any pair of vectors in ${\mathbb{R}}^{n}$ to a real number. To answer the above questions, we have formulas for the length (also called the norm) of a vector $||\vec{x}||$ , and the cosine of the angle $\theta$ between two vectors $\vec{x}$ and $\vec{y}$ :

$||\vec{x}||:=\sqrt{\langle\vec{x},\vec{x}\rangle}$ (3.1)
$\cos\theta=\frac{\langle\vec{x},\vec{y}\rangle}{||\vec{x}||\cdot||\vec{y}||}$

Furthermore, the distance between two vectors is defined to be $||\vec{x}-\vec{y}||$ . A vector of length 1 is sometimes called a unit vector.

Figure 3.1: You should visualize a vector as an arrow with the tail at the zero vector, and the head at the point which your vector represents. This image shows what is meant by the phrase *“angle between two vectors”*. Image credit: Wikipedia, File:Dot product cosine rule.svg

Exercise 3.2:

Let $\vec{x}=(1,2,3)$ and $\vec{y}=(0,3,4)$ . Calculate the lengths of $\vec{x}$ and $\vec{y}$ , as well as the angle between them (you may need a calculator).

[End of Exercise]

Exercise 3.3:

Find the distance between $(1,2,3,0,-1)$ and $(0,2,1,-2,1)$ in ${\mathbb{R}}^{5}$ .

[End of Exercise]

Definition 3.4:

Let $V$ be a vector space over a field $F$ . A function $\langle\cdot,\cdot\rangle:V\times V\to F$ is called a bilinear form if the following two conditions are satisfied for all $\alpha\in F,$ and all vectors $\vec{x},\vec{y},\vec{z}\in V$ :

i.

(Linearity in first argument) $\langle\alpha\vec{x}+\vec{y},\vec{z}\rangle=\alpha\langle\vec{x}$ , $\vec{z}\rangle+\langle\vec{y},\vec{z}\rangle$
ii.

(Linearity in second argument) $\langle\vec{x},\alpha\vec{y}+\vec{z}\rangle=\alpha\langle\vec{x}$ , $\vec{y}\rangle+\langle\vec{x},\vec{z}\rangle$

It is called bilinear, because it is linear in both arguments.

The formulas 3.1 don’t necessarily make sense for any vector space; for example, in an infinite-dimensional vector space, the expression $\sum x_{i}y_{i}$ probably won’t make sense. So the purpose of the definition of a bilinear form is to clarify the important features of the scalar product that we would like to be true in a more general setting. In the same way that fields are abstractions of ${\mathbb{R}}$ and ${\mathbb{C}}$ , and vector spaces are abstractions of ${\mathbb{R}}^{n}$ , we now have bilinear forms are abstractions of the scalar product.

Example 3.5.

i.

On the vector space $F^{n}$ , the function $\langle\vec{x},\vec{y}\rangle=\sum_{i=1}^{n}x_{i}y_{i}$ is a bilinear form. This follows from the field axioms, such as F11.
ii.

Let $C([0,1])$ be the (infinite-dimensional) vector space of all continuous real-valued functions $[0,1]\to{\mathbb{R}}$ . Then $\langle f,g\rangle:=\int_{0}^{1}f(t)g(t)dt$ is a bilinear form; see Example 3.15. This example and its variations are studied in some third year modules, such as MATH317.
iii.

Let $V$ be the (infinite-dimensional) vector space of real-valued random variables on some fixed probability space. Then the expectation $\langle X,Y\rangle:=\mathbb{E}(XY)$ defines a bilinear form. Similarly, the covariance $\langle X,Y\rangle:=\operatorname{Cov}(X,Y)$ defines a bilinear form. Both of these examples will be studied in MATH230, and used in many statistics modules.

For this Chapter, we will only consider the vector spaces over ${\mathbb{R}}$ , instead of an arbitrary field. Some statements, such as Theorems 3.17 and 3.30 are true for infinite-dimensional real vector spaces (think of Examples 3.5(ii) and (iii)).

We will consider ${\mathbb{R}}^{n}$ as the set of column vectors, also known as $n\times 1$ matrices. Since column vectors are sometimes cumbersome to typeset, for example $\vec{v}=\begin{bmatrix}1\\ 2\\ 3\end{bmatrix}$ , instead we will follow standard conventions, and often write vectors using the matrix transpose; for example, $\vec{v}=\begin{bmatrix}1&2&3\end{bmatrix}^{T}$ , which doesn’t cause unnecessary whitespace.

The next theorem gives a complete description of all bilinear forms on ${\mathbb{R}}^{n}$ .

Theorem 3.6.

Let $\langle\cdot,\cdot\rangle:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n}\to{\mathbb{R}}$ be a bilinear form. Then there is a unique matrix $A\in\operatorname{M}_{{n}}({{\mathbb{R}}})$ such that

\langle\vec{x},\vec{y}\rangle=\vec{x}^{T}A\vec{y}.

Conversely, for any $A\in\operatorname{M}_{{n}}({{\mathbb{R}}})$ , this formula defines a bilinear form, which we call $\langle\cdot,\cdot\rangle_{A}$ .

The proof of this Theorem is given as Exercise 3.46. The bilinear form of the matrix $A$ is the function whose formula is given in Theorem 3.6.

Example 3.7.

If $A=\begin{bmatrix}3&-1\\ -2&4\end{bmatrix}$ then find a formula for $\langle\vec{x},\vec{y}\rangle_{A}$ .

Solution: $\langle\begin{bmatrix}x_{1}\\ x_{2}\end{bmatrix},\begin{bmatrix}y_{1}\\ y_{2}\end{bmatrix}\rangle_{A}=\begin{bmatrix}x_{1}&x_{2}\end{bmatrix}A\begin{% bmatrix}y_{1}\\ y_{2}\end{bmatrix}=3x_{1}y_{1}-x_{1}y_{2}-2x_{2}y_{1}+4x_{2}y_{2}$ .

Exercise 3.8:

Compute $\langle\vec{x},\vec{x}\rangle_{A}$ , $\langle\vec{x},\vec{y}\rangle_{A}$ , $\langle\vec{y},\vec{x}\rangle_{A}$ , $\langle\vec{y},\vec{y}\rangle_{A}$ , in the following cases:

i.

$A:=\begin{bmatrix}1&0\\ 0&1\end{bmatrix}$ , $\vec{x}:=\begin{bmatrix}1\\ 0\end{bmatrix}$ , $\vec{y}:=\begin{bmatrix}1\\ 1\end{bmatrix}$ .
ii.

$A:=\begin{bmatrix}1&2\\ 3&4\end{bmatrix}$ , $\vec{x}:=\begin{bmatrix}1\\ 0\end{bmatrix}$ , $\vec{y}:=\begin{bmatrix}1\\ 1\end{bmatrix}$ .
iii.

$A:=\begin{bmatrix}2&-1\\ -1&2\end{bmatrix}$ , $\vec{x}:=\begin{bmatrix}1\\ 0\end{bmatrix}$ , $\vec{y}:=\begin{bmatrix}1\\ 1\end{bmatrix}$ .

[End of Exercise]

Exercise 3.9:

Find your own example of a bilinear form for which the following three conditions hold:

$\langle\begin{bmatrix}1\\ 0\\ 0\end{bmatrix},\begin{bmatrix}1\\ 0\\ 0\end{bmatrix}\rangle=1$
$\langle\begin{bmatrix}0\\ 1\\ 0\end{bmatrix},\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\rangle=5$
$\langle\begin{bmatrix}0\\ 0\\ 1\end{bmatrix},\begin{bmatrix}1\\ 1\\ -2\end{bmatrix}\rangle=3$

[ Hint: Try to write down a matrix $A$ , as in Theorem 3.6. ]

[End of Exercise]

Exercise 3.10:

Prove that the bilinear form $\langle\cdot,\cdot\rangle_{I_{n}}$ is the same as the standard scalar product. In other words, prove $\vec{x}^{T}I_{n}\vec{y}=\vec{x}\cdot\vec{y}$ , for any vectors $\vec{x},\vec{y}\in{\mathbb{R}}^{n}$ .

[End of Exercise]