Home page for accesible maths 3 Inner products 3.B Positive definiteness 3.D Orthogonality

Style control - access keys in brackets

Font (2 3) - + Letter spacing (4 5) - + Word spacing (6 7) - + Line spacing (8 9) - +

3.C The Cauchy-Schwarz inequality

An inner product space is a pair $(V,\langle\cdot,\cdot\rangle)$ , where $V$ is a real vector space, and $\langle\cdot,\cdot\rangle$ denotes an inner product on $V$ . Unless otherwise stated, the inner product on ${\mathbb{R}}^{n}$ will be the standard scalar product, and this is our most important inner product space.

The second most important inner product space is a vector space of continuous functions, where the inner product (see below) uses integration. For this module, you are only expected to know how to integrate polynomial functions.

Example 3.15.

Prove that the vector space $V$ of continuous real-valued functions on the unit interval $f:[0,1]\to{\mathbb{R}}$ , with the following bilinear form is an inner produce space:

\langle f,g\rangle:=\int_{0}^{1}f(t)g(t)dt.

Solution: Bilinearity follows from elementary properties of integrals, and it is symmetric because $f(t)g(t)=g(t)f(t)$ for all $t$ . Positive definiteness requires us to prove that if $f$ is not the zero function, then $\int_{0}^{1}[f(t)]^{2}dt>0$ . This is an exercise in analysis, which may be omitted from this module; but we include the proof for the benefit of those students taking MATH210. Clearly $(f(t))^{2}\geq 0$ , and for some $t\in[0,1]$ we have $(f(t))^{2}>0$ . By the definition of continuity, there is a small open interval of size $\delta$ around $t$ on which $(f(t))^{2}>\epsilon$ , for some $\epsilon,\delta>0$ . Therefore the area under the curve contains a rectangle of length $\delta$ and height $\epsilon$ . Hence, $\langle f,f\rangle\geq\delta\epsilon>0$ , as required.

Exercise 3.16:

Let $V$ be the inner product space from Example 3.15.

Then $f(t)=t$ and $g(t)=t^{2}$ are both in $V$ .

i.

Calculate the norms of $f$ and $g$ , and also $\langle f,g\rangle$ .
ii.

What is the distance between these two functions (i.e. what is $||f-g||$ )?
iii.

What is the “angle” between these two functions (from Equation 3.1)?

[End of Exercise]

In Exercise 3.16, you were only able to calculate the angle from the formula 3.1 because $-1\leq\frac{\langle f,g\rangle}{||f||\cdot||g||}\leq 1$ (otherwise, the inverse function of cosine is not defined). So you should be asking: “Is this always true, or did we just get lucky?”

The Cauchy-Schwarz inequality shows that it is always true, as long as we have an inner product (rather than just a bilinear form). You may recall this result from MATH115, where it was stated for the vector space ${\mathbb{R}}^{n}$ with the standard inner product; in that form is was proved by Cauchy in the 1820’s. Then, in the 1880’s, Schwarz proved the following more general version (which allows infinite dimensional spaces).

Theorem 3.17 (Cauchy-Schwarz inequality).

Let $V$ be an inner product space. Then

|\langle\vec{x},\vec{y}\rangle|\leq||\vec{x}||\cdot||\vec{y}||

for any $\vec{x},\vec{y}\in V$ .

Proof.

If $||\vec{x}||=0$ then $\vec{x}=0$ (by positive definiteness), in which case the inequality obviously holds. So assume $||\vec{x}||>0$ . Define the vector

\vec{z}:=\vec{y}-\frac{\langle\vec{x},\vec{y}\rangle}{||\vec{x}||^{2}}\vec{x}% \in V.

By Exercise 3.20, and positive definiteness, $||\vec{y}||^{2}-\frac{|\langle\vec{x},\vec{y}\rangle|^{2}}{||\vec{x}||^{2}}\geq 0$ . Multiplying both sides of this inequality by (the positive number) $||\vec{x}||^{2}$ , rearranging, and taking square roots, we obtain the result. ∎

[Aside: In MATH317, these notions will be generalized to include $F={\mathbb{C}}$ , where complex inner product spaces are not defined to be symmetric, but instead they obey: $\langle x,y\rangle=\overline{\langle y,x\rangle}$ . Also, as a historical aside, the reason Cauchy didn’t find the above proof was because the abstract notion of “inner product space” hadn’t been invented yet; neither had abstract “vector spaces”, or even “fields” for that matter.]

Exercise 3.18:

Let $A=\begin{bmatrix}3&1\\ 1&1\end{bmatrix}$ . Verify the Cauchy-Schwarz inequality for the inner product $\langle,\rangle_{A}$ on $\mathbb{R}^{2}$ , for the vectors $\vec{x}=(1,1)$ and $\vec{y}=(1,-1)$ .

[End of Exercise]

Exercise 3.19:

Choose your own 2 vectors in ${\mathbb{R}}^{5}$ , and verify that the Cauchy-Schwarz inequality holds for them, using the standard inner product.

[End of Exercise]

Exercise 3.20:

With the notation in the proof of Theorem 3.17, prove that

||\vec{z}||^{2}=||\vec{y}||^{2}-\frac{|\langle\vec{x},\vec{y}\rangle|^{2}}{||% \vec{x}||^{2}}.

[End of Exercise]