diff --git a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md index 620c680..d8fb9e8 100644 --- a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md +++ b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md @@ -372,4 +372,80 @@ move for some iterations. # Exercise 3 +We first show that the lemma holds for $\tau \in [0,1]$. Since + +$$\|\tilde{p}(\tau)\| = \|\tau p^U\| = \tau \|p^U\| \text{ for } \tau \in [0,1]$$ + +Then the norm of the step $\tilde{p}$ clearly increases monotonically as $\tau$ +increases. For the second criterion, we compute the quadratic model for a +generic $\tau \in [0,1]$: + +$$m(\tilde{p}(\tau)) = f - \frac{\tau^2 \|g\|^2}{g^TBg} - \frac12 \frac{\tau^2 +\|g\|^2}{(g^TBg)^2} g^TBg = f - \frac12 \frac{\tau^2 \|g\|^2}{g^TBg}$$ + +$g^T B g > 0$ since we assume $B$ is positive definite, therefore the entire +term in function +of $\tau^2$ is negative and thus the model for an increasing $\tau \in [0,1]$ +decreases monotonically (to be precise quadratically). + +Now we show that the monotonicity claims hold also for $\tau \in [1,2]$. We +define a function $h(\alpha)$ (where $\alpha = \tau - 1$) with same monotonicity +as $\|\tilde{p}(\tau)\|$ and we show that this function monotonically increases: + +$$h(\alpha) = \frac12 \|\tilde{p}(1 - \alpha)\|^2 = \frac12 \|p^U + \alpha(p^B - +p^U)\|^2 = \frac12 \|p^U\|^2 + \frac12 \alpha^2 \|p^B - p^U\|^2 + \alpha (p^U)^T +(p^B - p^U)$$ + +We now take the derivative of $h(\alpha)$ and we show it is always positive, +i.e. that $h(\alpha)$ has always positive gradient and thus that is it +monotonically increasing w.r.t. $\alpha$: + +$$h'(\alpha) = \alpha \|p^B - p^U\|^2 + (p^U)^T (p^B - p^U) \geq (p^U)^T (p^B - p^U) += \frac{g^Tg}{g^TBg}g^T\left(- \frac{g^Tg}{g^TBg}g + B^{-1}g\right) =$$$$= \|g\|^2 +\frac{g^TB^{-1}g}{g^TBg}\left(1 - \frac{\|g\|^2}{(g^TBg)(g^TB^{-1}g)}\right) $$ + +Since we know $B$ is symmetric and positive definite, then $B^{-1}$ is as well. +Therefore, we know that the term outside of the parenthesis is always positive +or 0. Therefore, we now only need to show that: + +$$\frac{\|g\|^2}{(g^TBg)(g^TB^{-1}g)} \leq 1 \Leftrightarrow \|g\|^2 \leq +(g^TBg)(g^TB^{-1}g)$$ + +since both factors in the denominator are positive for what we shown before. + +We now define a inner product space $\forall a, b \in R^N, \; {\langle a, +b\rangle}_B = a^T B b$. We now prove that this is indeed a linear product space +by proving all properties of such space: + +- **Linearity w.r.t. the first argument:** + + ${\langle x, y \rangle}_B + {\langle z, + y \rangle}_B = x^TBy + z^TBy = (x + z)^TBy = {\langle (x + z), y \rangle}_B$; + +- **Symmetry:** + + ${\langle x, y \rangle}_B = x^T B y = (x^T B y)^T = y^TB^Tx$, and + since $B$ is symmetric, $y^TB^Tx = y^TBx = {\langle y,x \rangle}_B$; + +- **Positive definiteness:** + + ${\langle x, x \rangle_B} = x^T B x > 0$ is true since B is positive definite. + +Since ${\langle x, y \rangle}_B$ is indeed a linear product space, then: + +$${\langle g, B^{-1} g \rangle}_B \leq {\langle g, g \rangle}_B, {\langle B^{-1} +g, B^{-1} g \rangle}_B$$ + +holds according to the Cauchy-Schwartz inequality. Now, if we expand each inner +product we obtain: + +$$g^T B B^{-1} g \leq (g^T B g) (g^T (B^{-1})^T B B^{-1} g)$$ + +Which, since $B$ is symmetric, in turn is equivalent to writing: + +$$g^T g \leq (g^TBg) (g^T B^{-1} g)$$ + +which is what we needed to show to prove that the first monotonicity constraint +holds for $\tau \in [1,2]$. + **TBD** diff --git a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf index 3205676..dd9522d 100644 Binary files a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf and b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf differ