diff --git a/Claudio_Maggioni_4/Claudio_Maggioni_4.md b/Claudio_Maggioni_4/Claudio_Maggioni_4.md new file mode 100644 index 0000000..55686a1 --- /dev/null +++ b/Claudio_Maggioni_4/Claudio_Maggioni_4.md @@ -0,0 +1,100 @@ + + +--- +title: Midterm -- Optimization Methods +author: Claudio Maggioni +header-includes: +- \usepackage{amsmath} +- \usepackage{hyperref} +- \usepackage[utf8]{inputenc} +- \usepackage[margin=2.5cm]{geometry} +- \usepackage[ruled,vlined]{algorithm2e} +- \usepackage{float} +- \floatplacement{figure}{H} +- \hypersetup{colorlinks=true,linkcolor=blue} + +--- +\maketitle + +# Exercise 1 + +## Exercise 1.1 + +The lagrangian is the following: + +$$L(X,\lambda) = f(X) - \lambda \left(c(x) - 0\right) = -3x^2 + y^2 + 2x^2 + +2(x+y+z) - \lambda x^2 - \lambda y^2 -\lambda z^2 + \lambda =$$$$= (-3 -\lambda)x^2 + (1- +\lambda)y^2 + (2-\lambda)z^2 + 2 (x+y+z) + \lambda$$ + +The KKT conditions are the following: + +First we have the condition on the partial derivatives of the Lagrangian w.r.t. +$X$: + +$$\nabla_X L(X,\lambda) = \begin{bmatrix}(-3-\lambda)x^* + 1\\(1-\lambda)y^* + +1\\(2-\lambda)z^* + 1\end{bmatrix} = 0 \Leftrightarrow +\begin{bmatrix}x^*\\y^*\\z^*\end{bmatrix} = +\begin{bmatrix}\frac1{3+\lambda}\\-\frac1{1-\lambda}\\-\frac{1}{2-\lambda}\end{bmatrix}$$ + +Then we have the conditions on the equality constraint: + +$$c(X) = {x^*}^2 + {y^*}^2 + {z^*}^2 - 1 = 0 \Leftrightarrow \|X^*\| = 1$$ + +$$\lambda^* c(X^*) = 0 \Leftarrow c(X^*) = 0 \text{ which is true if the above +condition is true.}$$ + +Since we have no inequality constraints, we don't need to apply the KKT +conditions realated to inequality constraints. + +## Exercise 1.2 + +To find feasible solutions to the problem, we apply the KKT conditions. Since we +have a way to derive $X^*$ from $\lambda^*$ thanks to the first KKT condition, +we try to find the values of $\lambda$ that satisfies the second KKT condition: + +$$c(x) = \left( \frac{1}{3+\lambda} \right)^2 + \left( -\frac{1}{1-\lambda} \right)^2 + +\left(-\frac{1}{2-\lambda}\right)^2 - 1 = +\frac{1}{(3+\lambda)^2} + \frac{1}{(1-\lambda)^2} + \frac{1}{(2-\lambda)^2} - 1 =$$$$= +\frac{(1-\lambda)^2(2-\lambda)^2 + (3+\lambda)^2(2-\lambda)^2 + +(3+\lambda)^2 +(1-\lambda)^2 - (3+\lambda)^2 (1-\lambda)^2 (2-\lambda)^2}{(3+\lambda)^2 +(1-\lambda)^2 (2-\lambda)^2} = 0 +\Leftrightarrow$$$$\Leftrightarrow +(1-\lambda)^2(2-\lambda)^2 + (3+\lambda)^2(2-\lambda)^2 + +(3+\lambda)^2 +(1-\lambda)^2 - (3+\lambda)^2 (1-\lambda)^2 (2-\lambda)^2 = 0 +\Leftrightarrow$$$$\Leftrightarrow +(\lambda^4 - 6\lambda^3 + 13\lambda^2 - 12\lambda + 16) + +(\lambda^4 + 2\lambda^3 - 11\lambda^2 - 12\lambda + 36) + +(\lambda^4 + 4\lambda^3 - 2\lambda^2 - 12\lambda + 9)$$$$ ++ (-\lambda^5 -14\lambda^4 +12\lambda^3 +49\lambda^2 -84\lambda + 36) = $$$$ +=-\lambda^5 +17\lambda^4 -12\lambda^3 -49\lambda^2 +48\lambda +13 = 0 +\Leftrightarrow $$$$ \Leftrightarrow +\lambda = \lambda_1 \approx -0.224 \lor +\lambda = \lambda_2 \approx -1.892 \lor +\lambda = \lambda_3 \approx 3.149 \lor +\lambda = \lambda_4 \approx -4.035$$ + +We then compute $X$ from each solution and evaluate the objective each time: + +$$X = \begin{bmatrix}\frac1{3+\lambda}\\-\frac1{1-\lambda}\\ +-\frac{1}{2-\lambda}\end{bmatrix} +\Leftrightarrow$$$$\Leftrightarrow +X = X_1 \approx \begin{bmatrix}0.360\\-0.817\\-0.450\end{bmatrix} \lor +X = X_2 \approx \begin{bmatrix}0.902\\-0.346\\-0.257\end{bmatrix} \lor +X = X_3 \approx \begin{bmatrix}0.163\\0.465\\0.870\end{bmatrix} \lor +X = X_4 \approx \begin{bmatrix}-0.966\\-0.199\\-0.166\end{bmatrix}$$ + +$$f(X_1) = -1.1304 \;\; f(X_2) = -1.59219 \;\; f(X_3) = 4.64728 \;\; f(X_4) = +-5.36549$$ + +We therefore choose $(\lambda_4, X_4)$ since $f(X_4)$ is the smallest objective +value out of all the feasible points. Therefore, the solution to the +minimization problem is: + +$$X \approx \begin{bmatrix}-0.966\\-0.199\\-0.166\end{bmatrix}$$ + + + + + diff --git a/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf b/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf new file mode 100644 index 0000000..8ed2a32 Binary files /dev/null and b/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf differ diff --git a/Claudio_Maggioni_4/gsppn.m b/Claudio_Maggioni_4/gsppn.m new file mode 100644 index 0000000..de2bb5c --- /dev/null +++ b/Claudio_Maggioni_4/gsppn.m @@ -0,0 +1,31 @@ +clc + +X = [1/(3+l); (-1)/(1-l); (-1)/(2-l)] + +syms l +a1 = (3+l)^2; +a2 = (1-l)^2; +a3 = (2-l)^2; + +t1 = a2*a3 +t2 = a1*a3 +t3 = a1*a2 +t4 = a1*a2*a3 + +c1 = fliplr(coeffs(t1)) +c2 = fliplr(coeffs(t2)) +c3 = fliplr(coeffs(t3)) +c4 = fliplr(coeffs(t4)) + +ctot = fliplr(coeffs(t1+t2+t3-t4)) + +sol = double(solve(t1+t2+t3-t4==0, l, 'Real', true)) + +for i=1:size(sol, 1) + Xi = double(subs(X,l,sol(i))) + Li = sol(i); + Ci = norm(Xi, 2)^2 - 1; + Yi = sum(Xi .* Xi .* [-3;1;2]) + sum(2 * Xi); + fprintf("lambda=%.03f ci=%g y=%g", Li, Ci, Yi); +end + diff --git a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md index 513eaeb..be75284 100644 --- a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md +++ b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.md @@ -281,11 +281,20 @@ since the model quality is good. ### (f) Does the energy decrease monotonically when Trust Region method is employed? Justify your answer. -In the trust region method the energy of the iterates does not always decrease -monotonically. This is due to the fact that the algorithm could actively reject -a step if the performance measure factor $\rho_k$ is less that a given constant -$\eta$. In this case, the new iterate is equal to the old one, no step is taken -and thus the energy norm does not decrease but stays the same. +When using the trust region method, the energy of the iterates decreases +monotonically. This is true because by construction the algorithm either makes +the next iterate equal to the current one (i.e. when the performance measure +$\rho_k$ is too poor to accept a step) or applies a linear, quadratic, or +blended descending step to the current iterate. + +When a step is taken, the step +by definition should be a solution (or a close approximation of such solution) +of the energy minimization problem inside the trust region itself. Therefore, +the step cannot lead to a point that has higher energy than the one from the +current iterate. + +Therefore, the energy either stays constant or decreases at every single +iteration, and therefore the energy decreases monotonically. ## Point 2 diff --git a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf index 2d1b093..aefba56 100644 Binary files a/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf and b/Claudio_Maggioni_midterm/Claudio_Maggioni_midterm.pdf differ