OM/Claudio_Maggioni_4/Claudio_Maggioni_4.md

---

title: Homework 4 -- Optimization Methods author: Claudio Maggioni header-includes:

• \usepackage{amsmath}
• \usepackage{hyperref}
• \usepackage[utf8]{inputenc}
• \usepackage[margin=2.5cm]{geometry}
• \usepackage[ruled,vlined]{algorithm2e}
• \usepackage{float}
• \floatplacement{figure}{H}
• \hypersetup{colorlinks=true,linkcolor=blue}

---

\maketitle

Exercise 1

Exercise 1.1

The lagrangian is the following:

$$L(X,\lambda) = f(X) - \lambda \left(c(x) - 0\right) = -3x^2 + y^2 + 2x^2 + 2(x+y+z) - \lambda x^2 - \lambda y^2 -\lambda z^2 + \lambda =$$$$= (-3 -\lambda)x^2 + (1- \lambda)y^2 + (2-\lambda)z^2 + 2 (x+y+z) + \lambda$$

The KKT conditions are the following:

First we have the condition on the partial derivatives of the Lagrangian w.r.t. X:

$$\nabla_X L(X,\lambda) = \begin{bmatrix}(-3-\lambda)x^* + 1\(1-\lambda)y^* + 1\(2-\lambda)z^* + 1\end{bmatrix} = 0 \Leftrightarrow \begin{bmatrix}x^\y^\z^*\end{bmatrix} = \begin{bmatrix}\frac1{3+\lambda}\-\frac1{1-\lambda}\-\frac{1}{2-\lambda}\end{bmatrix}

Then we have the conditions on the equality constraint:

c(X) = {x^*}^2 + {y^*}^2 + {z^*}^2 - 1 = 0 \Leftrightarrow \|X^*\| = 1

$$\lambda^* c(X^) = 0 \Leftarrow c(X^) = 0 \text{ which is true if the above condition is true.}

Since we have no inequality constraints, we don't need to apply the KKT conditions realated to inequality constraints.

Exercise 1.2

To find feasible solutions to the problem, we apply the KKT conditions. Since we have a way to derive X^* from \lambda^* thanks to the first KKT condition, we try to find the values of \lambda that satisfies the second KKT condition:

$$c(x) = \left( \frac{1}{3+\lambda} \right)^2 + \left( -\frac{1}{1-\lambda} \right)^2 + \left(-\frac{1}{2-\lambda}\right)^2 - 1 = \frac{1}{(3+\lambda)^2} + \frac{1}{(1-\lambda)^2} + \frac{1}{(2-\lambda)^2} - 1 =$$$$= \frac{(1-\lambda)^2(2-\lambda)^2 + (3+\lambda)^2(2-\lambda)^2 + (3+\lambda)^2 (1-\lambda)^2 - (3+\lambda)^2 (1-\lambda)^2 (2-\lambda)^2}{(3+\lambda)^2 (1-\lambda)^2 (2-\lambda)^2} = 0 \Leftrightarrow$$$$\Leftrightarrow (1-\lambda)^2(2-\lambda)^2 + (3+\lambda)^2(2-\lambda)^2 + (3+\lambda)^2 (1-\lambda)^2 - (3+\lambda)^2 (1-\lambda)^2 (2-\lambda)^2 = 0 \Leftrightarrow$$$$\Leftrightarrow (\lambda^4 - 6\lambda^3 + 13\lambda^2 - 12\lambda + 16) + (\lambda^4 + 2\lambda^3 - 11\lambda^2 - 12\lambda + 36) + (\lambda^4 + 4\lambda^3 - 2\lambda^2 - 12\lambda + 9)$$

• (-\lambda^5 -14\lambda^4 +12\lambda^3 +49\lambda^2 -84\lambda + 36) = $$$$ =-\lambda^5 +17\lambda^4 -12\lambda^3 -49\lambda^2 +48\lambda +13 = 0 \Leftrightarrow $$ \Leftrightarrow \lambda = \lambda_1 \approx -0.224 \lor \lambda = \lambda_2 \approx -1.892 \lor \lambda = \lambda_3 \approx 3.149 \lor \lambda = \lambda_4 \approx -4.035$$

We then compute X from each solution and evaluate the objective each time:

$$X = \begin{bmatrix}\frac1{3+\lambda}\-\frac1{1-\lambda}\ -\frac{1}{2-\lambda}\end{bmatrix} \Leftrightarrow$$$$\Leftrightarrow X = X_1 \approx \begin{bmatrix}0.360\-0.817\-0.450\end{bmatrix} \lor X = X_2 \approx \begin{bmatrix}0.902\-0.346\-0.257\end{bmatrix} \lor X = X_3 \approx \begin{bmatrix}0.163\0.465\0.870\end{bmatrix} \lor X = X_4 \approx \begin{bmatrix}-0.966\-0.199\-0.166\end{bmatrix}

$$f(X_1) = -1.1304 ;; f(X_2) = -1.59219 ;; f(X_3) = 4.64728 ;; f(X_4) = -5.36549$$

Exercise 1.3

To find the optimal solution, we choose (\lambda_4, X_4) since f(X_4) is the smallest objective value out of all the feasible points. Therefore, the solution to the minimization problem is:

X \approx \begin{bmatrix}-0.966\\-0.199\\-0.166\end{bmatrix}

Exercise 2

Exercise 2.1

To reformulate the problem, we first rewrite the explicit values of G, c, A and b:

G = \begin{bmatrix}3&0&0\\2&2.5&0\\1&2&2\end{bmatrix}

$$c = \begin{bmatrix}-8\\-3\\-3\end{bmatri

A = \begin{bmatrix}1&0&1\\0&1&1\end{bmatrix}

b = \begin{bmatrix}3\\0\end{bmatrix}