diff --git a/Claudio_Maggioni_4/Claudio_Maggioni_4.md b/Claudio_Maggioni_4/Claudio_Maggioni_4.md index 0efa064..50a96d9 100644 --- a/Claudio_Maggioni_4/Claudio_Maggioni_4.md +++ b/Claudio_Maggioni_4/Claudio_Maggioni_4.md @@ -93,7 +93,7 @@ $$f(X_1) = -1.1304 \;\; f(X_2) = -1.59219 \;\; f(X_3) = 4.64728 \;\; f(X_4) = To find the optimal solution, we choose $(\lambda_4, X_4)$ since $f(X_4)$ is the smallest objective value out of all the feasible points. Therefore, the solution to the -minimization problem is: +constrained minimization problem is: $$X \approx \begin{bmatrix}-0.966\\-0.199\\-0.166\end{bmatrix}$$ @@ -102,15 +102,70 @@ $$X \approx \begin{bmatrix}-0.966\\-0.199\\-0.166\end{bmatrix}$$ ## Exercise 2.1 To reformulate the problem, we first rewrite the explicit values of $G$, $c$, -$A$ and $b$: +$A$ and $b$. -$$G = 2 \cdot \begin{bmatrix}3&0&0\\2&2.5&0\\1&2&2\end{bmatrix}$$ -$$c = \begin{bmatrix}-8\\-3\\-3\end{bmatrix}$$ -$$A = \begin{bmatrix}1&0&1\\0&1&1\end{bmatrix}$$ -$$b = \begin{bmatrix}3\\0\end{bmatrix}$$ +We first define matrix $G$ as a set of 9 unknown variables and $c$ a set of 3 +unknown variables: -Then, using these variable values and the formulation given on the assignment -sheet the problem is restated in this new form. +$$G = \begin{bmatrix}g_{11}&g_{12}&g_{13}\\g_{21}&g_{22}&g_{23}\\ +g_{31}&g_{32}&g_{33}\end{bmatrix} \;\;\; c = +\begin{bmatrix}c_1\\c_2\\c_3\end{bmatrix}$$ + +We then define $f(x)$ in the following way: + +$$f(x) = \frac12 \cdot \begin{bmatrix}x_1&x_2&x_3\end{bmatrix} +\begin{bmatrix}g_{11}&g_{12}&g_{13}\\g_{21}&g_{22}&g_{23}\\ +g_{31}&g_{32}&g_{33}\end{bmatrix} \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} +- \begin{bmatrix}c_1&c_2&c_3\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} +=$$$$=x_1^2 \cdot \frac{g_{11}}{2} + x_2^2 \cdot \frac{g_{22}}{2} + +x_3^2 \cdot \frac{g_{33}}{2} + +\left(\frac{g_{12} + g_{21}}{2}\right) x_1 x_2 ++ \left(\frac{g_{13} + g_{31}}{2}\right) x_1 x_3 + +\left(\frac{g_{23} + g_{32}}{2}\right) x_2 x_3 + c_1 x_1 + c_2 x_2 + c_3 x_3$$ + +Then, we equal this polynomial to the given one, finding the following values +and constraints for the coefficients of $G$ and $g$: + +$$\begin{cases}g_{11} = 3 \cdot 2 = 6 \\ +g_{22} = 2.5\cdot 2 = 5 \\ +g_{33} = 2 \cdot 2 = 4 \\ +c_1 = -8 \\ +c_2 = -3 \\ +c_3 = -3 \\ +g_{13} + g_{31} = 1 \cdot 2 = 2 \\ +g_{12} + g_{21} = 2 \cdot 2 = 4 \\ +g_{23} + g_{32} = 2 \cdot 2 = 4 \end{cases}$$ + +As it can be seen by the system of equations above, we have infinite possibility +for choosing the components of the $G$ matrix that are not on the main diagonal. +Due to personal taste, we choose those components in such a way that the +resulting $G$ matrix is symmetric. We therefore obtain: + +$$G = \begin{bmatrix}6&2&1\\2&5&2\\1&2&4\end{bmatrix} \;\;\; +c = \begin{bmatrix}-8\\-3\\-3\end{bmatrix}$$ + +We perform a similar process for matrix $A$ and vector $b$ + +$$Ax = b \Leftrightarrow +\begin{bmatrix}a_{11}&a_{12}&a_{13}\\a_{21}&a_{22}&a_{23}\end{bmatrix} +\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = +\begin{bmatrix}b_1\\b_2\end{bmatrix} \Leftrightarrow +\begin{cases}a_{11}x_1 + a_{12}x_2 + a_{13}x_3 = b_1\\a_{21}x_1 + a_{22}x_2 + +a_{23}x_3 = b_2\end{cases}$$ + +To make this system match the given system of equality constraints, we need to +set the components of $A$ and $b$ in the following way: + +$$\begin{cases}a_{11} = 1\\a_{12} = 0\\a_{13} = 1\\a_{21} = +0\\a_{22}=1\\a_{23}=1 \\b_1 = 3 \\b_2 = 0\end{cases}$$ + +Therefore, we obtain the following $A$ matrix and $b$ vector: + +$$A = \begin{bmatrix}1&0&1\\0&1&1\end{bmatrix} \;\;\; b = \begin{bmatrix}3\\0\end{bmatrix}$$ + +Then, using these $G$, $c$, $A$ and $b$ values, and using the quadratic +formulation of the problem written on the assignment +sheet, the problem is restated in the desired new form. ## Exercise 2.2 @@ -118,7 +173,7 @@ The lagrangian for this problem is the following: $$L(x, \lambda) = \frac12\langle x, Gx\rangle + \langle x, c \rangle - \lambda (Ax - b) =$$$$= \begin{bmatrix}x_1&x_2&x_3\end{bmatrix} - \begin{bmatrix}3&0&0\\2&2.5&0\\1&2&2\end{bmatrix} + \begin{bmatrix}6&2&1\\2&5&2\\1&2&4\end{bmatrix} \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} + \begin{bmatrix}x_1&x_2&x_3\end{bmatrix} \begin{bmatrix}-8\\-3\\-3\end{bmatrix} - \lambda @@ -132,9 +187,11 @@ The KKT conditions are the following: First we have the condition on the partial derivatives of the Lagrangian w.r.t. $X$: -$$\nabla_x L(x, \lambda) = Gx + c - A^T \lambda = \begin{bmatrix}3 x_1 - 8 + - \lambda_1\\ 2x_1 + 2.5 x_2 - 3 + \lambda_2\\x_1 + 2x_2 + 2x_3 - 3 + \lambda_1 -+ \lambda_2\end{bmatrix} > 0$$ +$$\nabla_x L(x, \lambda) = Gx + c - A^T \lambda = +\begin{bmatrix} +6 x_1 + 2 x_2 + x_3 - 8 + \lambda_1\\ +2 x_1 + 5 x_2 + 2 x_3 - 3 + \lambda_2\\ +1 x_1 + 2 x_2 + 4 x_3 - 3 + \lambda_1 + \lambda_2\end{bmatrix} = 0$$ Then we have the conditions on the equality constraint: @@ -168,18 +225,24 @@ The KKT conditions are the following: $$x \geq 0$$ 4. The lagrangian multipliers for inequality constraints are non-negative: $$s \geq 0$$ -5. The complementarity condition holds (here considering only inequality constraints, - since the condition trivially holds for equality ones): - $$s^T x \geq 0$$ +5. The complementarity condition holds (here considering only inequality + constraints, since the condition trivially holds for equality ones): $$s^T x + \geq 0$$ ## Exercise 3.2 We define the dual problem is the following way: +$$\max b^T \lambda \;\; \text{ s.t. } \;\; A^T \lambda \leq c \;$$ + +We then introduce a slack variable $s$ to find the equality and inequality +constraints: + $$\max b^T \lambda \;\; \text{ s.t. } \;\; A^T \lambda + s = c \; \text{ and }\; s \geq 0$$ -To convert this maximization problem in a minimization one, we flip the sign of +To convert this maximization problem in a minimization one (in order to achieve +standard form), we flip the sign of the objective and we find: $$\min - b^T \lambda \;\; \text{ s.t. } \;\; A^T \lambda + s = c \; \text{ and @@ -187,22 +250,21 @@ $$\min - b^T \lambda \;\; \text{ s.t. } \;\; A^T \lambda + s = c \; \text{ and We then compute the Lagrangian of the dual problem: -$$L(\lambda, x) = -b^T \lambda + x^T (A^T \lambda + s - c) - x^T s = - b^T \lambda + x^T (A^T \lambda - c)$$ +$$L(\lambda, x) = -b^T \lambda + x^T (A^T \lambda + s - c) - x^T s = - b^T +\lambda + x^T (A^T \lambda - c)$$ The KKT conditions are the following: -1. The partial derivative of the lagrangian w.r.t. $x$ is 0: - $$\nabla_{\lambda} L(\lambda, x) = - b^T + x^T A^T = 0 \Leftrightarrow Ax = b$$ -2. Equality constraints hold: - $$A^T \lambda + s = c$$ -3. Inequality constraints hold: - $$c - A^T \lambda \geq 0 \Leftrightarrow s \geq 0 \;\;\; \text{ using 2.\ to find - that } s = c - A^T \lambda$$ -4. The lagrangian multipliers for inequality constraints are non-negative: - $$x \geq 0$$ -5. The complementarity condition holds (here considering only inequality constraints, - since the condition trivially holds for equality ones): - $$x^T s \geq 0 \Leftrightarrow s^T x \geq 0$$ +1. The partial derivative of the lagrangian w.r.t. $x$ is 0: $$\nabla_{\lambda} + L(\lambda, x) = - b^T + x^T A^T = 0 \Leftrightarrow Ax = b$$ +2. Equality constraints hold: $$A^T \lambda + s = c$$ +3. Inequality constraints hold: $$c - A^T \lambda \geq 0 \Leftrightarrow s \geq + 0 \;\;\; \text{ using 2.\ to find that } s = c - A^T \lambda$$ +4. The lagrangian multipliers for inequality constraints are non-negative: $$x + \geq 0$$ +5. The complementarity condition holds (here considering only inequality + constraints, since the condition trivially holds for equality ones): $$x^T s + \geq 0 \Leftrightarrow s^T x \geq 0$$ Then, if we compare the KKT conditions of the primal problem with the ones above we can match them to see that they are identical: diff --git a/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf b/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf index fcbd8e3..653098e 100644 Binary files a/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf and b/Claudio_Maggioni_4/Claudio_Maggioni_4.pdf differ