hw4 is done

Claudio_Maggioni_4/Claudio_Maggioni_4.md

@@ -36,7 +36,7 @@ $$\nabla_X L(X,\lambda) = \begin{bmatrix}(-3-\lambda)x^* + 1\\(1-\lambda)y^* +
 \begin{bmatrix}x^*\\y^*\\z^*\end{bmatrix} =
 \begin{bmatrix}\frac1{3+\lambda}\\-\frac1{1-\lambda}\\-\frac{1}{2-\lambda}\end{bmatrix}$$
 
-Then we have the conditions on the equality constraint:
+Then we have the complementarity condition:
 
 $$c(X) = {x^*}^2 + {y^*}^2 + {z^*}^2 - 1 = 0 \Leftrightarrow \|X^*\| = 1$$
 
@@ -141,10 +141,76 @@ Then we have the conditions on the equality constraint:
 $$Ax - b = 0 \Leftrightarrow \begin{bmatrix}x_1 + x_3\\x_2 + x_3\end{bmatrix} =
   \begin{bmatrix}3\\0\end{bmatrix}$$
 
-Then we have the conditions on the equality constraint:
+Then we have the complementarity condition:
 
 $$\lambda^T (Ax - b) = 0 \Leftarrow Ax - b = 0 \text{ which is true if the above
 condition is true.}$$
 
 Since we have no inequality constraints, we don't need to apply the KKT
 conditions realated to inequality constraints.
+
+# Exercise 3
+
+## Exercise 3.1
+
+The lagrangian of this problem is the following:
+
+$$L(x, \lambda) = c^T x - \lambda^T (Ax - b) - s^T x$$
+
+The KKT conditions are the following:
+
+1. The partial derivative of the lagrangian w.r.t. $x$ is 0:
+   $$\nabla_x L(x, \lambda) = c - A^T \lambda - s = 0 \Leftrightarrow A^T \lambda
+   + s = c$$
+2. Equality constraints hold:
+   $$Ax - b = 0 \Leftrightarrow Ax = b$$
+3. Inequality constraints hold:
+   $$x \geq 0$$
+4. The lagrangian multipliers for inequality constraints are non-negative:
+   $$s \geq 0$$
+5. The complementarity condition holds (here considering only inequality constraints,
+   since the condition trivially holds for equality ones):
+   $$s^T x \geq 0$$
+
+## Exercise 3.2
+
+We define the dual problem is the following way:
+
+$$\max b^T \lambda \;\; \text{ s.t. } \;\; A^T \lambda + s = c \; \text{ and
+}\; s \geq 0$$
+
+To convert this maximization problem in a minimization one, we flip the sign of
+the objective and we find:
+
+$$\min - b^T \lambda \;\; \text{ s.t. } \;\; A^T \lambda + s = c \; \text{ and
+}\; s \geq 0$$
+
+We then compute the Lagrangian of the dual problem:
+
+$$L(\lambda, x) = -b^T \lambda + x^T (A^T \lambda + s - c) - x^T s  = - b^T \lambda + x^T (A^T \lambda - c)$$
+
+The KKT conditions are the following:
+
+1. The partial derivative of the lagrangian w.r.t. $x$ is 0:
+   $$\nabla_{\lambda} L(\lambda, x) = - b^T + x^T A^T = 0 \Leftrightarrow Ax = b$$
+2. Equality constraints hold:
+   $$A^T \lambda + s = c$$
+3. Inequality constraints hold:
+   $$c - A^T \lambda \geq 0 \Leftrightarrow s \geq 0 \;\;\; \text{ using 2.\ to find
+   that } s = c - A^T \lambda$$
+4. The lagrangian multipliers for inequality constraints are non-negative:
+   $$x \geq 0$$
+5. The complementarity condition holds (here considering only inequality constraints,
+   since the condition trivially holds for equality ones):
+   $$x^T s \geq 0 \Leftrightarrow s^T x \geq 0$$
+
+Then, if we compare the KKT conditions of the primal problem with the ones above
+we can match them to see that they are identical:
+
+- 1.\ from the dual is identical to 2.\ from the primal;
+- 2.\ from the dual is identical to 1.\ from the primal;
+- 3.\ from the dual is identical to 4.\ from the primal;
+- 4.\ from the dual is identical to 3.\ from the primal;
+- 5.\ from the dual is identical to 5.\ from the primal.
+
+Therefore, the primal and the dual problem are equivalent.