OM/Claudio_Maggioni_5/Claudio_Maggioni_5.md

<!-- vim: set ts=2 sw=2 et tw=80: -->

---
title: Homework 5 -- Optimization Methods
author: Claudio Maggioni
header-includes:
- \usepackage{amsmath}
- \usepackage{hyperref}
- \usepackage[utf8]{inputenc}
- \usepackage[margin=2.5cm]{geometry}
- \usepackage[ruled,vlined]{algorithm2e}
- \usepackage{float}
- \floatplacement{figure}{H}
- \hypersetup{colorlinks=true,linkcolor=blue}

---
\maketitle

# Exercise 2

## Exercise 2.1

The resulting MATLAB plot of each constraint and of the feasible region is shown
below:

![Plot of feasible region and constraints\label{fig:a}](./ex2-1.png)

## Exercise 2.2

<!--The Simplex method is used to solve linear programs which are defined as
follows:

$$\min c^Tx, \text{ subject to } Ax = b, x > 0$$

And when we have inequalities constraints such as:

$$\min c^Tx,\text{ subject to }Ax \leq b$$

We can introduce slack variable to convert the inequalities into equalities:

$$\min c^Tx,\text{ subject to }Ax + z = 0, z>0$$

Each iterate generated by the simplex method is a basic feasible point which
is a-->

According to Nocedal, a vector $x$ is a basic feasible point if it is in the
feasible region and if there exists a subset $\beta$ of the
index set $1, 2, \ldots, n$ such that:

- $\beta$ contains exactly $m$ indices, where $m$ is the number of rows of $A$;
- For any $i \notin \beta$, $x_i = 0$, meaning the bound $x_i \geq 0$ can be
  inactive only if $i \in \beta$;
- The $m$ x $m$ matrix $B$ defined by $B = [A_i]_{i \in \beta}$ (where $A_i$ is
  the i-th column of A) is non-singular, i.e. all columns corresponding to the
  indices in $\beta$ are linearly independent from each other.

The geometric interpretation of basic feasible points is that all of them
are verticies of the polytope that bounds the feasible region. We will use this
proven property to manually solve the constrained minimization problem presented
in this section by aiding us with the graphical plot of the feasible region in
figure \ref{fig:a}.

<!--
And as already been
said, the basic feasible point is the basic feasible solution for the problem.
Regarding the section 13.3, which outline how the steps are done, we know that
most steps consist of a move from one vertex to an adjacent one for which the
basis $\beta$ differs exactly one component.The step is an edge along which the
objective function is reduced.

Then, one major issue at every simplex iteration
is to decide which index must be removed from the basis. As the book specify,
unless the step is a direction of unboundedness, a single index must be removed
by replacing it with another from outside $\beta$.

From a geometrical point of view,
a move, is along an edge of the feasible polytope that decreases $c^Tx$ and we
continue moving along that edge until we find a new vertex. One we find a vertex
we defined the new constraint $x_p > 0$ which is one of the component $x_p,p \in \beta$
decreased to zero. Afterwards we can remove the index $p$ from the basis $\beta$ and
replace it with $q$. A more detailed visualisation can be see in the following
Figure 1 taken from the book.-->

## Exercise 2.3

Since the geometrical interpretation of the definition of basic feasible point
states that these point are non-other than the vertices of the feasible region,
we first look at the plot above and to these points (i.e. the verticies of the
bright green non-trasparent region). Then, we look which constraint boundaries cross these
edges, and we formulate an algebraic expression to find these points. In
clockwise order, we have:

- The lower-left point at the origin, given by the boundaries of the constraints
 $x_1 \geq 0$ and $x_2 \geq 0$:
 $$x^*_1 = \begin{bmatrix}0\\0\end{bmatrix}$$
- The top-left point, at the intersection of constraint boundaries $x_1 \geq 0$ and
  $-3x_1 + 2x_2 \leq 3$:
  $$x_1 = 0 \;\;\; 2x_2 = 3 \Leftrightarrow x_2 = \frac32 \;\;\; x^*_2 =
  \frac12 \cdot \begin{bmatrix}0\\3\end{bmatrix}$$
- The top-center-left point at the intersection of constraint boundaries $-3x_1 +
  2x_2 \leq 3$ and $2x_1 + 3x_2 \leq 6$:
  $$-3x_1 + 2x_2 + 3x_1 + \frac92 x_2 = \frac{13}{2} x_2 = 3 + 9 = 12
  \Leftrightarrow x_2 = 12 \cdot \frac2{13} = \frac{24}{13}$$$$
  -3x_1 + 2 \cdot \frac{24}{13} = 3 \Leftrightarrow x_1 = \frac{39 - 48}{13}
  \cdot \frac{1}{-3} = \frac{3}{13} \;\;\; x^*_3 = \frac{1}{13} \cdot
  \begin{bmatrix}3\\24\end{bmatrix}$$
- The top-center-right point at the intersection of constraint boundaries $2x_1
  + 3x_2 \leq 6$ and $2x_1 + x_2 \leq 4$:
  $$2x_1 + 3x_2 - 2x_1 - x_2 = 2x_2 = 6 - 4 = 2 \Leftrightarrow x_2 = 1 \;\;\;
  2x_1 + 1 = 4 \Leftrightarrow x_1 = \frac32 \;\;\; x^*_4 = \frac12 \cdot
  \begin{bmatrix}3\\2\end{bmatrix}$$
- The right point at the intersection of $2x_1 + x_2 \leq 4$ and $x_2 \geq
  0$:
  $$x_2 = 0 \;\;\; 2x_1 + 0 = 4 \Leftrightarrow x_1 = 2 \;\;\; x^*_5 =
  \begin{bmatrix}2\\0\end{bmatrix}$$

Therefore, $x^*_1$ to $x^*_5$ are all of the basic feasible points for this
constrained minimization problem.

We then compute the objective function value for each basic feasible point
found, The smallest objective value will correspond with the constrained
minimizer problem solution.

$$
x^*_1 = \begin{bmatrix}0\\0\end{bmatrix} \;\;\; f(x^*_1) = 4 \cdot 0 + 3 \cdot 0 =
0$$$$
x^*_2 = \frac12 \cdot \begin{bmatrix}0\\3\end{bmatrix} \;\;\;
f(x^*_2) = 4 \cdot 0 + 3 \cdot \frac{3}{2} = \frac92$$$$
x^*_3 = \frac{1}{13} \cdot \begin{bmatrix}3\\24\end{bmatrix} \;\;\; f(x^*_3) = 4
\cdot \frac{3}{13} + 3 \cdot \frac{24}{13} = \frac{84}{13}$$$$
x^*_4 = \frac12 \cdot \begin{bmatrix}3\\2\end{bmatrix} \;\;\; f(x^*_4) = 4 \cdot
\frac32 + 3 \cdot 1 = 9$$$$
x^*_5 = \begin{bmatrix}2\\0\end{bmatrix} \;\;\; f(x^*_5) = 4 \cdot 2 + 1 \cdot 0 = 8$$

Therefore, $x^* = x^*_1 = \begin{bmatrix}0 & 0\end{bmatrix}^T$ is the global
constrained minimizer.

# Exercise 3

## Exercise 3.1

## Exercise 3.2

The MATLAB code used to find the solution can be found under section 3.2 of the
`main.m` script. The solution is:

$$x=\begin{bmatrix}0.7692\\-2.2308\\2.2308\end{bmatrix} \;\;\; \lambda=
\begin{bmatrix}-10.3846\\2.1538\end{bmatrix}$$