mp1: waiting for Edoardo's benedition for 1.1
This commit is contained in:
parent
b5cc7065de
commit
18e275d120
2 changed files with 59 additions and 3 deletions
BIN
mp1/template.pdf
BIN
mp1/template.pdf
Binary file not shown.
|
@ -22,11 +22,67 @@ The purpose of this assignment\footnote{This document is originally based on a S
|
||||||
|
|
||||||
\subsection{Theory [20 points]}
|
\subsection{Theory [20 points]}
|
||||||
|
|
||||||
|
\subsubsection{Show that the order of convergence of the power method is linear,
|
||||||
|
and state what the asymptotic error constant is.}
|
||||||
|
|
||||||
|
First of all, we show the the sequence of vectors computed by power iteration
|
||||||
|
indeed converges to $\lambda_1$ or the biggest eigenvector (we assume we name
|
||||||
|
eigenvectors in decreasing order of magnitude, with $|\lambda_1| > |\lambda_i|$
|
||||||
|
for $i \in 2..n$).
|
||||||
|
|
||||||
|
We can express the seed for the eigenvector (i.e. the initial value of $v$ of
|
||||||
|
the power iteration) as a linear combination of eigenvalues:
|
||||||
|
|
||||||
|
\[v_0 = \sum_{i=1}^n a_i x_i\]
|
||||||
|
|
||||||
|
We can then express the result of the n-th power method as
|
||||||
|
|
||||||
|
\[v_n = \gamma A v_{n-1} = A^n v_0 = \sum_{i=1}^n \gamma a_i \lambda_i^n x_i =
|
||||||
|
\lambda_1^n \sum_{i=1}^n \gamma a_i \left( \frac{\lambda_i}{\lambda_1} \right)^n x_i =
|
||||||
|
\gamma a_1 \lambda_1^n x_1 + \lambda_1^n \sum_{i=2}^n \gamma a_i
|
||||||
|
\left(\frac{\lambda_i}{\lambda_1}\right)^n x_i \]
|
||||||
|
|
||||||
|
Here, $\gamma$ is just a normalization term to make $||v_n|| = 1$. $v_n$ clearly
|
||||||
|
converges to $x_1$ since all the terms in the $\sum_{i=2}^n$ contain
|
||||||
|
$\frac{\lambda_i}{\lambda_1}$, which is always less than 0 if $i > 1$ for the
|
||||||
|
sorting of eigenvalues we did before. Therefore, these terms to the power of n
|
||||||
|
converge to 0, and $\gamma$ will cancel out $a_1 \lambda_1^k$ due to the
|
||||||
|
normalization, thus making the sequence converge to $\lambda_1$.
|
||||||
|
|
||||||
|
To see if the sequence converges linearly we use the definitions of rate of
|
||||||
|
convergence:
|
||||||
|
|
||||||
|
\[\lim_{n \to \infty}\frac{|x_{n+1} - \lambda_1|}{|x_n - \lambda_1|^1} = \mu\]
|
||||||
|
|
||||||
|
If this limit has a finite solution then the sequence converges linearly with
|
||||||
|
rate $\mu$.
|
||||||
|
|
||||||
|
\[\lim_{n \to \infty}\frac{\left| a_1 \lambda_1^{n+1} x_1 + \lambda_1^{n+1}
|
||||||
|
\sum_{i=2}^n a_i \left(\frac{\lambda_i}{\lambda_1}\right)^{n+1}
|
||||||
|
x_i - \beta_{n+1} x_1\right|}
|
||||||
|
{\left| a_1 \lambda_1^n x_1 + \lambda_1^n \sum_{i=2}^n a_i
|
||||||
|
\left(\frac{\lambda_i}{\lambda_1}\right)^n x_i - \beta_n x_1\right|^1} = \mu\]
|
||||||
|
|
||||||
|
To simplify calculations, we consider the sequence without the normalization
|
||||||
|
factor $\gamma$ that will converge to a denormalized version of $x_1$, named
|
||||||
|
$\beta x_1$. We can then simplify the $a_1\lambda_1^{i}x_1$ terms in the
|
||||||
|
sequences with $\beta_{i} x_1$ since $\beta_i$ can be set freely.
|
||||||
|
|
||||||
|
Now we consider that if $|\lambda_2| > |\lambda_i| \forall i \in 3..n$ (since we
|
||||||
|
sorted the eigenvalues), then
|
||||||
|
$\left(\frac{\lambda_i}{\lambda_1}\right)^n$ for $i > 2$ will always converge faster to
|
||||||
|
0 than $\left(\frac{\lambda_2}{\lambda_1}\right)^n$ thus all terms other than
|
||||||
|
$i=2$ can be ignored in the limit computation. Therefore, the limit has finite
|
||||||
|
solution and the convergence rate
|
||||||
|
is
|
||||||
|
|
||||||
|
\[\mu = \frac{\lambda_2}{\lambda_1}\]
|
||||||
|
|
||||||
\subsubsection{What assumptions should be made to guarantee convergence of the power method?}
|
\subsubsection{What assumptions should be made to guarantee convergence of the power method?}
|
||||||
The first assumption to make is that the biggest eigenvalue in terms of absolute values should (let's name it $\lambda_1$)
|
The first assumption to make is that the biggest eigenvalue in terms of absolute values should (let's name it $\lambda_1$)
|
||||||
be strictly greater than all other eigenvectors, so:
|
be strictly greater than all other eigenvectors, so:
|
||||||
|
|
||||||
$$|\lambda_1| < |\Lambda_i| \forall i \in \{2..n\}$$
|
\[|\lambda_1| < |\lambda_i| \forall i \in \{2..n\}\]
|
||||||
|
|
||||||
Also, the eigenvector \textit{guess} from which the power iteration starts must have a component in the direction of $x_i$, the eigenvector for the eigenvalue $\lambda_1$ from before.
|
Also, the eigenvector \textit{guess} from which the power iteration starts must have a component in the direction of $x_i$, the eigenvector for the eigenvalue $\lambda_1$ from before.
|
||||||
|
|
||||||
|
@ -38,11 +94,11 @@ The shift and invert approach is a variant of the power method that may signific
|
||||||
|
|
||||||
where $\alpha$ is an arbitrary constant that must be chosen wisely in order to increase the rate of convergence. Since the eigenvalues $u_i$ of B can be derived from the eigenvalues $\lambda_i$ of A, namely:
|
where $\alpha$ is an arbitrary constant that must be chosen wisely in order to increase the rate of convergence. Since the eigenvalues $u_i$ of B can be derived from the eigenvalues $\lambda_i$ of A, namely:
|
||||||
|
|
||||||
$$u_i = \frac{1}{\lambda_i - \alpha}$$
|
\[u_i = \frac{1}{\lambda_i - \alpha}\]
|
||||||
|
|
||||||
the rate of convergence of the power method on B is:
|
the rate of convergence of the power method on B is:
|
||||||
|
|
||||||
$$\left|\frac{u_2}{u_1}\right| = \left|\frac{\frac1{\lambda_2 - \alpha}}{\frac1{\lambda_1 - \alpha}}\right| = \left|\frac{\lambda_1 - \alpha}{\lambda_2 - \alpha}\right|$$
|
\[\left|\frac{u_2}{u_1}\right| = \left|\frac{\frac1{\lambda_2 - \alpha}}{\frac1{\lambda_1 - \alpha}}\right| = \left|\frac{\lambda_1 - \alpha}{\lambda_2 - \alpha}\right|\]
|
||||||
|
|
||||||
By choosing $\alpha$ close to $\lambda_1$, the convergence is sped up. To further increase the rate of convergence (up to a cubic rate), a new $\alpha$, and thus a new $B$, may be chosen for every iteration.
|
By choosing $\alpha$ close to $\lambda_1$, the convergence is sped up. To further increase the rate of convergence (up to a cubic rate), a new $\alpha$, and thus a new $B$, may be chosen for every iteration.
|
||||||
|
|
||||||
|
|
Reference in a new issue