\serieheader{Numerical Computing}{2020}{Student: Claudio Maggioni}{Discussed with: --}{Solution for Project 1}{}
\newline
\assignmentpolicy
The purpose of this assignment\footnote{This document is originally based on a SIAM book chapter from \textsl{Numerical Computing with Matlab} from Clever B. Moler.} is to learn the importance of numerical linear algebra algorithms to solve fundamental linear algebra problems that occur in search engines.
\subsubsection{What assumptions should be made to guarantee convergence of the power method?}
The first assumption to make is that the biggest eigenvalue in terms of absolute values should (let's name it $\lambda_1$)
be strictly greater than all other eigenvectors, so:
$$|\lambda_1| < |\Lambda_i| \forall i \in\{2..n\}$$
Also, the eigenvector \textit{guess} from which the power iteration starts must have a component in the direction of $x_i$, the eigenvector for the eigenvalue $\lambda_1$ from before.
\subsubsection{What is a shift and invert approach?}
The shift and invert approach is a variant of the power method that may significantly increase the rate of convergence where some application of the vanilla method require large numbers of iterations. This improvement is achieved by taking the input matrix $A$ and deriving a matrix $B$ defined as:
\[B =(A -\alpha I)^{-1}\]
where $\alpha$ is an arbitrary constant that must be chosen wisely in order to increase the rate of convergence. Since the eigenvalues $u_i$ of B can be derived from the eigenvalues $\lambda_i$ of A, namely:
$$u_i =\frac{1}{\lambda_i -\alpha}$$
the rate of convergence of the power method on B is:
By choosing $\alpha$ close to $\lambda_1$, the convergence is sped up. To further increase the rate of convergence (up to a cubic rate), a new $\alpha$, and thus a new $B$, may be chosen for every iteration.
\subsubsection{What is the difference in cost of a single iteration of the power method, compared to the inverse iteration?}
Inverse iteration is generally more expensive than a regular application of the power method, due to the overhead caused by the intermediate matrix B. One must either recompute B every time $\alpha$ changes, which is rather expensive due to the inverse operation in the definition of B, or one must solve the matrix equation $(A -\alpha I)v_k = v_{k-1}$ in every iteration.
\subsubsection{What is a Rayleigh quotient and how can it be used for eigenvalue computations?}
The Railegh quotient is an effective way to either compute the corresponding eigenvalue of an eigenvector or the corresponding eigenvalue approximation of an eigenvector approximation. I.e., if $x$ is an eigenvector, then:
\[\lambda=\mu(x)=\frac{x^T A x}{x^T x}\]
is the corresponding eigenvalue, while if $x$ is an eigenvector approximation, for example found through some iterations of the power method, then $\lambda$ is the closest possible approximation to the corresponding eigenvalue in a least-square sense.
The provided PageRank MATLAB implementation was run 3 times on the starting websites \texttt{http://atelier.inf.usi.ch/~maggicl}, \texttt{https://www.iisbadoni.edu.it}, and \texttt{https://www.usi.ch}, with results listed respectively in Figure \ref{fig:run1}, Figure \ref{fig:run2} and Figure \ref{fig:run3}.
One patten that emerges on the first and third execution is the presence of 1s in the main diagonal. This indicates that several pages found have a link to themselves. Another interesting pattern, this time observable in all executions, is the presence of contiguous rectangular regions filled with 1s, especially along the main diagonal. This may be due to the presence of pages belonging to the same website, thus having a common layout and perhaps linking to a common set of internal (when near to the main diagonal) or external pages.
\subsubsection{What is the connectivity matrix G (w.r.t figure 5)?}
The connectivity matrix G, with U being defined as $\{"alpha", "beta", "gamma", "delta", "rho", "sigma"\}$ is:
\[G =\begin{bmatrix}
0&0&0&1&0&0\\
1&0&0&0&0&0\\
1&1&0&0&0&0\\
0&1&1&0&0&0\\
0&0&0&0&0&1\\
0&0&0&0&1&0\\
\end{bmatrix}\]
\subsubsection{What are the PageRanks if the hyperlink transition probability $p$ is the default value 0.85?}
First we compute the matrix A, finding:
\[A =\frac1{40}\begin{bmatrix}
1 &1 &1 &35&1 &1 \\
18&1 &1 &1 &1 &1 \\
18&18&1 &1 &1 &1 \\
1 &18&35&1 &1 &1 \\
1 &1 &1 &1 &1 &35 \\
1 &1 &1 &1 &35&1 \\
\end{bmatrix}\]
We then find the eigenvectors and eigenvalues of A through MATLAB, finding that the solution of $A x =1 x$ is:
\[x\approx\begin{bmatrix}
0.4771\\
0.2630\\
0.3747\\
0.4905\\
0.4013\\
0.4013\\
\end{bmatrix}\]
Thus the pageranks are the components of vector $x$, w.r.t. the order given in U.
\subsubsection{Describe what happens with this example to both the definition of PageRank and the computation done by pagerank in the limit $p \to1$.}
If $p$ is closer to 1, then the probability a web user will visit a certain page randomly decreases, thus giving more weight in the computation of PageRank to the links between one page and another.
In the computation, increasing $p$ decreases $\delta$ (which represents the probability of a user randomly visiting a page), eventually making it 0 when $p$ is 1.
