\subsection{Brief review}
\begin{frame}{Stencils}

Lets consider approximation of the second derivative of a function $u(x)$
\begin{itemize}
\item we can derive an approximation from a truncated Taylor expansion with step size $h$ 
\[\frac{d^2u}{dx^2}(x)\approx\frac{u(x-h)-2u(x)+u(x+h)}{h^2}\]
\item such approximations of derivatives can be represented by a {\bf stencil}
{

\centering
\includegraphics[width=1.9in]{diagrams/stencil}

}
which is applied for every node in the mesh
\item the application of this 1D $3$-point stencil to $n$ grid-nodes, can be done via SpMV with a tridiagonal matrix, like
\begin{align*}
\begin{pmatrix} \frac{d^2u}{dx^2} (h) \\ \vdots \\ \frac{d^2u}{dx^2} (nh)\end{pmatrix} = \frac{1}{h^2}\begin{pmatrix} -2 & 1 &  \\ 1 & \ddots & \ddots \\ & \ddots & &\end{pmatrix}
\begin{pmatrix} u(h)  \\ \vdots \\ u(nh) \end{pmatrix} 
\end{align*}
\end{itemize}

\end{frame}


\subsection{Avoiding synchronizations}

\begin{frame}{Lets start with a 1D 2-point stencil}

Normally, synchronize between every stencil application
\vspace{.5in}

\centering
\includegraphics[width=2.5in]{diagrams/bubble_sstep_d1_syncall}

\end{frame}

\begin{frame}{In-time blocking (matrix-powers kernel)}


Avoid synchronization by applying stencil repeatedly before synchronizing
\vspace{.5in}

\centering
\includegraphics[width=2.9in]{diagrams/bubble_sstep_d1_matpowers}

\end{frame}
\begin{frame}{In-time blocking (matrix-powers kernel)}


Avoid synchronization by applying stencil repeatedly before synchronizing
\vspace{.5in}

\centering
\includegraphics[width=2.9in]{diagrams/bubble_sstep_d1_matpowers_1}

\end{frame}
\begin{frame}{In-time blocking (matrix-powers kernel)}


Avoid synchronization by applying stencil repeatedly before synchronizing
\vspace{.5in}

\centering
\includegraphics[width=2.9in]{diagrams/bubble_sstep_d1_matpowers_2}

\end{frame}
\begin{frame}{In-time blocking (matrix-powers kernel)}


Avoid synchronization by applying stencil repeatedly before synchronizing
\vspace{.5in}

\centering
\includegraphics[width=2.9in]{diagrams/bubble_sstep_d1_matpowers_3}

\end{frame}

\begin{frame}{Analysis of in-time blocking for 1D mesh}

For 1D mesh its a tasty free lunch
\begin{itemize}[<+->]
\item lets consider $t$ steps, and execute $s$ without synchronization
\item we are constrained by $s\leq n/P$
\item bring down latency cost by a factor of $s$, for $t=\Theta(n)$, we improve latency cost from $O(n\cdot \alpha)$ to $O(P\cdot \alpha)$
\item Q: can we improve memory-bandwidth cost via in-time blocking?
\item A: yes, but it depends on whether we need to output all vectors or just the last one we compute
\item in the latter case, we can improve reuse by factor of $\Theta(\max(s,H))$
\end{itemize}

\end{frame}

\begin{frame}{Analysis of in-time blocking for $d$D mesh}

For $d$D mesh, there is more complexity
\begin{itemize}[<+->]
\item again consider $t$ steps, and execute $s$ without synchronization
\item we are constrained by $s\leq (n/P)^{1/d}$
\begin{itemize}[<+->]
\item otherwise we need to do asymptotically more computation and interprocessor communication
\centering
\includegraphics[width=3in]{diagrams/sstep_d2_imports}
\end{itemize}
%\item bring down latency cost by a factor of $k$, for $s=\Theta(n)$, we improve latency cost from $O(n\cdot \alpha)$ to $O(P\cdot \alpha)$
%\item Q: can we also improve memory-bandwidth cost using in-time blocking?
%\item A: it depends on whether we need to output all vectors or just the final one
%\item in the latter case, we can improve reuse by factor of $O(\max(k,H))$ for 1D
\end{itemize}

\end{frame}


\begin{frame}{Analysis of in-time blocking for $d$D mesh}

For $d$D mesh with $t$ total steps, $s=\Theta((n/P)^{1/d})$ without syncs.
\begin{itemize}[<+->]
\item for $t=\Theta(n^{1/d})$ we lower latency cost to $\Theta(t/s)=\Theta(P^{1/d})$
\item Q: if we only need to output the final vector, by how much can we improve memory bandwidth cost with $s=\Theta((n/P)^{1/d})$?
\item A: $\Theta(H^{1/d})$, by doing $O(H)$ loads and stores for computation volumes of size $H^{1/d}\times \cdots \times H^{1/d}$
%\item Q: can we also improve memory-bandwidth cost using in-time blocking?
%\item A: it depends on whether we need to output all vectors or just the final one
%\item in the latter case, we can improve reuse by factor of $O(\max(k,H))$ for 1D
\end{itemize}

\end{frame}

\begin{frame}{Analysis of in-time blocking for $d$D mesh contd.}

Lets now consider the full cost of $t$ applications with $s$ steps at a time
\begin{itemize}[<+->]
\item recall from last lecture the cost of the one-at-a-time SpMV approach
\[t\cdot T_\text{SpMV-d}(n,d,P)=O\bigg(\frac{tn}{P}\cdot \nu + t\Big(\frac{n}{P}\Big)^{(d-1)/d}\cdot \beta + t\cdot \alpha\bigg)\]
\item we now obtain a cost
\begin{align*}
T_\text{CA-St}(n,d,P,t,s)=O\bigg(&\frac{tn}{\min(s,H^{1/d})P}\cdot \nu  \\
&+ \bigg[t\Big(\frac{n}{P}\Big)^{(d-1)/d}+ts^{d-1}\bigg]\cdot \beta 
+ \frac{t}{s}\cdot \alpha\bigg) 
\end{align*}
\item when we pick $s=(n/P)^{1/d}$, we obtain
\begin{align*}
T_\text{CA-St}(n,d,P,t,(n/P)^{1/d})=O\bigg(&\frac{tn}{H^{1/d}P}\cdot \nu \\
&+t\Big(\frac{n}{P}\Big)^{(d-1)/d}\cdot \beta 
+ \frac{t}{(n/P)^{1/d}}\cdot \alpha\bigg)
\end{align*}
\end{itemize}
\end{frame}

