To the right of this matrix I have a big right curly bracket with the text "2m terms" that spans the height of the matrix. I need to replace this right bracket with three smaller ones that together span the height of the matrix.
\documentclass[a4paper,12pt,times,numbered,print,index]{article}
\RequirePackage[left=37mm,right=30mm,top=35mm,bottom=30mm]{geometry}
\RequirePackage{libertine}
\RequirePackage[small,bf,labelsep=space,
tableposition=top]{caption}
\usepackage{multirow}
\usepackage{subfig,amsfonts,amsmath,amssymb,braket}
\usepackage{algorithm}% http://ctan.org/pkg/algorithms
\usepackage{algpseudocode}% http://ctan.org/pkg/algorithmicx
\begin{document}
\newcolumntype{C}[1]{>{\centering\arraybackslash$}p{#1}<{$}}
\def\rot#1{\rotatebox{90}{#1}}
\begin{figure}
\centering
\[
\setcounter{MaxMatrixCols}{20}
\begin{array}{@{}r@{}}
\overbrace{\begin{array}{@{}*{6}{C{1.5em}}@{}}
&&&&&&
\end{array}}^{\text{$m$ terms}}
\overbrace{\begin{array}{@{}*{6}{C{1.5em}}@{}}
&&&&&&
\end{array}}^{\text{$m$ terms}} \hspace*{30pt}\\[-.5\normalbaselineskip]
% \overbrace{\begin{array}{@{}*{5}{C{1.5em}}@{}}
% &&&&
% \end{array}}^{\text{Columns: $0$ to $\frac{m}{2}-1$}}
% \overbrace{\begin{array}{@{}*{5}{C{1.5em}}@{}}
% &&&&
% \end{array}}^{\text{Columns: $\frac{m}{2}$ to $m$}} \hspace*{25pt}\\[-.5\normalbaselineskip]
H_{ab} =
\left.\hspace*{-\nulldelimiterspace}
\left[\begin{array}{@{}*{12}{C{1.5em}}@{}}
\underset{1}\leftarrow & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \cdots & \underset{2}\rightarrow & \cdots & \cdots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \underset{5}\rightarrow & \cdots & \cdots & \cdots & \vdots \\
\vdots & \underset{6}\leftarrow & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \underset{9}\rightarrow & \cdots & \cdots & \vdots \\
\vdots & \cdots & \underset{10}\leftarrow & \cdots& \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \ddots & \ddots & \cdots & \ddots & \ddots & \ddots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \ddots & \ddots & \ddots & \ddots & \ddots & \ddots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \underset{m-3}\leftarrow & \cdots & \cdots & \cdots & \cdots & \cdots & \vdots \\
\vdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \underset{m-2}\rightarrow \\
\cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\
\end{array}\right]\right\} \rotatebox[origin=c]{-90}{\scriptsize $2m$ terms}
\end{array}
\]
\end{figure}
\end{document}
How can I change the bracket on the right to three smaller brackets?
C{1.5em}column type come from? The onlyCI know of comes from thetabularypackage and it does not accept any argument if I'm not mistaken. – Bernard Apr 06 '14 at 10:46