8

Suppose that we have a $\\n\times p$ matrix $\mathbf{M}$. Different transformations using different column-wise operators can lead to a new $\\p\times p$ symmetric matrix $\mathbf{S}$. For example, the covariance matrix $\mathbf{C}$ can be computed using the dot product operator, whereby each value of the covariance matrix is the dot product of two columns of the original matrix $\mathbf{M}$ (divided by $n-1$):

$\mathbf{C} = { 1 \over {n-1} } \mathbf{M}^{T} \cdot \mathbf{M}$

Similarly, the correlation matrix $\mathbf{P}$ can be defined by $\mathbf{P}_{ij} = \mathrm{corr}(C_i,C_j)$ where $\mathbf{C}_i$ and $\mathbf{C}_j$ are columns of $\mathbf{C}$, and $\mathrm{corr}$ is a measure of correlation like the Pearson product-moment coefficient. In this case, the operator is the bivariate correlation coefficient.

Does this operator-dependent transformation have a name?

Ismael Ghalimi
  • 2,166
  • 1
    In this answer I've outlined how all these matrices are the modifications of the SSCP X'X matrix. For further scholastic interest: Zegers, ten Berge (Psychometrica, 1985) unite by the single general formula 4 coefficients: identity coef, additivity coef (based on covariance coef), cosine, Pearson correlation. – ttnphns Mar 29 '16 at 13:14
  • 1
    I would not recommend to believe the Wikipedia article on "Gramian matrix". Actually, "Gramian" means different things. For example, within factor analysis literature this word is long (since 1930s) used as a synonym to "positive semidefinite square matrix". In many sources, "Gram matrix" is not the same as "Gramian matrix", and so on. – ttnphns Mar 29 '16 at 13:18
  • 2
    Some vector-oriented computing systems, such as APL and Mathematica, have built-in generalizations of this type. See https://reference.wolfram.com/language/ref/Inner.html for instance. – whuber Mar 29 '16 at 14:00
  • @ttnphns Thank you so much for your answer. It is exactly what I was looking for. In such a context, is it correct to say that the transformation I was looking for is called "Cross Product"? – Ismael Ghalimi Mar 29 '16 at 15:54
  • I'm at risk of pointing out the obvious, but If the result is PSD, it's a Mercer kernel. – Sycorax Mar 29 '16 at 16:13
  • "Gram[ian] matrix" is usually understood to be $n\times n$, not $p \times p$ (similar to the kernel matrix in kernel-based methods). – amoeba Mar 29 '16 at 16:19
  • 2
    Ismael: If you need a label for X'X matrix - there are many synonymous or equivalent, coming from people of different fields and backgrounds. Multivariate statistical data analysis is one of the oldest branch. There it is called Sums-of-Squares-and-Crossproducts matrix or simply Crossproduct matrix. I don't recommend much using words Gramian or Gram due to reasons pointed in my comment above. – ttnphns Mar 29 '16 at 18:04
  • 1
    @ttnphns I think Ismael's question is more broad, he wants a name for any $f(X_i, X_j)$ matrix where $f()$ can be any (perhaps only symmetric?) function (and $X_i$ are columns of $X$); e.g. a $p\times p$ matrix of mutual information between variables or a $p\times p$ matrix of distance covariances between them, of whatever. Ismael, did I understand correctly? If so, then I don't think there is a general name for that. – amoeba Mar 29 '16 at 21:58
  • @amoeba Yes, you understood correctly. If you had to come up with a name, what would you call it? – Ismael Ghalimi Mar 30 '16 at 05:28
  • @whuber Your comment regarding Mathematica's Inner is right on. Inner could be a name for the generic operator I'm talking about (dot Product, Pearson, etc.), but I'm also looking for a name to call the related matrix transformation, which seems to be a generalization of Crossproduct. – Ismael Ghalimi Mar 30 '16 at 05:33
  • 1
    I don't like "crossproduct" (even "generalized") because it makes it sound that there is some "product", i.e. multiplication, involved. But many $f(\cdot,\cdot)$ cannot be written as a product at all. If you really have to have a term for that, I think you should invent one. "Cross-relationship matrix"? Your operator would then be a "relationship" operator. – amoeba Mar 30 '16 at 12:34
  • @amoeba I like Cross-relationship matrix. Could you submit that as an answer? – Ismael Ghalimi Mar 30 '16 at 17:00

1 Answers1

3

My understanding of the question is that it asks for a name for any $p\times p$ matrix $\mathbf F$ with elements $F_{ij} = f(\mathbf X_i, \mathbf X_j)$, where $\mathbf X_k$ are columns of a $n\times p$ data matrix $\mathbf X$ and $f(\cdot, \cdot)$ is some arbitrary function.

This can be seen as a generalization of covariance matrix (if $f$ is covariance), correlation matrix (if $f$ is correlation), sum-of-squares-and-cross-products matrix (if $f$ is dot product), etc.

Note that even if $f$ is symmetric and "sensible", the resulting matrix $\mathbf F$ can fail to be positive-definite. This is the case e.g. when $f$ is mutual information (as stated in the title of this paper).

I doubt that there is a generic term for $\mathbf F$. If you really have to have a name for it, I think you should invent one. If your $f$ is supposed to measure some relationship between variables $i$ and $j$, then perhaps $\mathbf F$ can be called "cross-relationship matrix" or simply "relationship matrix"?

amoeba
  • 104,745
  • I think you can use the language of reproducing kernel Hilbert spaces and consider $\mathbf F$ to be the Gram matrix associated with the kernel $f$, provided $f$ satisfies the standard conditions. – cardinal Mar 30 '16 at 21:02
  • 2
    @cardinal I was thinking whether I should mention this in my answer. But usually kernel matrix (Gram matrix) is understood to be $n\times n$, i.e. it measures the "relationships" (kernel distances) between the samples, whereas here OP considers a $p \times p$ matrix measuring the "relationships" between the variables. So I figured that to call it kernel/Gram matrix might be confusing; what do you think? (Also, one can consider functions that do not satisfy kernel conditions, e.g. apparently mutual information.) – amoeba Mar 30 '16 at 21:08
  • 1
    (+1) While, in some sense, the distinction between "rows" and "columns" may seem a bit superficial, I think you raise a very valid point regarding the potential confusion that could be caused by the proposal in my comment. – cardinal Mar 30 '16 at 21:17
  • Thank you so much for clarifying the distinction with Gram matrices. Very helpful. – Ismael Ghalimi Mar 31 '16 at 05:10