Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition

P Howland, M Jeon, H Park - SIAM Journal on Matrix Analysis and …, 2003 - SIAM
P Howland, M Jeon, H Park
SIAM Journal on Matrix Analysis and Applications, 2003SIAM
In today's vector space information retrieval systems, dimension reduction is imperative for
efficiently manipulating the massive quantity of data. To be useful, this lower-dimensional
representation must be a good approximation of the full document set. To that end, we adapt
and extend the discriminant analysis projection used in pattern recognition. This projection
preserves cluster structure by maximizing the scatter between clusters while minimizing the
scatter within clusters. A common limitation of trace optimization in discriminant analysis is …
In today's vector space information retrieval systems, dimension reduction is imperative for efficiently manipulating the massive quantity of data. To be useful, this lower-dimensional representation must be a good approximation of the full document set. To that end, we adapt and extend the discriminant analysis projection used in pattern recognition. This projection preserves cluster structure by maximizing the scatter between clusters while minimizing the scatter within clusters. A common limitation of trace optimization in discriminant analysis is that one of the scatter matrices must be nonsingular, which restricts its application to document sets in which the number of terms does not exceed the number of documents. We show that by using the generalized singular value decomposition (GSVD), we can achieve the same goal regardless of the relative dimensions of the term-document matrix. In addition, applying the GSVD allows us to avoid the explicit formation of the scatter matrices in favor of working directly with the data matrix, thus improving the numerical properties of the approach. Finally, we present experimental results that confirm the effectiveness of our approach.
Society for Industrial and Applied Mathematics
以上显示的是最相近的搜索结果。 查看全部搜索结果