Automatic Text Classification Using Random Subspaces and an Ensemble of Classifiers

• Chu Chia Gean PUCPR
• Celso Antônio Alves Kaestner PUCPR

Abstract

Nowadays, due to the large volume of text available in electronic media, the automatic document classification becomes an important modern Information Retrieval task. In this paper we describe a new approach to the problem, based on the classical vector space model for text treatment and on a Pattern Recognition approach. As texts collections produce huge dimensional vector spaces, we attack the problem using several preprocessing techniques, and a set of k-Nearest-Neighbors classifiers, each of them dedicated to a subspace of the original space. The final classification is obtained by a combination of the results of the individual classifiers. We apply our approach to a collection of documents extracted from the TIPSTER and REUTERS databases, and the obtained results are presented.

References

Baeza-Yates, R.; Ribeiro-Neto, B. Modern Information Retrieval. Addison-Wesley, 1999.

Belkin, N.; Croft, W. “Information Filtering and Information Retrieval: Two Sides of the Same Coin”. Communications of the ACM, Nº 35, pp. 29-38, 1992. .

Berry, M.; Drmac, Z.; Jessup, E. “Matrices, Vector Spaces, and Information Retrieval”, SIAM Review, Vol. 41, Nº 2, pp.335-362, 1999.

Cavnar, W. B. “Using An N-Gram-Based Document Representation With a Vector Processing Retrieval Model”. In Proceedings Of TREC-3 (Third Text Retrieval Conference). Gaithersburg, Maryland, USA, 1994.

Deb, K. Multi-Objective Optimization using Evolutionary Algorithms, John Wiley & Sons, 2001.

Deerwester, S.; Dumais, S.; Furnas, G.; Landauer, T. “Indexing by Latent Semantic Analysis”, Journal of the American Society of Information Science, Vol. 41, Nº 6, pp. 391-407, 1990.

Dhillon, I.; Modha, D. “Concept Decompositions for Large Sparse Text Data using Clustering”. Machine Learning, Vol. 42, Nº 1, pp. 143-175, 2001.

Duda, R.; Hart, P.; Stork, D. Pattern Classification (2nd. Edition), Wiley Interscience, 654 p., 2000.

Ho, T.K. “The Random Subspace Method for Constructing Decision Forests”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, Nº 8, pp. 832-844, 1998.

Lewis, D.D. [link] acessado em [08/03/2004].

Lyman, P. and Varian H.R. (2003). How Much Information. Retrieved from [link] acessado em [19/01/2004].

Mitchell, T. Machine Learning. McGraw-Hill, 414p., 1997.

Porter, M.F. “An algorithm for suffix stripping”. Program 14, 130-137. 1980. Reprinted in: Sparck-Jones, K.; Willet, P. (eds.) Readings in Information Retrieval. Morgan Kaufmann, pp. 313-316, 1997.

Salton, G.; Buckley, C. “Term-weighting approaches in automatic text retrieval”. Information Processing and Management 24, 513-523. 1988. Reprinted in: Sparck-Jones, K.; Willet, P. (eds.) Readings in Information Retrieval. Morgan Kaufmann, pp. 323-328, 1997.

Sparck-Jones, K.; Willet, P. (Eds.) Readings in Information Retrieval. Morgan Kaufmann, 1997. [Trec 04] [link] acessado em [08/03/2004].

van Rijsbergen, C.J. Probabilistic retrieval revisited. The Computer Journal, Vol. 35, No. 3, pp. 291-298, 1992.

Wartik, S. “Boolean Operations”. In Information Retrieval: Data Structures and Algorithms. Frakes, W.B.; Baeza-Yates, R. (Eds.), Prentice Hall, pp. 264-292, 1992.

Zha, H.; Simon, H. “On Updating Problems in Latent Semantic Indexing”. SIAM Journal of Scientific Computing, Vol. 21, pp. 782-791, 1999.

Zha, H.; Marques, O.; Simon, H. “A Subspace-Based Model for Information Retrieval with Applications in Latent Semantic Indexing”. IRREGULAR '98, Berkeley, California, USA, Lecturer Notes in Computer Science Nº 1457, Springer Verlag, pp.29-42, 1998.

Zhong, N.; Liu, J.; Yao, Y. “In Search of the Wisdom Web”. IEEE Computer, Vol. 35, Nº 1, pp. 27-31, 2002.