A Study about Metrics for Defining the Author Reputation of Web Comments on Products
DOI:
https://doi.org/10.5753/isys.2019.595Keywords:
Author Reputation, Opinion mining, Artificial neural networksAbstract
Knowing the reputation of the author of opinion texts on the Web is of utmost importance for the development of systems based on open data. This paper presents a study on measures used in the process of evaluating the author's reputation on product sales sites. Two experiments were carried out with neural networks Multilayer Perceptron (MLP) and Radial Basis Function (RBF), and the results show that the MLP gave slightly better performance, but not significantly so. In addition, an experiment was carried out to compare the TOP(X) approach, which is used to infer the best comments, with the new approach that uses MLP in the author's reputation dimension. The results showed that the new approach obtained a gain in the classification of the importance of the comments. In addition, a fourth experiment with other machine learning algorithms was performed to observe the behavior of the data.
Downloads
Download data is not yet available.
References
[1] Adler, B. T. and De Alfaro, L. (2007). “A content-driven reputation system for the Wikipedia”. In Proc. of the Int. Conference on World Wide Web, pages 261–270. ACM.
[2] Adler, B. T., De Alfaro, L., and Pye, I. (2010). “Detecting wikipedia vandalism using wikitrust”. Notebooks papers of CLEF.
[3] Aggarwal, A. and Kumaraguru, P. (2014). “Followers or phantoms? an anatomy of purchased twitter followers”. CoRR.
[4] Bruce, R. F. and Wiebe, J. M. (1999). “Recognizing subjectivity: A case study of manual tagging”. Natural Language Engineering, v. 5, p. 187–205.
[5] Cappelletti, R. and Sastry, N. (2012). “IARank: Ranking users on twitter in near real-time, based on their information amplification potential”. In SocialInformatics, pages 70–77.
[6] Carvalho, P., Sarmento, L., Silva, M. J., and de Oliveira, E. (2009). “Clues for detecting irony in user-generated contents: Oh...!! it’s ”so easy”;-)”. In Proc. of the Int. Workshop on Topic-sentiment Analysis for Mass Opinion, pages 53–56.
[7] De Sousa, R. F., Rabelo, R. A. L., and Moura, R. S. (2015). “A fuzzy system-based approach to estimate the importance of online customer reviews”. In International Conference on Fuzzy Systems (FUZZ-IEEE), pages 1–8.
[8] George-Nektarios, T. (2013). "Weka Classifiers Summary", [online] Available: www.academia.eduI5167325IWeka_Classifiers_Summary.
[9] Gonçalves, P., Dalip, D., Reis, J., Messias, J., Ribeiro, F., Melo, P., Araújo, L., Gonçalves, M., and Benevenuto, F. (2015). “Bazinga! caracterizando e detectando sarcasmo e ironia no Twitter”. In Proc. of the Brazilian Workshop on Social Network Analysis and Mining (BraSNAM).
[10] Halfaker, A., Kittur, A., Kraut, R. and Riedl, J. (2009). “A jury of your peers: Quality, experience and ownership in Wikipedia”. In Proc. of the Int. Symposium on Wikis and Open Collaboration, pages 15:1–15:10. ACM.
[11] Hamilton, R., Vohs, K. D. and McGill, A. L. (2014). “We’ll be honest, this won’t be the best article you’ll ever read: The use of dispreferred markers in word-of-mouth communication”. Journal of Consumer Research, 41(1):197 – 212.
[12] Hartmann, N., Avanço, L., Balage, P., Duran, M., Nunes, M., Pardo, T. and Aluísio, S. (2014). “A large corpus of product reviews in portuguese: Tackling out-of-vocabulary words”. In Proc. of the Int. Conference on Language Resources and Evaluation (LREC’14).
[13] Hoisl B., Aigner W., Miksch S. (2007) “Social Rewarding in Wiki Systems – Motivating the Community”. In: Int. Conf. on Online Communities and Social Computing. Springer.
[14] Jackson, P. and Moulinier, I. (2007). “Natural language processing for online applications: Text retrieval, extraction and categorization”. John Benjamins, Amsterdam.
[15] Jones, C., Hesterly, W. S. and Borgatti, S. P. “A general theory of network governance: Exchange conditions and social mechanisms”. The Academy of Management Review, Academy of Management, v. 22, n. 4, p. 911–945, 1997.
[16] Kohavi, R. (1995). “The Power of Decision Tables”. In: 8th European Conference on Machine Learning,174-189, 1995.
[17] Ku, L.-W., Liang, Y.-T. and Chen, H.-H. (2006) “Opinion extraction, summarization and tracking in news and blog Corpora”. In: Proceedings of AAAI-CAAW-06, the Spring Symposia on Computational Approaches to Analyzing Weblogs.
[18] Kwak, H., Lee, C., Park, H. and Moon, S. (2010). “What is Twitter, a social network or a news media?” In Proc. of the Int. Conf. on World Wide Web. ACM.
[19] Li, B., Li, RH, King, I., Lyu MR and Yu, JX (2014). “A topic-biased user reputation model in rating systems”. Knowledge and Information Systems.
[20] Li, M., Huang, L., Tan, C. and Wei, K. (2013). “Helpfulness of online product reviews as seen by consumers: Source and content features”. Int. J. Electronic Commerce, 17(4):101–136.
[21] Liu, B. (2011). “Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data”. Springer-Verlag New York, Inc., Secaucus, NJ, USA.
[22] Mackay, D. (1998). “Introduction to Gaussian Processes”. Dept. of Physics, Cambridge University, UK.
[23] Pang, B., Lee, L. and Vaithyanathan, S. (2002) “Thumbs up?: Sentiment classification using machine learning techniques”. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing.
[24] Powers, D. M. W. (2011). “Evaluation: From precision, recall and f-measure to roc., informedness, markedness & correlation”. Journal of Machine Learning Tech- nologies, 2(1):37–63.
[25] Shevade, S., Keerthi, S., Bhattacharyya, C., Murthy, K. (1999). “Improvements to the SMO Algorithm for SVM Regression”. In: IEEE Transactions on Neural Networks.
[26] SPSS (2007). SPSS for Windows, Version 16.0. Chicago USA: [s.n.].
[27] Wang, A. H. (2010). “Don’t follow me: Spam detection in Twitter”. In Proc. of the Int. Conference on Security and Cryptography (SECRYPT), pages 1–10.
[28] Weitzel, L., de Oliveira, J. P. and Quaresma, P. (2014). “Measuring the reputation in user-generated-content systems based on health information”. Procedia Computer Science, 29:364 – 378.
[29] Wiebe, J., Wilson, T. and Cardie, C. (2005) “Annotating expressions of opinions and emotions in language”. In: Language Resources and Evaluation, 39 2-3.
[30] Weng, J., Lim, E.-P., Jiang, J. and He, Q. (2010). “Twitterrank: Finding topic-sensitive influential Twitterers”. In Proc. of the Int. Conference on Web Search and Data Mining, pages 261–270. ACM.
[31] Wöhner, T., Köhler, S., and Peters, R. (2011). “Automatic reputation assessment in Wikipedia”. In Proc. of the Int. Conference on Information Systems.
[32] Zhao, H., Ye, S., Bhattacharyya, P., Rowe, J., Gribble, K. and Wu, S. F. (2010). “Socialwiki: Bring order to wiki systems with social context”. In: Int. Conf. on Social Informatics.
[2] Adler, B. T., De Alfaro, L., and Pye, I. (2010). “Detecting wikipedia vandalism using wikitrust”. Notebooks papers of CLEF.
[3] Aggarwal, A. and Kumaraguru, P. (2014). “Followers or phantoms? an anatomy of purchased twitter followers”. CoRR.
[4] Bruce, R. F. and Wiebe, J. M. (1999). “Recognizing subjectivity: A case study of manual tagging”. Natural Language Engineering, v. 5, p. 187–205.
[5] Cappelletti, R. and Sastry, N. (2012). “IARank: Ranking users on twitter in near real-time, based on their information amplification potential”. In SocialInformatics, pages 70–77.
[6] Carvalho, P., Sarmento, L., Silva, M. J., and de Oliveira, E. (2009). “Clues for detecting irony in user-generated contents: Oh...!! it’s ”so easy”;-)”. In Proc. of the Int. Workshop on Topic-sentiment Analysis for Mass Opinion, pages 53–56.
[7] De Sousa, R. F., Rabelo, R. A. L., and Moura, R. S. (2015). “A fuzzy system-based approach to estimate the importance of online customer reviews”. In International Conference on Fuzzy Systems (FUZZ-IEEE), pages 1–8.
[8] George-Nektarios, T. (2013). "Weka Classifiers Summary", [online] Available: www.academia.eduI5167325IWeka_Classifiers_Summary.
[9] Gonçalves, P., Dalip, D., Reis, J., Messias, J., Ribeiro, F., Melo, P., Araújo, L., Gonçalves, M., and Benevenuto, F. (2015). “Bazinga! caracterizando e detectando sarcasmo e ironia no Twitter”. In Proc. of the Brazilian Workshop on Social Network Analysis and Mining (BraSNAM).
[10] Halfaker, A., Kittur, A., Kraut, R. and Riedl, J. (2009). “A jury of your peers: Quality, experience and ownership in Wikipedia”. In Proc. of the Int. Symposium on Wikis and Open Collaboration, pages 15:1–15:10. ACM.
[11] Hamilton, R., Vohs, K. D. and McGill, A. L. (2014). “We’ll be honest, this won’t be the best article you’ll ever read: The use of dispreferred markers in word-of-mouth communication”. Journal of Consumer Research, 41(1):197 – 212.
[12] Hartmann, N., Avanço, L., Balage, P., Duran, M., Nunes, M., Pardo, T. and Aluísio, S. (2014). “A large corpus of product reviews in portuguese: Tackling out-of-vocabulary words”. In Proc. of the Int. Conference on Language Resources and Evaluation (LREC’14).
[13] Hoisl B., Aigner W., Miksch S. (2007) “Social Rewarding in Wiki Systems – Motivating the Community”. In: Int. Conf. on Online Communities and Social Computing. Springer.
[14] Jackson, P. and Moulinier, I. (2007). “Natural language processing for online applications: Text retrieval, extraction and categorization”. John Benjamins, Amsterdam.
[15] Jones, C., Hesterly, W. S. and Borgatti, S. P. “A general theory of network governance: Exchange conditions and social mechanisms”. The Academy of Management Review, Academy of Management, v. 22, n. 4, p. 911–945, 1997.
[16] Kohavi, R. (1995). “The Power of Decision Tables”. In: 8th European Conference on Machine Learning,174-189, 1995.
[17] Ku, L.-W., Liang, Y.-T. and Chen, H.-H. (2006) “Opinion extraction, summarization and tracking in news and blog Corpora”. In: Proceedings of AAAI-CAAW-06, the Spring Symposia on Computational Approaches to Analyzing Weblogs.
[18] Kwak, H., Lee, C., Park, H. and Moon, S. (2010). “What is Twitter, a social network or a news media?” In Proc. of the Int. Conf. on World Wide Web. ACM.
[19] Li, B., Li, RH, King, I., Lyu MR and Yu, JX (2014). “A topic-biased user reputation model in rating systems”. Knowledge and Information Systems.
[20] Li, M., Huang, L., Tan, C. and Wei, K. (2013). “Helpfulness of online product reviews as seen by consumers: Source and content features”. Int. J. Electronic Commerce, 17(4):101–136.
[21] Liu, B. (2011). “Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data”. Springer-Verlag New York, Inc., Secaucus, NJ, USA.
[22] Mackay, D. (1998). “Introduction to Gaussian Processes”. Dept. of Physics, Cambridge University, UK.
[23] Pang, B., Lee, L. and Vaithyanathan, S. (2002) “Thumbs up?: Sentiment classification using machine learning techniques”. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing.
[24] Powers, D. M. W. (2011). “Evaluation: From precision, recall and f-measure to roc., informedness, markedness & correlation”. Journal of Machine Learning Tech- nologies, 2(1):37–63.
[25] Shevade, S., Keerthi, S., Bhattacharyya, C., Murthy, K. (1999). “Improvements to the SMO Algorithm for SVM Regression”. In: IEEE Transactions on Neural Networks.
[26] SPSS (2007). SPSS for Windows, Version 16.0. Chicago USA: [s.n.].
[27] Wang, A. H. (2010). “Don’t follow me: Spam detection in Twitter”. In Proc. of the Int. Conference on Security and Cryptography (SECRYPT), pages 1–10.
[28] Weitzel, L., de Oliveira, J. P. and Quaresma, P. (2014). “Measuring the reputation in user-generated-content systems based on health information”. Procedia Computer Science, 29:364 – 378.
[29] Wiebe, J., Wilson, T. and Cardie, C. (2005) “Annotating expressions of opinions and emotions in language”. In: Language Resources and Evaluation, 39 2-3.
[30] Weng, J., Lim, E.-P., Jiang, J. and He, Q. (2010). “Twitterrank: Finding topic-sensitive influential Twitterers”. In Proc. of the Int. Conference on Web Search and Data Mining, pages 261–270. ACM.
[31] Wöhner, T., Köhler, S., and Peters, R. (2011). “Automatic reputation assessment in Wikipedia”. In Proc. of the Int. Conference on Information Systems.
[32] Zhao, H., Ye, S., Bhattacharyya, P., Rowe, J., Gribble, K. and Wu, S. F. (2010). “Socialwiki: Bring order to wiki systems with social context”. In: Int. Conf. on Social Informatics.
Downloads
Published
2019-09-18
How to Cite
Sá, C. A., & Moura, R. S. (2019). A Study about Metrics for Defining the Author Reputation of Web Comments on Products. ISys - Brazilian Journal of Information Systems, 12(3), 6–23. https://doi.org/10.5753/isys.2019.595
Issue
Section
Extended versions of selected articles
