Revising Bayesian Networks through the Introduction of Latent Variables
Abstract
An important issue on learning Bayesian networks is how to effectively learn their structure in the presence of hidden variables. In this paper, we propose a new approach based on theory revision. Our algorithm searches for the best place to introduce a hidden variable guided by the examples and through the use of a discriminative approach. The hidden variables are included through a revision operator defined in this paper. Moreover, our algorithm is capable of introducing as many hidden variables as necessary to improve the performance of the Bayesian network and it can be applied even on sparse Bayesian networks. We successfully evaluated our algorithm on 12 real datasets.References
Binder, J., Koller, D., Russell, S., and Kanazawa, K. (1997). Adaptive probabilistic networks with hidden variables. Machine Learning, 29:213–244.
Buntine, W. (1991). Theory refinement on bayesian networks. In Proceedings Seventeenth Conference Uncertainty in Artificial Intelligence, pages 52–60, San Mateo, CA.
Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Royal Stat Soc, 39:1–39.
Elidan, G. and Friedman, N. (2005). Learning hidden variable networks: The information bottleneck approach. Journal of Machine Learning Research, 6:81–127.
Elidan, G., Lotner, N., Friedman, N., and Koller, D. (2000). Discovering hidden variables: a structure-based approach. In Neural Information Processing Systems, volume 13, pages 479–485.
Friedman, N. (1998). The bayesian structural EM algorithm. In UAI, pages 129–138.
Friedman, N., Geiger, D., and Goldszmidt, M. (1997). Bayesian network classifiers. Machine Learning, 29:131–163.
Grossman, D. and Domingos, P. (2004). Learning bayesian network classifiers by maximizing conditional likelihood. In Proc. 21th Int. Conference on Machine Learning, pages 361–368.
Heckerman, D. (1995). A tutorial on learning bayesian networks. Technical report, Microsoft Research.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence(IJCAI), pages 1137–1145.
Lauritzen, S. L. (1995). The em algorithm for graphical association models with missing data. Comp. Stat.and Data Ana., 19:191–201.
Nadeau, C. and Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52(3):239–281.
Ramachandran, S. and Mooney, R. (1998). Theory refinement of bayesian networks with hidden variables. In Proc. 15th Int. Conference on Machine Learning, pages 454–462.
Wrobel, S. (1996). First-order theory refinement. Advances in Inductive Logic Programming, pages 14–33.
Buntine, W. (1991). Theory refinement on bayesian networks. In Proceedings Seventeenth Conference Uncertainty in Artificial Intelligence, pages 52–60, San Mateo, CA.
Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Royal Stat Soc, 39:1–39.
Elidan, G. and Friedman, N. (2005). Learning hidden variable networks: The information bottleneck approach. Journal of Machine Learning Research, 6:81–127.
Elidan, G., Lotner, N., Friedman, N., and Koller, D. (2000). Discovering hidden variables: a structure-based approach. In Neural Information Processing Systems, volume 13, pages 479–485.
Friedman, N. (1998). The bayesian structural EM algorithm. In UAI, pages 129–138.
Friedman, N., Geiger, D., and Goldszmidt, M. (1997). Bayesian network classifiers. Machine Learning, 29:131–163.
Grossman, D. and Domingos, P. (2004). Learning bayesian network classifiers by maximizing conditional likelihood. In Proc. 21th Int. Conference on Machine Learning, pages 361–368.
Heckerman, D. (1995). A tutorial on learning bayesian networks. Technical report, Microsoft Research.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence(IJCAI), pages 1137–1145.
Lauritzen, S. L. (1995). The em algorithm for graphical association models with missing data. Comp. Stat.and Data Ana., 19:191–201.
Nadeau, C. and Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52(3):239–281.
Ramachandran, S. and Mooney, R. (1998). Theory refinement of bayesian networks with hidden variables. In Proc. 15th Int. Conference on Machine Learning, pages 454–462.
Wrobel, S. (1996). First-order theory refinement. Advances in Inductive Logic Programming, pages 14–33.
Published
2009-07-20
How to Cite
REVOREDO, Kate; PAES, Aline; ZAVERUCHA, Gerson; COSTA, Vitor Santos.
Revising Bayesian Networks through the Introduction of Latent Variables. In: NATIONAL MEETING ON ARTIFICIAL AND COMPUTATIONAL INTELLIGENCE (ENIAC), 7. , 2009, Bento Gonçalves/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2009
.
p. 422-431.
ISSN 2763-9061.
