Channel Sensing Order in Multi-User Cognitive Radio Networks
Abstract
This work investigates the problem of channel sensing order used in a multi-user environment, where each user is able to perform sensing on only one channel at a time. We consider a multichannel cognitive network where the probability of each communication channel being available, and the channel capacity, are not known a priori. Thus, becomes necessary a careful ordering of the sequence of channels that will be sense each time and a tradeoff between maximizing the immediate reward, given by choosing the best sequence, and the refinement of the channel statistics, obtained by exploitation of sub-optimal channels. Therefore, we propose and evaluate an approach using reinforcement learning to search dynamically for the optimal sensing order, comparing its performance with other mechanisms, and the results obtained are superior to the other mechanisms in most of the scenarios.
References
Auer, P., Cesa-Bianchi, N., and Fischer, P. (2002). Finite time analysis of the multi-armed bandit problem. Machine Learning, 47.
Berry, D. and Fristedt, B. (1985). Bandit problems: Sequential Allocation of Experiments. Chapman e Hall.
Chow, Y. S., Robbins, H., and Siegmund, D. (1971). Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin Company.
FCC (2003). FCC-03-322 NOTICE OF PROPOSED RULE MAKING AND ORDER. Technical report, Federal Communications Commission.
H. Jiang, L. Lai, R. Fan, and H. V. Poor (2009). Optimal Selection of Channel Sensing Order in Cognitive Radio. IEEE Transactions in Wireless Communications.
Han Han, Jin-long Wang, Qi-hui Wu, and Yu-zhen Huang (2010). Optimal Wideband Spectrum Sensing Order Based on Decision-making Tree in Cognitive Radio. International Conference on Wireless Communications and Signal Processing (WCSP).
Ho Ting Cheng and Weihua Zhuang (2011). Simple Channel Sensing Order in Cognitive Radio Networks. IEEE Journal on Selected Areas in Communications.
J. Jia, Q.Z., and X. Shen (2008). HC-MAC: a Hardware Constrained Cognitive MAC for Efficient Spectrum Management. IEEE Journal on Selected Areas in Communications.
J. Mitola III and G. Q. Maguire Jr. (1999). Cognitive Radio: Making Software Radio more Personal. IEEE Personal Communications, 6(4):13–18.
Jelle R. Kok and Nikos Vlassis (2006). Collaborative Multiagent Reinforcement Learning by Payoff Propagation. J. Mach. Learn. Res., 7:1789–1828.
John Ousterhout (1988). Tcl Tool Command Language. [link].
K. A. Yau, P. Komisarczuk, and P. D. Teal (2010). Enhancing Network Performance in Distributed Cognitive Radio Networks using Single-agent and Multi-agent Reinforcement Learning. In IEEE Conference on Local Computer Networks (LCN).
L. Lai, H. El Gamal, H. Jiang, and H. V. Poor (2007). Cognitive Medium Access: Exploration, Exploitation and Competition. IEEE Transactions on Networking.
Liu, K. and Zhao, Q. (2010). Distributed learning in multi-armed bandit with multiple players. IEEE Transactions on Signal Processing.
Mendes, A. C., Augusto, C. H. P., Silva, M. W. R. d., Guedes, R. M., and Rezende, J. F. d. (2011). Seleção da Ordem de Sensoreamento de Canais em uma Rede Cognitiva Oportunista. In I Workshop de Redes de Acesso de Banda Larga WRA’11.
R. Fan and H. Jiang (2009). Channel sensing order setting in cognitive radio networks: a two user case. IEEE Transactions on Vehicular Technology.
Shu, T. and Krunz, M. (2009). Throughput efficient sequential channel sensing and probing in cognitive radio networks under sensing errors. In MobiCom.
Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: an Introduction. MP.
Vu, H. L. and Sakurai, T. (2006). Collision probability in saturated ieee 802.11 networks. In Australian Telecommunication Networks and Applications Conference.
