Resource Allocation for Heterogeneous Wireless Networks: A Robust Layered Game Learning Solutions
SHAO Hongxiang①④ ZHAO Hangsheng② SUN Youming③ SUN Fenggang①
①(College of Communications Engineering, PLA University of Science and Technology, Nanjing 210007, China) ②(Nanjing Telecommunication Technology Institute, Nanjing 210007, China) ③(Institute of Information System Engineering, PLA Information Engineering University, Zhengzhou 450000, China) ④(Luoyang Institute of Science and Technology, Luoyang 471023, China)
This paper investigates a resource allocation scheme in heterogeneous wireless small cell networks with imperfect Channel State Information (CSI). In this work, the math expression for the stochastic dynamic uncertainty in CSI is proposed for model analysis and the robust Stackelberg game model with various interference power constraints is established firstly. Then, the Stackelberg game Equilibrium (SE) is obtained and analyzed. Lastly, an improved hierarchical Q-learning algorithm is also given to search the Stackelberg equilibrium strategies of macro-cell base station and small-cell base station. Both theoretical analysis and simulation results verify the proposed scheme can effectively restrain declining revenue due to incomplete CSI and the proposed algorithms can improves the convergence rate, especially applicable to the fast varying communication environment.
FU Fengchao, ZHANG Zhicai, LU Zhaoming, et al. Energy- efficient power control algorithm based on Stackelberg game in two-tier femtocell Networks[J]. Journal of University of Electronic Science and Technology of China, 2015, 44(3): 363-368.
[4]
LASHGARI M, MAHAM B, KEBRIAEI H, et al. Distributed power allocation and interference mitigation in two-tier femtocell networks: A game-theoretic approach[C]. Wireless Communications and Mobile Computing Conference, Dubrovnik, Croatia, 2015: 55-60.
[5]
DUONG N D, MADHUKUMAR A S, and NIYATO D. Stackelberg Bayesian game for power allocation in two-tier networks[J]. IEEE Transactions on Vehicular Technology, 2016, 65(4): 2341-2354. doi: 10.1109/TVT.2015.2418297.
[6]
ZHU Kun, HOSSAIN E, and ANPALAGAN A. Downlink power control in two-tier cellular OFDMA networks under uncertainties: A robust Stackelberg game[J]. IEEE Transactions on Communications, 2015, 63(2): 520-535. doi: 10.1109/TCOMM.2014.2382095.
[7]
吴敏, 何勇. 鲁棒控制理论[M]. 北京: 高等教育出版社, 2010.
[8]
ZHANG H, VENTURINO L, PRASAD N, et al. Weighted sum-rate maximization in multi-cell networks via coordinated scheduling and discrete power control[J]. IEEE Journal on Selected Areas in Communications, 2011, 29(6): 1214-1224. doi: 10.1109/JSAC.2011.110609.
[9]
YANG K, WU Y, and HUANG J. Distributed robust optimization for communication networks[C]. IEEE Infocom Conference, Phoenix, AZ, USA, 2008: 1157-1165. doi: 10.1109/ INFOCOM.2008.171.
[10]
FUDENBURG D and TIROLE J. Game Theory[M]. Cambridge, MA, USA, The MIT Press, 1991: 29-34.
[11]
CHEN X, ZHANG H, CHEN T et al. Improving energy efficiency in femtocell networks: A hierarchical reinforcement learning framework[C]. IEEE International Conference on Communications (ICC), Budapest, Hungary, 2013: 2241- 2245. doi: 10. 1109/ICC.2013.6654861.
[12]
WATKINS C and DAYAN P. Q-learning[J]. Journal of Machine Learning Research, 1992, 8(1): 279-292.
