Stochastic Logics with Two-dimensional State Transfer Structure and Its Application in the Artificial Neural Network
JI Yuan①② CHEN Wendong① RAN Feng①② ZHANG Jinyi① David LILJA③
①(Microelectronic Research and Development Center, Shanghai University, Shanghai 200072, China) ②(School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200072, China) ③(Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis 55455, USA)
Stochastic computing is a special algorithm that performs mathematical operations with probabilistic values of bit streams rather than traditional deterministic values. The main advantage of stochastic computing is its great simplicity of hardware arithmetic units for mathematical operations to reduce the circuit cost. This paper discusses the principle of the stochastic computing and its main arithmetic logic. It analyzes a two-dimension state transition topology structure, and discusses the Gaussian function implementation method based on the two-dimension Finite State Machin (FSM). Then, a low cost stochastic radial basis function neural network model is proposed. Results from two pattern recognition tests show that the difference of the mean squared error between the stochastic network output value and the corresponding deterministic network output value can be less than 1.3%. FPGA implementation results show that the hardware resource requirement of the proposed stochastic hidden neuron is only 1.2% of the corresponding deterministic hidden neuron with the interpolated look-up table, and is 2.0% of the CORDIC algorithm. The accuracy, speed and power of the stochastic network can be tradeoff dynamically. This network is suitable for the low cost and low power applications like embedded, portable and wearable devices.
季渊,陈文栋,冉峰,张金艺,David LILJA. 具有二维状态转移结构的随机逻辑及其在神经网络中的应用[J]. 电子与信息学报, 2016, 38(8): 2099-2106.
JI Yuan, CHEN Wendong, RAN Feng, ZHANG Jinyi, David LILJA. Stochastic Logics with Two-dimensional State Transfer Structure and Its Application in the Artificial Neural Network. JEIT, 2016, 38(8): 2099-2106.
GAINES B R. Stochastic Computing Systems (Chapters) in Advances in Information Systems Science[M]. New York: Plenum, 1969: 37-172.
[2]
HAYES J P. Introduction to stochastic computing and its challenges[C]. 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC), San Francisco, CA, USA, 2015: 1-3. doi: 10.1145/2744769.2747932.
[3]
ALAGHI A and HAYES J P. Survey of stochastic computing[J]. ACM Transactions on Embedded Computing Systems, 2013, 12(2s): 1-19. doi: 10.1145/2465787.2465794.
[4]
MOONS B and VERHELST M. Energy-efficiency and accuracy of stochastic computing circuits in emerging technologies[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2014, 4(4): 475-486. doi: 10.1109/JETCAS.2014.2361070.
[5]
BROWN B D and CARD H C. Stochastic neural computation. I. Computational elements[J]. IEEE Transactions on Computers, 2001, 50(9): 891-905. doi: 10.1109/12.954505.
[6]
QIAN Weikang, LI Xin, RIEDEL M D, et al. An architecture for fault-tolerant computation with stochastic logic[J]. IEEE Transactions on Computers, 2011, 60(1): 93-105. doi: 10.1109/TC.2010.202.
[7]
HAN Jie, CHEN Hao, LIANG Jinghang, et al. A stochastic computational approach for accurate and efficient reliability evaluation[J]. IEEE Transactions on Computers, 2014, 63(6): 1336-1350. doi: 10.1109/TC.2012.276.
[8]
ALAWAD M and LIN Mingjie. FIR filter based on stochastic computing with reconfigurable digital fabric[C]. 2015 IEEE 23rd Annual International Symposium on Field- Programmable Custom Computing Machines (FCCM), Vancouver, BC, Canada, 2015: 92-95. doi: 10.1109/FCCM. 2015.32.
[9]
TEHRANI S S, NADERI A, KAMENDJE G A, et al. Majority-based tracking forecast Memories for Stochastic LDPC Decoding[J]. IEEE Transactions on Signal Processing, 2010, 58(9): 4883-4896. doi: 10.1109/TSP.2010.2051434.
[10]
LI Peng, LILJA D J, QIAN Weikang, et al. Computation on stochastic bit streams digital image processing case studies[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2014, 22(3): 449-462. doi: 10.1109/TVLSI.2013. 2247429.
[11]
ZHANG Da and LI Hui. A stochastic-based FPGA controller for an induction motor drive with integrated neural network algorithms[J]. IEEE Transactions on Industrial Electronics, 2008, 55(2): 551-561. doi: 10.1109/TIE.2007.911946.
WANG Shoujue, LI Zhaozhou, CHEN Xiangdong, et al. Discussion on the basic mathematical models of neurons in general purpose neurocomputer[J]. Acta Electronica Sinica, 2001, 29(5): 576-580.
WU Dapeng, ZHAO Ying, XIONG Yu, et al. Alarm information relevance mining mechanism based on wavelet neural network[J]. Journal of Electronics & Information Technology, 2014, 36(10): 2379-2384. doi: 10.3724/SP.J.1146. 2013.01701.
[14]
BROWN B D and CARD H C. Stochastic neural computation. II. Soft competitive learning[J]. IEEE Transactions on Computers, 2001, 50(9): 906-920. doi: 10.1109/12.954506.
[15]
LI Peng, LILJA D J, QIAN W K, et al. The synthesis of complex arithmetic computation on stochastic bit streams using sequential logic[C]. 2012 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), San Jose, CA, USA, 2012: 480-487. doi: 10.1145/2429384.2429483.
[16]
JI Yuan, RAN Feng, MA Cong, et al. A hardware implementation of a radial basis function neural network using stochastic logic[C]. 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE), Grenoble, France, 2015: 880-883.
MA Chengguang, ZHONG Shunan, LILJA D J, et al. Analysis method of stochastic computing system based on hypergeometric decomposition[J]. Journal of Electronics & Information Technology, 2013, 35(2): 355-360. doi: 10.3724/ SP.J.1146.2012.00711.
