|
|
|
| DBN Model Based Multi-stream Asynchrony Triphone for Audio-Visual Speech Recognition and Phone Segmentation |
| Lü Guo-yun①; Jiang Dong-mei①; Fan Yang-yu①;Zhao Rong-chun①; H. Sahli②; W. Verhelst② |
| ①Northwestern Polytechnical University, Xi’an 710072, China;②Department ETRO, Vrije Universiteit Brussel, Brussel, B-1050, Belgium |
|
|
|
|
Abstract In this paper, a novel Multi-stream Multi-states Asynchronous Dynamic Bayesian Network based context-dependent TRIphone (MM-ADBN-TRI) model is proposed for audio-visual speech recognition and phone segmentation. The model looses the asynchrony of audio and visual stream to the word level. Both in audio stream and in visual stream, word-triphone-state topology structure is used. Essentially, MM-ADBN-TRI model is a triphone model whose recognition basic units are triphones, which captures the variations in real continuous speech spectra more accurately. Recognition and segmentation experiments are done on continuous digit audio-visual speech database, and results show that: MM-ADBN-TRI model obtains the best overall performance in word accuracy and phone segmentation results with time boundaries, and more reasonable asynchrony between audio and visual speech.
|
|
Received: 23 July 2007
|
|
|
|
|
|
|
|
|
2009, Vol. 31 
