Journal Published Online: 23 May 2019
Volume 47, Issue 6

Audio–Visual (Multimodal) Speech Recognition System Using Deep Neural Network

CODEN: JTEVAB

Abstract

Recognizing the voice of a person by a smart device has become the need of the hour. Our cell phones are equipped with such features that converts speech to text, receives a command as a voice instruction instead of screen touch or pressing of a button. There are numerous intelligent machines such as automatic cars, robots, smart phones that use speech recognition features. These features are also very useful for people who are disabled and cannot use the smart devices with their hands. There are numerous techniques available to implement a speech recognition system. The need of such a system is to accurately recognize human voice at any circumstances, because the accent of each human differs for a single language, and also, the human voice is never devoid of noise and other emotional elements. So, the aim of such systems is to recognize human voices of different accents, even in a noisy environment. Because of this, an effective and robust speech recognition system has become important for machine and human interaction. Much voice-recognizing software depends purely on audio to recognize a voice, and one cannot guarantee that the audio will be noise free. In order to improve the accuracy and performance of traditional speech recognition system in a noisy environment, a new, nontraditional approach called a multimodal speech recognition system that uses both the audio and video features such as lip movement facial expression to recognize a speech is used. This article discusses the advantage of using video features along with the audio features with an experiment that proves it.

Author Information

Paulin, Hebsibah
Department of Computer Science, R.V. Govt Arts College, Chengalpattu and Bharathiar University, Chengalpattu, Tamil Nadu, India
Milton, R. S.
Department of Computer Science and Engineering, Sri Sivasubramaniya Nadar College of Engineering, Chennai, India
JanakiRaman, S.
Department of Banking and Information Technology, Pondicherry University, Kalapet, Pondicherry, India
Chandraprabha, K.
JPJ Lending Library, Kanagachettikulam, Puducherry, India
Pages: 12
Price: $25.00
Related
Reprints and Permissions
Reprints and copyright permissions can be requested through the
Copyright Clearance Center
Details
Stock #: JTE20180505
ISSN: 0090-3973
DOI: 10.1520/JTE20180505