Sign Language Detection and Translator with Learning Platform Usability
Downloads
Even though the area of assistive technologies has just improved significantly, and there have been efforts to achieve the digital inclusiveness of the deaf and the hard-of-hearing community. The deaf and hard-of-hearing population in India have met enormous communication barriers due to lack of awareness and resources. Sign language is one of the necessary tools of communication, that are vital to deaf and hearing-impaired people, however, communication between sign language speakers and none-signers is one of the greatest problems. In this paper, a real-time and multilingual translator and sign language recognition system will be proposed to overcome the communication barrier between deaf and hearing communities using a browser-based system. The suggested system makes use of MediaPipe to extract landmarks of the hand efficiently and a Convolutional Neural Network (CNN) model to recognize gestures. The identified signs have been converted to various languages, with the help of external translation APIs, which allows them to be accessible to all of the language territories like Indian regional languages i.e. Hindi, Marathi, Bengali, Tamil and Punjabi, etc. The first priority has been given to DeepL. The architecture is also designed to use commodity hardware and client-side processing reducing computational overhead and making it scalable to deploy. The system also incorporates a user dashboard on activity tracking and learning analytics where the users are able to track the frequency of sign practice and history of interaction. The experimental outcomes prove that the suggested method is moderately accurate with low latency, which means that it is possible to recognize sign language in real-time on Internet-based platforms without special equipment. The platform inspires the hearing and deaf students to acquire the skills of learning the sign language at a long-term level with the assistance of learning modules that are flexible in the sense of difficulty and analytic-based feedback, recognition, and translation.
https://kaushikenthospital.com/how-common-is-hearing-loss-in-india-key-statistics-what-they-mean/
R. Thakkar, P. Kittur, and A. Munshi, “Deep Learning Based Sign Language Gesture Recognition with Multilingual Translation,” in Proc. Int. Conf. on Computer, Electronics, Electrical Engineering and Applications(IC2E3),2024.
A. Gupta, A. Sawan, S. Singh, and S. Kumari, “Dynamic Sign Language Recognition With Hybrid CNN–LSTM and 1D Convolutional Layers,” in Proc. 11th Int. Conf. on Reliability, Infocom Technologies and Optimization (ICRITO), 2024.
R. Saravanan and S. Veluchamy, “Sign Language Classification With MediaPipe Hand Landmarks,” in Proc. Int. Conf. on Energy, Materials and Communication Engineering (ICEMCE), 2023.
G. M. Rao, P. A. Sujasri, C. Sowmya, S. Anitha, D. Mamatha, and R. Alivela, “Sign Language Recognition Using LSTM and MediaPipe,” in Proc. 7th Int. Conf. on Intelligent Computing and Control Systems (ICICCS), 2023.
R. Raj, R. Sreemathy, J. Jagdale, and M. Turuk, “Real-Time Dynamic Sign Language Recognition with MediaPipe, LSTM, and Dynamic Time Warping,” in Proc. Global Conf. on Information Technology and Communication Networks (GITCON), 2025.
P. Edward, B. S. W. Alexan, and W. Alexan, “Comparative Study Between CNN and LSTM Approaches for Sign Language Recognition,” in Proc. 6th Novel Intelligent and Leading Emerging Sciences Conf. (NILES), 2024.
T. Sharma, S. Upadhyay, V. Kumar, and P. Jain, “Real-Time American Sign Language Translation Using Deep Learning,” in Proc. 3rd Int. Conf. on Disruptive Technologies (ICDT), 2025.
A. R. Agnihotri and D. Arora, “Vision Based Interpreter for Sign Languages and Static Gesture Control Using Convolutional Neural Network,” in Proc. 2nd Int. Conf. on Edge Computing and Applications (ICECAA), 2023.
S. Salim, M. M. A. Jamil, R. Ambar, R. Roslan, and M. G. Kamardan, “Sign Language Digit Detection with MediaPipe and Machine Learning Algorithm,” in Proc. IEEE Int. Conf. on Control System, Computing and Engineering (ICCSCE), 2022.
H. Yoo, I. Goncharenko, and Y. Gu, “Real-Time Dynamic Sign Language Recognition Using LSTM Based on MediaPipe Hand Data,” in 2023 International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan),2023.
https://learnopencv.com/gesture-control-in-zoom-call-using-mediapipe/
