It is estimated that more than 9% of the Russian population is hearing impaired, and the development of dactyl recognition systems is becoming critical to facilitate their social communication. The introduction of dactyl recognition systems will improve communication for these people, providing them with equal opportunities and improving their quality of life. The research focused on learning the characters of the dactyl alphabet, as well as developing a labeled dataset and training a neural network for gesture recognition. The aim of the work is to create tools capable of recognizing the signs of the Russian dactyl alphabet. Within the framework of this research the method of computer vision was applied. The process of gesture recognition consists of the following steps: first the camera captures the video stream, after the images of hands are preprocessed. Then a pre-trained neural network analyzes these images and extracts important features. Next, gesture classification takes place, where the model determines whether the sign belongs to a certain letter of the alphabet. Finally, the recognition results are interpreted into a suitable symbol associated with the gesture. During the research process, the signs of the dactyl alphabet and interaction features of people with auditory impairment were studied and a dataset of more than 25000 trained data was also created. A model was developed and trained based on the most appropriate architecture for the task of the work. The model was tested and optimized to improve its accuracy. The results of this work can be used in the creation of devices to compensate for poor hearing, providing people with hearing impairment comfort in society.

Keywords: computer vision, sign recognition, dactyl classification, transfer learning, Russian dactyl alphabet, deep learning, computerization, software, assistive technology, convolutional neural networks