Patients with speech disorders often find it difficult to communicate their needs to the general audience. These patients include the mute, senior citizens, paralyzed, and patients with diseases such as dysarthria, aphasia, to name a few. To satisfy their requirements, the Gesture Gloves have been designed. These gloves ease the communication, without much ado, by engendering predefined gestures to voice. The input is in the form of hand gestures which are converted to text and speech. The gloves are equipped with multiple flex sensors that produce varying resistance for every gesture made by the person.

1. Gesture Gloves
1
Neha Udeshi, 2
Yashvi Upadhyaya, 3
Gargi Vyas, 4
Maitri Thaker,
*
Prof. Mitchell D’Silva
1, 2, 3, 4
Student, *Faculty
Department of Information Technology,
Dwarkadas J. Sanghvi College of Engineering, Mumbai, India
1
udeshineha9@gmail.com, 2
yashviu@gmail.com, 3
maitrithaker2@gmail.com,
4
gargi.vyas@gmail.com, *
Mitchell.Dsilva@djsce.ac.in
Abstract
Patients with speech disorders often find it difficult to communicate their needs to the
general audience. These patients include the mute, senior citizens, paralyzed, and patients
with diseases such as dysarthria, aphasia, to name a few. To satisfy their requirements,
the Gesture Glove have been designed. These gloves ease the communication, without
much ado, by engendering predefined gestures to voice. The input is in the form of hand
gestures which is converted to text and speech. The gloves are equipped with multiple flex
sensors which produce varying resistance for every gesture made by the person.
Keywords: Hand Gesture, Assistive Technology, Signal Processing, Speech-impaired
People, Sign Language, Communication Process, Flex Sensor Resistance, Arduino,
Gesture
1. Introduction
Patients with speech disorders live an extremely unfortunate life, being unable to
communicate their thoughts to others. Such patients often find difficult to convey
what they require at a particular moment. This makes it impossible for the person
assisting them to understand what help is needed by the patient. The patient might
be hungry, thirsty or may be suffering from an asthma attack . In such circumstances,
if the objects of reference are out of sight, he or she cannot point out what is
required. Currently, bells are being used to call for help and the attendant deciphers
the patient’s needs by providing them a list of possible options and acting upon the
option with a positive response. This process increases the time of action, thus,
decreasing the efficiency and effectiveness . There is a need for a portable electronic
device which acts as an interpreter for these patients. The device should facilitate
easy communication. For this very purpose, the authors propose the system named
Gesture Gloves. These gloves are equipped internally with flex sensors. The system
uses a microcontroller for computing, bluetooth module for wireless connectivity
and an android application for providing the user interface.
2. Literature Survey
Researchers have found solutions to the problem. Ahmed et al [1] proposed a
hand glove to translate hand gestures into sound command using AVR
ATMEGA32L. Satpute et al [2] developed a data glove using PIC18F4620 for the
same purpose by playing recorded audio corresponding to the gesture of hand. Wald
[3] developed software for editing automatic speech recognition in real-time for
hearing impaired. Itkarkar et al [4] proposed a system which translates hand
gestures into speech using MATLAB. Zhao et al [5] created a five-fingered
prosthetic hand system. Praveen Kumar et al [6] developed a wireless glove that
International Journal of Management, Technology And Engineering
Volume IX, Issue XI, NOVEMBER/2019
ISSN NO : 2249-7455
Page No: 62

2. translates sign language into speech. Speak up[7], a glove developed to register
hand gestures and play pre recorded audio along with textual display on LCD to
enable the speechless communicate with normal people. Bhatti et al [8] developed a
hand glove with the support of text on LCD display via computer interface with
PIC18F8680 microcontroller having DC power supply instead of a battery. All the
above systems use an LCD screen or a speaker along with microcontroller to display
the output, resulting from different degree of bends observed in the flex sensors. A
major drawback of these systems is that they require a lot of wired connections to
display the output. This makes the system less portable.
3. Methodology
Since the glove uses flex sensors, a major drawback that exists in all the existing
systems is that the user needs to be conscious about their finger bends. There can be
unnecessary outputs in such systems as the patient may unconsciously bend their
fingers while resting. Therefore it becomes necessary to make the system
considering the natural bends of the fingers and the bends when the patient is
resting. The joints of the finger include metacarpophalangeal joint (MCP), proximal
interphalangeal joint (PIP) and distal interphalangeal joint (DIP) and the angles that
these joints make are called flexion angles [9]. The glove proposed would measure
and detect the flexion angle at the PIP joint. To determine the flexion angle at PIP
when the patient is resting or sleeping, a research conducted amongst 10 people of
different age groups. Pictures of their fingers were taken. From this it was
concluded that naturally, the fingers are never straight or completely curled while
the thumb almost straight . All these bend angles fell in a range between 30 degrees
to 50 degrees. Thus, for our system a threshold angle of 60 degrees was chosen,
beyond which all the user commands will be considered and the output will be
given.
Fig 1.Finger joints Fig 2. Flexion angle at PIP
4. Proposed System
In order to improve portability and reduce the wiring, an alternative approach is
proposed. The proposed system uses 5 flex sensors attached to a glove. These sensors and
connected through wires to Arduino microcontroller. Finally, to display the output
wirelessly, an android application is being used. The digital output from the
microcontroller is transmitted to the application via a bluetooth module. One advantage of
using an android app is that in case of emergency multiple people can be informed
simultaneously. Next, the architecture of the proposed system is presented.
International Journal of Management, Technology And Engineering
Volume IX, Issue XI, NOVEMBER/2019
ISSN NO : 2249-7455
Page No: 63

3. 5. System Architecture
The proposed system consists of flex sensors, an arduino microcontroller, a bluetooth
module and an android application. Each component has its own functionality. This can
be understood better by the architecture shown in Figure 3.
Fig 3. System Architecture
6. System Components
6.1 Flex Sensor
The flex sensor is a thin and flexible carbon substrate sensor which changes its
resistance proportional to its inward bend, as shown in the diagram. The resistance varies
from approximately 10KΩ to 50KΩ. Flex sensors are thin and also light weight, thus can
be attached to the gesture sensing glove. The feature of the flex sensor are given in the
table below. Flex sensors used for this are 0.28 inch wide and 3 inch long as only flexion
angle at PIP is to be calculated. The resistance ranges between 1.5 to 40 K-ohms
corresponding to the voltage range of 5 to 12 Volts.
6.2 Microcontroller
Arduino Uno (ATmega328P) – This belongs to the 8 bit AVR microcontroller family.
This is lightweight and has low power consumption supported by five software selectable
power saving modes which makes is suitable for a battery driven portable gesture glove. It
has high speed computation supported by 32KB flash memory and 2KB SRAM. [10]
6.3 Bluetooth Module
Bluetooth Module HC-05 is a Bluetooth serial port protocol used for wireless serial
connection. This module is used to wirelessly transmit data from the glove to the android
application.
6.4 Android Application
Arduino Bluetooth Text to Speech developed by Color Lab was used to convert the
text received to speech. Arduino Bluetooth CH-05, CH-06 can communicate with the app
by sending the text. This module gives the textual and audio output of the digital signal
transmitted by the Bluetooth module.
International Journal of Management, Technology And Engineering
Volume IX, Issue XI, NOVEMBER/2019
ISSN NO : 2249-7455
Page No: 64

4. 7. Working
Fig 4. Circuit Diagram
The system consists of a glove with 5 flex sensors attached individually to each finger.
For convenience, 3 flex sensors have been used during implementation. Further, the
connections are made to the microcontroller which is attached to the glove positioned at
the backside of the palm. The connections are made as shown in the circuit diagram .
There will be a predefined table of words corresponding to the binary code. There will be
only one value corresponding to the bends of the flex sensor. This is done to prevent
unnecessary outputs corresponding to small bends in the finger. Bend angle is calculated
using the voltage and resistance of the flex sensors and if greater than 60 degrees then
only output is generated. Using the microcontroller,which also performs the required
signal conditioning, the digital output is transmitted to the android application via
bluetooth. The android app voices the audio and displays text, corresponding to the
digital string.
Table 1. Reference Table
Gesture Output String
Index Finger “Pardon”
Middle Finger “Yes”
Ring Finger “No”
Index Finger + Middle Finger “Medicine”
Index Finger + Ring Finger “Food”
Middle Finger + Ring Finger “Water”
Index Finger + Middle Finger + Ring Finger “Help”
International Journal of Management, Technology And Engineering
Volume IX, Issue XI, NOVEMBER/2019
ISSN NO : 2249-7455
Page No: 65

5. Fig 5. Implementation
8. Results
A table of reference [Table 1] for the patients was created where they could see the
sign and its corresponding message. With this, the patients were asked to use the glove to
communicate. It was seen that there was effective communication between the patient and
the attendant. Also after using the glove for multiple times, the patient gets acquainted
with the signs and did not refer to the table. An overall improvement in terms of speed
and clarity in the communication was seen.
Fig 6. Android Application Results
International Journal of Management, Technology And Engineering
Volume IX, Issue XI, NOVEMBER/2019
ISSN NO : 2249-7455
Page No: 66

6. 9. Future Scope
It is said that there is always scope for improvement, nothing is perfect. To enhance
this project, memory could be used. A database of recorded human voice can be
integrated with this system which would be heard with ease and clarity, enabling more
natural communication. Actual voice, recorded by the humans, could generate an
elaborate speaking dictionary and hence, the natural voice could be heard with more ease.
Using machine learning further conversation related to the user’s requirement can be
modelled to personalize the utility of the glove. The project can further be expanded to a
proper sign language translator for the deaf and dumb.
10.Conclusion
In this paper, a technique of converting sign to text and voice is implemented with the
help of flex sensors, Arduino and a Bluetooth application. It compares all the existing
systems, their methodology and results. With this, a solution to the existing problem of
communication for the speechless is provided. The results show that the patient can
effectively communicate for their basic needs. However, a wider range of signs pose a
problem of frustrating the user’s memory and hence customization of the system is
required to manipulate the messages of the glove according to the user’s need.
References
[1] S. F. Ahmed, S. Muhammad, B. Ali, S. Saqib, M. Qureshi, "Electronic Speaking Glove for Speechless
Patients A Tongue to", pp. 56-60, 2010.
[2] A. Y. Satpute, A. D. Bhoi, T. Engineering, "ELECTRONIC SPEAKING SYSTEM FOR DUMB", vol.
6, no. 3, pp. 1132-1139, 2013.
[3] M. Wald, "Captioning for Deaf and Hard of Hearing People by Editing Automatic Speech Recognition
in Real Time", Proceedings of the 10th International Conference on Computers Helping People with
Special Needs ICCHP 2006 LNCS, vol. 4061, pp. 683-690.
[4] R. R. Itkarkar, A. V. Nandi, "Hand gestures to speech conversion using Matlab", 2013 Fourth
International Conference on Computing Communications and Networking Technologies (ICCCNT), pp.
1-4, 2013.
[5] Jingdong Zhao, Li Jiang, Shicai Shi, Hegao Cai, Hong Liu, G. Hirzinger, "A Five-fingered
Underactuated Prosthetic Hand System", Proceedings of the 2006 IEEE International Conference on
Mechatronics and Automation, pp. 1453-1458, June 2006.
[6] S. U. N. Praveen Kumar, S Havalagi, "THE AMAZING DIGITAL GLOVES THAT GIVE VOICE TO
THE VOICELESS", vol. 6, no. 1, pp. 471-480, 2013.
[7] Safayet Ahmed,Rafiqul Islam, Md.Saniat Rahman Zishan, Mohammed Rabiul Hasan,Md.Nahian Islam,
“ Electronic speaking system for speech impaired people: Speak up”
[8] N. P. Bhatti, A. Baqai, B. S. Chowdhry, M. A. Unar, "Electronic Hand Glove for Speech Impaired and
Paralyzed Patients" in EIR Magazine, Karachi, Pakistan, pp. 59-63, May 2009.
[9] Lee, Kyung-Sun, and Myung-Chul Jung. "Flexion and extension angles of resting fingers and wrist."
International Journal of Occupational Safety and Ergonomics 20.1 (2014): 91-101.
International Journal of Management, Technology And Engineering
Volume IX, Issue XI, NOVEMBER/2019
ISSN NO : 2249-7455
Page No: 67