Kumar, Verma, Bhattacharya, and Lahiri: Audio-Visual Stimulation in Conjunction with Functional Electrical Stimulation to Address Upper Limb and Lower Limb Movement Disorder

Audio-Visual Stimulation in Conjunction with Functional Electrical Stimulation to Address Upper Limb and Lower Limb Movement Disorder


Neurological disorders often manifest themselves in the form of movement deficit on the part of the patient. Conventional rehabilitation often used to address these deficits, though powerful are often monotonous in nature. Adequate audio-visual stimulation can prove to be motivational. In the research presented here we indicate the applicability of audio-visual stimulation to rehabilitation exercises to address at least some of the movement deficits for upper and lower limbs. Added to the audio-visual stimulation, we also use Functional Electrical Stimulation (FES). In our presented research we also show the applicability of FES in conjunction with audio-visual stimulation delivered through VR-based platform for grasping skills of patients with movement disorder.

Every individual uses his/her upper and lower limbs to perform activities of daily living. However, if an individual suffers from limb movement disorders as a result of disability, then that individual becomes dependent on caregivers. Also, consequently, his participation in community life gets adversely affected. Disability is often a consequence of neurological disorders, e.g., stroke. In fact, stroke is the second leading cause of death and the third leading cause of disability-adjusted life-years (DALYs) worldwide.1 In the past two decades, the number of stroke cases and the overall global burden of stroke have been increasing.2 Rehabilitation offers stroke patients an avenue for practicing skills that can lead to enhancement of functional ability and subsequent realization of greater participation in community life. Therefore, many researchers are working on developing efficient skill training platforms for post-stoke rehabilitation. Following stroke, patients often need to re-learn how to perform motor activities. Learning requires practice, and feedback is important for practice to be effective.3 During rehabilitation, delivery of extrinsic audio-visual feedback either in the form of knowledge of results or knowledge of progress of performance can motivate the participant for continuing the repeated rehabilitation therapy. In conventional rehabilitation practices, the therapists prescribe the dosage of therapy required based on the capabilities of the patients in an individualized manner. Additionally, since the repetitive exercises often turn out to be monotonous to the patients during the rehabilitation the therapists often deliver encouraging feedback in an audible manner so as to motivate the patients to do the tasks. However, in developing countries like India, restricted access to specialized health clinics and expert clinicians often causes huge hindrance to availing specialized post-stroke care. Thus, researchers are exploring different technology-assisted techniques that can provide individualized exercise platforms and are also motivating for use by the patients through feedback delivered in the form imagery and audio stimulus. Out of the available technology- assisted mechanisms we chose Virtual Reality (VR). The VR platform allows the designer to create synthetic environments with precise control over a large number of task parameters that influence one’s behavior in an individualized manner. In fact, VR provides an option for effortlessly manipulating the number, speed or order of stimulus presentation while maintaining an objective means of data collection on relevant target responses.4 The inherent flexibility of VR-based system allows incremental variations in task difficulty thereby scaffolding the development in the participant’s skill in a precise, objective and quantitative manner. Additionally, VR-based platform is capable of providing audio-visual feedback in an individualized manner which often serves to be motivational. In our research, we have applied VR-based platforms for rehabilitation of upper limb and lower limb disorders. Also the flexibility of VR to interface it with different hardware peripherals has enabled us to augment the VR-based rehabilitation platform with Wii Balance Boards (BB), Functional Electrical Stimulation (FES), etc. The rest of the paper is organized as follows: In Material and Methods we present our system design and methodology. Results, discussion and future work follow.

Materials and Methods

VR-based upper limb rehabilitation platform integrated with FES

Among the different types of forearm motor activities required in simple day-to-day tasks, two primary types of movements required are closing and opening of hand for grasping and releasing objects of different sizes respectively. Research shows that about 80% of stroke survivors lack in movement control and coordination skill that are vital requirements specifying the grasping abilities of one’s upper limb.5 Here we used the concept of classifying the movement of the affected hand while using the physiological data (surface electromyogram (sEMG)) from the healthy hand as template, to develop an individualized rehabilitative platform for stroke patients. We also deliveredrehabilitation by making our system capable of providing automated FES based on the participant’s performance in an exercise task. based tasks that required the participant to completely open and/or completely close his affected hand and maintain his hand in that configuration for 30 seconds. If the participant was able to keep his hand open (for hand extension tasks) or closed (for hand flexion tasks) for the entire 30 seconds, the task was successfully completed. We designed such VR-based tasks which would give the participants a feeling of navigating through some interconnected events from their daily life. For example, the tasks were opening a door to enter a room with dim lighting, then closing the door, followed by lighting bulbs in the room, then moving through the room while switching off the lights at the back, to move to a corridor leading to a backyard garden having birds and grow plants by watering the garden. The complete game had six tasks (T1 - T6) with different environments in the VR world. After completion of each task, our system delivered audio-visual feedback to the participant regarding his performance in terms of motivational messages (e.g. 'Good Job!') and performance score points (0-100). The VR platform was also interfaced with a transcutaneous electrical stimulator. In case a participant’s performance score during T1 (hand ‘open’ configuration) and/or T2 (hand ‘close’ configuration) was less than 90, our system was designed to give an electrical stimulation to the participant’s affected hand preceding each of the subsequent tasks with similar hand configuration (T3 and T5 for hand ‘open’; and T4 and T6 for hand ‘closed’). We have used Microstim Genius, a portable muscle stimulator from Technomed Electronics comprising of metal electrodes to give surged faradic pulse stimulation for 20 seconds with an intermediate 5 seconds of rest to the motor points controlling the Flexor and Extensor group of muscles, responsible respectively for hand flexion and extension, of the affected hand of the participants.

VR-based lower limb rehabilitation platform

One of the most common medical complications after a stroke are falls with a reported incidence of up to 73% in the first year post-stroke.6 Fig 2(a) shows the experimental setup for lower limb rehabilitation exercise platform. The participants were asked to stand on the BBkept in front of the Task Computer (PC). The BB was connected to task computer and measured the participant's center of pressure (CoP) data while he performed the tasks shown on the computer monitor. Fig. 2(b) shows an example of the GUI for VR-based tasks. The task required the participant to shift weight in the instructed direction (namely North, East, South and West), while moving a virtual object in the VR environment to a predefined position and then hold the object in that position for one second. One’s performance in a task was evaluated for each direction and subsequently the cumulative total score was computed for that task.


We designed proof-of-concept usability studies using VR-based platform.

A. Implication of audio-visual stimulation integrated with FES on grasping performance

As a proof-of-concept for upper limb rehabilitation platform, we conducted a preliminary usability study with one exposure to two stroke-surviving (SS) and six age-matched able bodied (AB) participants . Fig. 3 shows an improvement in the average performance score points obtained by SS participants for both the hand flexion and extension tasks upon application of electrical stimulation while doing the tasks. On the other hand, the AB participants were able to achieve 100 points in nearly all the tasks, as expected, even without stimulation.

B. Implication of audio-visual stimulation on balance performance

Here we present our findings on a preliminary usability study where we used audio-visual stimulation for patients with balance disorders. As mentioned in our prior work on lower limb rehabilitation, designed as a proof-of-concept platform, we conducted a 3-4 day usability study on 4 hemiparetic volunteers (P1-P4), who were attending a rehabilitation program at a local civil hospital.7 Each participant was exposed to our system for 3-4 sessions lasting over 3-4 days. Fig 4 shows the task progression of each participant on first day (DayF) and last day (DayL). On each day, all the participants started with a VR-based task of difficulty level 1 (DL1), and then progressed through the VR-based tasks of different difficulty levels increasing from DL1 to DL3, based on his task performance. Although all the participants were able to reach tasks belonging to DL3 on both DayF and DayL of training, yet for most of them, there was a visible distinction in task progression pattern between the DayF and DayL. Specifically, for participants P1 and P3, the task progression had lesser switching of tasks belonging to different difficulty levels on the DayL compared to the DayF. With regard to participant P4, the pattern of task progression on the DayL was almost similar to that on the DayF with minor improvement in terms of lesser switching of tasks belonging to different difficulty levels. For participant P2, the task progression pattern on the DayL was similar to that on the DayF in terms of switching between tasks belonging to different difficulty levels. However, both P2 and P4 were able to interact with more number of VR-based tasks of DL3 on their DayL compared to the DayF.


The systems presented here are built as a proof-of-concept application and not as a full-fledged stroke-rehabilitation platform. Initial results indicate improvement of hand flexion and extension skills of stroke-surviving individuals after the application of FES added to audio-visual stimulation. Also, the results indicate the potential of audio-visual stimulation to bring in improvements in performance in lower limb balance related tasks. While demonstrating proof-of-concept of the technology and trends of “improved” hand-movement skills and lower limb balance skills with the help of a VR-based task providing audio-visual and electrical stimulation, questions about the practicality, efficacy, and ultimate benefit of the use of these technological tools for demonstrating clinically significant improvements in terms of stroke rehabilitation remain, which will be addressed by empirical investigation in the future. The ultimate aim is to potentially reduce the burden on the healthcare system of our country and give a new hope to individuals with movement disorders.


The authors would like to thank DST, India and INRIA, France for funding this research and also to the participants for making this study possible. We gratefully acknowledge the support of Spine Institute, Civil Medical Hospital, Ahmedabad, Gujarat, India.



R Lozano, M Naghavi, K Foreman. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study. The Lancet 2010;380:2095–2128


VL Feigin, MH Forouzanfar, R Krishnamurthi. Global and regional burden of stroke during 1990–2010: findings from the global burden of disease study 2010. The Lancet 2014;383(9913):245-54. Review. Erratum in: Lancet. 2014;383(9913):218.


R Stanton, L Ada, CM Dean, E. Preston Biofeedback improves activities of the lower limb after stroke: a systematic review. J Physiother 2011;57(3):145-55. doi: 10.1016/S1836-9553(11)70035-2. Review.


TM Schultheis, AA Rizzo. The application of virtual reality technology in rehabilitation. Rehabilitation Psychology 2001;46:296-311.


AL Delden, CL Peper, J Harlaar. Comparing unilateral and bilateral upper limb training: the ULTRA-stroke program design. BMC Neurology 6;9:57. doi: 10.1186/1471-2377-9-57.


GS Verheyden, V Weerdesteyn, RM Pickering. Interventions for preventing falls in people after stroke. Cochrane Database Syst Rev 31;5:CD008728. doi: 10.1002/14651858. CD008728.pub2.


S Verma, A Kumawat, D Kumar, U. Lahiri Virtual reality based balance training platform for stroke patients: A usability study. 2nd IEEE International Conference on Human Computer Interactions 2016.

Fig 1:

Left panel, Screenshot of one GUI for data collection from healthy hand;Right panels, Screenshots of the six VR- based tasks.

Fig 2.

Left panel, Experimental setup; Right panel, GUIs of VR-based tasks.

Fig 3.

Group average ofperformance score points of all the participants.

Fig 4.

Comparative analysis of task progression pattern of P1-P4 between first day and last day of interaction.

Abstract views:


Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Copyright (c) 2016 Deepesh Kumar, Sunny Verma, Sutapa Bhattacharya, Uttama Lahiri

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
© PAGEPress 2008-2018     -     PAGEPress is a registered trademark property of PAGEPress srl, Italy.     -     VAT: IT02125780185