The aim of this book is to document the state-of-the-art technology that has been developed to facilitate rehabilitation both for physical and for mental health. As the technology advances, we anticipate that technology will play an even greater role in rehabilitation. The scope of the book is limited to rehabilitation via non-intrusive physical activities. The topics on the use of exoskeleton and more intrusive methods for rehabilitation are intentionally excluded from the book, not because they are not important, but to make the book more coherent. Those topics deserve books on their own. The book includes three parts.

An inertial measurement unit (IMU) is an electronic device that consists of a set of inertial sensors. The purpose of an IMU is to measure inertial parameters of a moving object, such as the position (including altitude), orientation, velocity, rotation (i.e., angular speed), and sometimes the heading of the object. Inertia is a physical property of a mass to remain at its current state (such as moving along a straight line with a fixed velocity) as long as there is no external force acted upon it. An IMU is equipped with at least an accelerometer sensor and a gyroscope sensor. If the moving direction needs to be determined, one can use an IMU that is equipped with the magnetometer sensor in addition to the accelerometer and the gyroscope sensor. IMU was initially used almost entirely for navigation. That is why the device is also referred to as inertial navigation unit (INU). INU was first used in World War II as part of the inertial guidance system for rockets. It was later also used for spacecraft and aircraft. With the rapid technology development in microelectromechanical systems (MEMS), IMU has significantly shrunk its size and it has been embedded in many wearable devices (such as smartwatches and wristbands) and mobile devices (such as smartphones and tablets). That is why IMU is also referred to as a kind of wearable device.

In addition to IMU, human motion can also be measured by the force exerted on the entity that a person is in contact with, such as the floor when walking. Furthermore, while a person's joint moves, such as joint extension or flexion, the movement of the body segment around the joint could also be estimated by the force applied to the wearable materials if they are used. Similarly, the trunk posture can be estimated by the force applied to the sensors attached to the body segment.

When some force is applied to an object, the force can be measured through the pressure experienced by the object per unit of 2-dimensional area. Given a force F, and the area A that the force is in contact with an object, the pressure P can be defined as P = F/A. The smaller the area, the higher the pressure. The unit of pressure is Pascal, or Pa for short, which is defined as one Newton per square meter.

The fundamental idea of force sensing is to transform the pressure into some measurable quantity, such as the change in voltage (in piezoelectric pressure sensors), resistance (in resistive pressure sensors), capacitance (in capacitive pressure sensors), and wavelength (in optical pressure sensors).

If textile can be instrumented with stretchable thin electronics, human motion could be monitored much more conveniently and comfortably than using IMU devices by wearing smart clothes made from such textile. With the advancement of stretchable electronics [131], researchers are indeed making inroad in this type of textile, which is often referred to as electronic-textile, or e-textile for short [132].

The different types of e-textiles proposed so far all use some form of strain sensors to detect the strains caused by human motion. The predominant approach is to apply a small DC current and measure the resistance changes in the strain sensors attached to the clothing. Some work used AC current and measure the impedance changes in the strain sensor instead [133]. Due to the particular requirement of applying the strain sensors on clothing (or sometimes directly on the skin), the strain sensors must be thin (typically under a 1 mm) and highly stretchable. Hence, traditional strain sensors are not a good fit. For example, according to [131], the skin on the related body segments (feet, waist, knees) can stretch and contract by up to 55% while traditional strain sensors can only allow 5% stretch/contraction. While several different types of thin wearable strain sensors have been proposed, it appears that the conductive elastomer has been the dominating approach since the early days because they can be applied to the fabric either using a mask or attached to the fabric easily [134-141]. Newer proposed sensors relied on various materials, from commercial stretch sensors [142] and commercial sliver-coated yarn [143], to nanotube-based sensors [131], to copper wire stitched to the shirt [133].

That said, compared with IMU, e-textile is still in its infancy. Although various e-textile-like sensors have been proposed, they need to add electrodes and wires connected to a processing unit with a microprocessor, storage, and wireless transmitter. The processing unit and the associated wiring would make the actual e-textile far less-convenient from the dream of monitoring human motion by only wearing clothes made from e-textile alone. Furthermore, e-textile sensors do not come in the form of textile that can be used to make clothing. They need to be attached to the clothes or directly to the human skin at particular locations. Nevertheless, this is an exciting new research area for human motion tracking.

In this section, we summarize the different types of myography in terms of their technical maturity and their applications in rehabilitation in Table 4.4. Among the four types of myography, EMG is undoubtedly the most mature and has the widest application rehabilitation studies. FMG appears to offer a viable alternative to EMG with inexpensive commercial FSR sensors available. MMG is promising. However, it would require the researchers to develop the sensor from scratch, which could limit its adoption. The setup of OMG is extremely simple and cost is very low because it only requires a conventional webcam and custom-made markers. Unfortunately, the current setup has strong limitations, e.g., the subject's forearm has to be fixed to a flat surface, and the camera has to be placed fairly close to the marker areas. So far, OMG has only been used to study finger movements and hand postures. How to create an untethered setup for OMG remains to be seen.

While vision-based human motion tracking systems have long been used for research, such as Vicon motion tracking systems. These systems use multiple cameras to obtain synchronized video frames from different angles around the subject and develop a 3D model of the subject for motion tracking and analysis. The subject typically would have to wear multiple markers at key positions depending on the activity to be tracked. Such systems are very expensive and requires periodically maintenance. Hence, they are only used in research laboratories and clinical facilities for rehabilitation studies. The release of the Kinect sensor by Microsoft in late 2010 completely transformed the paradigm of vision-based motion sensing. Although the Kinect sensor was released as an add-on gadget for the Microsoft's Xbox game console, soon it was used for motion tracking in many non-gaming applications, particularly for healthcare. A search using "Kinect" as the keyword at Web of Service Core Collection returned over 8,500 papers as of September 2021. A search using "Kinect rehabilitation" as the keywords returned over 1,135 papers. This revealed that a considerable portion of the Kinect-based research is for related to rehabilitation, as shown in Figure 5.1.

Hand is perhaps the most complex and amazing segment in the human body. Hand dexterity is essential for activities of daily living, and it may be impaired by neurological disorders such as stroke [9], spinal cord injury [9], and Parkinson's disease [232], and it could also be impacted by rheumatoid arthritis [1]. To help rehabilitation and monitor the progress of hand rehabilitation, it is important to measure hand dexterity. Hand dexterity consists of several key components, as summarized in [233]: Control of force. This refers to the degree of control of each individual finger in applying force when doing tasks such as power grip, precision grip, and grasp-and-lift; Finger independence. This refers to the degree of control of individual fingers independently of each other; Synchronization of finger movements. This refers to the degree of control of individual fingers on temporal movements.

In this chapter, we present some basic background information on human motion basic parameters, and on common statistical methods used to evaluate the feasibility of a new sensor-based measurement instrument (such as an IMU or Kinect-based system) to measure human motion, which would offer much lower cost and provide much better accessibility to patients, and ultimately facilitate practicing rehabilitation exercises at home.

Rehabilitation exercises are complex human activities. While automating the measurement of basic parameters such as joint angle is relatively straightforward using various sensing modalities, it is a challenge to perform automated recognition and assessment at the entire exercise level. In this chapter, we review the studies that have used various machine-learning algorithms and models to perform various levels of recognition and assessment. Major steps in machine-learning-based activity recognition is shown in Figure 8.1. In the context of rehabilitation exercises, machine learning has been used in tasks of different levels of challenges as shown in Figure 8.2, from recognizing the exercise being performed (which could be useful to perform automated repetition count), to assessing the quality of the exercise being performed (correct/incorrect or multi-categorical such as excellent, very good), to performing automated scoring according to some well-established clinical assessment scales.

Unlike the learning-based approach, where one does not need to know the details of a rehabilitation exercise, the rule-based approach requires a detailed description of each exercise to be recognized and evaluated in terms of a set of rules. Exactly because of this, it is difficult for the learning-based approach to provide specific feedback to patients regarding exactly what was performed correctly and what requires improvements. The rule-based approach could provide much more specific feedback to the patients. Studies on rule-based activity recognition can be roughly divided into two categories: (1) ad hoc rules that are applicable to a particular rehabilitation exercise; and (2) general purpose rules that are designed to define a broader range of exercises or activities. Another perspective related to rehabilitation exercises is the tolerance on movements within which are considered correct.

For both types studies, such uncertainty can be captured explicitly by a threshold in the measurement as part of the rule, or by using the fuzzy interference using some membership definition. Fuzzy inference has also been used to detect activities.

Traditional prescriptive rehabilitation exercises would work best in the clinical setting under direct supervision of a clinician. However, often it is necessary for a patient to perform additional repetitions of the prescribed exercises at home. A typical practice is for the patient to be given a written description of the exercises and perhaps video demonstrations as well. It is not surprising that the rate of adherence to the prescribed amount of repetitions is low. The low adherence rate may have many reasons, but lack of motivation to follow through with the prescribed exercises is obvious. The lack of motivation is caused by several factors: Written instruction on how to perform a prescribed exercise is hard to follow. A video instruction would be better, but it still leaves a lot to be desired; The patient receives no feedback on the exercise performed and the patient would have to perform repetition count; Doing the prescribed exercise, which is intrinsically repetitive, could be boring without any entertaining elements; The patient will not be held accountable for not adhering to the prescribed exercises because there is no way for a clinician to know the truth.

According to the National Institute of Health (https://clinicalcenter.nih.gov/rmd/pt/index.html), the goal of physical therapy is to restore or maintain sensory and motor abilities for patients who have functional impairments. Patients with a wide-spectrum of diseases and conditions can be benefited by engaging in physical rehabilitation prescribed by a certified physical therapist. Before a physical therapy can develop a plan of care, a thorough examination and evaluation of the patient's function levels are usually performed [395], which include the assessment of musculoskeletal functions, neuromuscular functions, cardiovascular and pulmonary metrics, integumentary status, communication ability, affect, and language levels, and cognitive ability.

Physical rehabilitation is one of the specializations in the larger scope of rehabilitation, as shown in Figure 11.1. In this part of the book, we will be covering occupational rehabilitation, pulmonary rehabilitation, cognitive rehabilitation, speech and language rehabilitation, and mental health rehabilitation. Among them, occupational rehabilitation has the broadest scope and it overlaps with physical and pulmonary rehabilitation (such as for the ability of walk and engage in activities of daily living), speech and language rehabilitation (such as for the ability to work and interact with others socially), cognitive rehabilitation (such as for the ability to engage in complex instrumental activities of daily living), and mental health rehabilitation (such as the ability to engage in normal social interaction). Cognitive rehabilitation and mental health rehabilitation also have overlap.

Occupational therapy is a discipline that studies how a human being engages in various activities in life [451], and how to help people to perform in these activities [452]. Trained occupational therapists are regarded as experts in doing [451]. Compared with other fields in rehabilitation, the scope of occupational therapy is extremely broad because it spans not only physical activities but also cognitive, psychological, and social aspects of human activities. Also because of this, intervention methods in occupational therapy vary significantly for different patients.

The common understanding of occupation is related to one's employment or means of making a living. In the discipline of occupational therapy, the term occupation is broadly defined, encompassing all forms of activities of daily life [451], which include those for self-care, for leisure, and for productivity. More specifically, an occupation is understood as an abstract concept, such as teaching, engineering, and singing. A list of typical occupations are provided in the fourth edition of occupational therapy practice framework: domain and process [453], as shown in Figure 12.1.

Oral communication is the dominating means of social interaction between humans. Hence, speech impairments could significantly impact an individual's career, mental health, and quality of life. Speech rehabilitation is a field that aims to address various speech impairments. Traditionally, speech rehabilitation is conducted by a trained clinician. In the United States, such clinicians are referred to as speech-language pathologists. To reduce the financial cost and to increase the convenience for the patients, various technologies have been adopted in speech rehabilitation, particularly mobile apps, which could enable a patient to practice alone at home. The main form of technology-facilitated speech rehabilitation is computer-based speech therapy.

As shown in Figure 13.1, speech therapy typically adopts a variety of speech exercises to improve one or more specific target areas that could improve the patient's ability to properly speech as well as to understand the other person's speech, ranging from loudness, phonation, articulation, resonance, prosody, to oral muscles, social communication, and cognitive communication. There are a large number of disorders that could lead to speech impairments, including aphasia, dysarthria, apraxia of speech, dyslalia, hearing impairment, resonance disorders, fluency disorders, articulation disorders, expressive disorders, and cognitive-communication disorders. Due to the heterogeneity of the disorders, speech exercises could differ significantly.

Many respiratory and lung diseases are chronic and some of them are incurable. Respiratory diseases impact hundreds of millions of people worldwide and incur significant medical costs and loss of productivity. One of the most well-studied respiratory diseases is the chronic obstructive pulmonary disease (COPD), which is currently incurable and is projected to be the third most frequent cause of death worldwide [517].

Although the terms "cognition" and "cognitive" have been pervasively used, they are rarely defined clearly. In [589, p. XV], cognitive science is defined as "the study of mental representations and computations and of the physical systems that support those processes." Presumably, "those processes" refer to the processes of mental computations. The Wikipedia page used the dictionary definition for cognition. Nevertheless, it appears that cognition is an umbrella term referring to anything with mind and brain. Hence, the scope of cognition is very broad, and there are many theories under the scope. The Wikipedia defined the scope of cognition as "all aspects of intellectual functions and processes such as: perception, attention, thought, intelligence, the formation of knowledge, memory and working memory, judgment and evaluation, reasoning and computation, problem solving and decision making, comprehension and production of language" without citing any references (https://en.wikipedia.org/wiki/Cognition). The scope of cognition can be implied from the chapters included in [589], which have the following: coordinate transformations in the genesis of directed action, attention, categorization, reasoning, cognitive development, the brain basis of syntactic processes, cognitive neuroscience, and emotion. In [590], 20 conscious brain events and 20 unconscious brain events are provided. Some of the listed conscious brain events include immediate memory, declarative memory, episodic memory, automatic memory, intentional learning, attended information, effortful tasks, strategic control, explicit inferences. Cognitive rehabilitation often aimed to improve these functions in the patients.

Mental health is a serious issue globally. According to [686], direct spending on mental health illnesses exceeded $225 Billion US Dollars in the United States in 2019, which is about 5.5% of all health spending in the country. According to Diagnostic and statistical manual of mental disorders: DSM-5 [687], there are a long list of mental health disorders. The primary categories are shown in Figure 16.1, including neurodevelopmental disorders (autism spectrum disorder and attention-deficit/hyperactivity disorder belong to this category), trauma- and stressor-related disorders (post-traumatic stress disorder is a well-known example in this category), neurocognitive disorders (major and mild neurocognitive disorders due to Alzheimer's diseases are well-known examples), substance-related and addictive disorders, depressive disorders, anxiety disorders, schizophrenia spectrum and other psychotic disorders, bipolar and related disorders, obsessive-compulsive and related disorders, dissociative disorders, somatic symptom and related disorders, feeding and eating disorders, elimination disorders, sleep-wake disorders, sexual dysfunctions, gender dysphoria, disruptive, impulse-control, and conduct disorders, personality disorders, and paraphilic disorders.