Remote Gait Analysis with Multimodal Test-time Adaptation on Edge Devices
Professor: Tingting Chen
Description: Gait analysis plays a critical role in understanding human motor control, rehabilitation and early detection of some neurological disorders including Parkinson’s disease and stroke. This project aims to design an adaptable gait analysis deep learning framework based on video data, and implement it on an edge device for feasibility testing. The design goal is to and to provide a safety rail, i.e., fall back or request recalibration when detecting significant domain shifts before adapting. Cal-Bridge scholars will be engaged in the following activities: using public datasets VIDIMU and accelerometry-walk-climb-drive from PhysioNet to 1) conduct experiments to verify domain shifts and the impacts on deep learning structures (such as CNN, ResNet, LSTM) for gait-based tasks such as activity recognition and disease prediction; 2) build continual test-time adaption solutions for gait analysis, for example, by exploring body-structure informed strategies to encourage smooth latent trajectories across adjacent windows, reflecting continuity of motor states; 3) implement and evaluate their framework accuracy and efficiency on the NVIDIA Jetson Orin Nano.
Preferred Qualifications:
