Publications

Crowd monitoring in sports stadiums is important to enhance public safety and improve audience experience. Existing approaches mainly rely on manual observation, cameras, and microphones, which can be disruptive and often raise privacy issues. Recently, floor vibration sensing has emerged as a less disruptive and more non-intrusive method for crowd monitoring in sports stadiums. However, because vibration-based crowd monitoring is newly developed, open-source datasets are lacking, making it challenging to develop data-driven models. In this paper, we introduce Vibration Leverages Audio (ViLA), a vibration-based crowd monitoring method that reduces the reliance on labeled data by pre-training with unlabeled cross-modality data. Specifically, ViLA is first pre-trained on general-purpose audio data in an unsupervised manner, and then fine-tuned with a limited amount of labeled vibration data in sensing domains. Through this approach, ViLA learns general spectral pattern representations from audio, then adapts this knowledge to vibrations. By leveraging general-purpose audio datasets, ViLA reduces the reliance on large quantities of domain-specific vibration data. This is particularly important in sensing environments characterized by high data variance and limited sensing durations, such as sports games. Our real-world experiments demonstrate that pre-training the vibration model using publicly available audio data (YouTube clips) achieved up to a 5.8X error reduction compared to the model without audio pre-training.

Heart rate is one of the most critical metrics for human health. Most common methods for measuring human heart rate involve body contact, whether from wearable devices or manual measurement. However, such devices can cause discomfort to some patients. Past work for non-contact or remote heart rate measurement (e.g., camera or radio) is often limited by line-of-sight requirements that are not always possible in the real-world environment. This paper presents FloHR, an indirect heart rate monitoring system for human beings using heartbeat-induced floor vibrations. The key insight is that the human body generates a small wave of pressure and sound with each heartbeat. These are propagated as vibration through the structures the person is in contact with (e.g., a chair) and through the floor. FloHR then detects and interprets these small floor vibrations. We developed a highly sensitive vibration sensing system and heartbeat pattern modelling to identify these tiny vibrations among other body motions and ambient noise. We evaluated FloHR in a real home environment, demonstrating an average heart rate error similar to medical device standards on the floor near the subjects’ chair, and on the order of 10 beats per minute (bpm) on the floor 2 meters away from the subject.