Projects

There are many applications that would benefit from a platform that allows users and developers to select which sounds to keep and enhance and sounds/noises to remove and filter out. Such a platform is difficult to realize because of the large number of different sounds, models, and signal representations that developers may use in their applications. We introduce a novel acoustic selective filtering architecture that intelligently integrates the physics of sound waves with a wide range of data-driven models in an adaptive feedback architecture to filter and enhance sounds depending on the application. In creating this platform, we have demonstrated the benefits of acoustic filtering in a wide range of domains, ranging from audio privacy to mobile health.

Project Page: Link
Publications: [IPSN 2022], [IASA 2022], [IPSN 2021 (Poster)], [AIChallengeIoT 2020]

We have become engrossed in our personal smartphones and other devices in today's society, even during instances that place our lives at risk. We design a low-power wearable headset mounted with an array of MEMS microphones to detect and localize oncoming cars to be worn by pedestrians engaged in music or talking on the phone while on the streets. The headset extracts and transmits audio features to the smartphone, which detects and localizes oncoming cars using novel machine learning and signal processing techniques. Our wearable platform provides pedestrians with a second set of ears to detect life-threatening situations in urban environments when their own ears are preoccupied.

Project Page: Link
Publications: [IoT Journal 2019], [IoTDI 2018], [IoTDI 2018 (Demo)], [VNC 2018], [SenSys 2016 (Demo)]
News and Media: New York Post, IEEE Spectrum, Fast Company, Mashable, Gizmodo, The Telegraph, Engineering.com, India Times, IEEE Signal Processing Magazine

Construction worker safety is one of the most challenging scenarios in urban safety because construction sounds are often orders of magnitude louder than vehicles. Building on our work on pedestrian safety, we developed and integrated a novel adaptive noise filtering architecture into a construction helmet wearable that filters out construction sounds, improving vehicle detection and localization. Our architecture leverages data-driven models of common construction tool sounds in conjunction with spatial audio filtering to robustly remove construction noises, while remaining light-weight.

Project Page: Link
Publications: [IPSN 2021]

Homes are becoming more intelligent due to the growth of smart sensors and devices found in typical homes. However, most of these sensors and devices function independently from one another, limiting the amount of utility and services a truly “smart” home can provide. We introduce two key ideas towards more intelligent homes. First, we explore the usage of mobile drones in the home environment. Second, we propose a system that seamlessly connects to the home environment and automatically discovers and jointly utilizes smart sensors and actuators around the home to provide services that are otherwise not possible.

Publications: [SenSys 2021 (Demo)], [Demo]

Calibrating and setting up camera networks for indoor tracking and a wide range of other tasks often requires professionals to calibrate and adapt each camera to every unique building floor map. We created an easily-deployed and privacy-preserving camera network system for indoor occupant tracking that self-localizes and calibrates each camera within the network without requiring users to provide any real-world references. Our system accomplishes this by leveraging the movements of people using the space in daily life to self-localize cameras, before generating the floor map of the covered space to begin tracking occupants throughout the environment.

Publications: [DCOSS 2022]

With the rise of population and emergence of smart buildings and smart cities, conserving energy is of prime importance. While people are careful about their usage at home, energy wastage in public buildings/offices goes unaccounted for as users do not pay for it. We develop a system that can generate real-time energy footprint for all users and provide related data analytics to help conserve their energy and even implement advanced functionalities such as prediction and modeling.

Project Page: Link
Publications: [TOSN 2018], [BuildSys 2017]

Recent research efforts have made significant progress in reducing commercial building energy consumption through a variety of methods, including optimizing building heating, ventilation, and air conditioning (HVAC), lighting, and personal electric devices. However, these works focus on reducing energy consuming resources while treating occupants as immovable objects separate from the building energy optimization problem. In this work, we create a recommender system for reducing energy consumption in commercial buildings with human-in-the-loop, by recommending users energy-saving actions (move to room A, change setpoint temperature, etc.). We formulate the building energy optimization problem as a Markov Decision Process, show how deep reinforcement learning can be used to learn energy saving recommendations, and effectively engage occupants in energy-saving actions.

Project Page: Link
Publications: [IoTJ 2020], [UMAP 2018]

With the recent societal impact of COVID-19, companies and government agencies alike have turned to thermal camera based skin temperature sensing technology to help screen for fever. However, the cost and deployment restrictions limit the wide use of these thermal sensing technologies. Existing works require an employee to manually take temperature measurements or require the user to stand facing a thermal scanner. In this work, we present a low-cost system based on a RGB-thermal camera for continuous fever screening of multiple people. This system detects and tracks heads in the RGB and thermal domains and constructs thermal heat map models for each tracked person, as they move about daily life. Combining these thermal heat map models of the head with physical models of heat dissipation, we robustly detect and measure temperature without requiring users to stop or manually take a measurement.

Project Page: Link
Publications: [IPSN 2022]

Mental status examination (MSE) is an important tool used by mental health providers for assessing mental health. Currently, MSEs are conducted by licensed professionals, which is a barrier for patients in low-income and remote areas. We create an AI-based Personal Online Mental Status Examination, the first interactive mental status examination platform, where users can self-administer MSEs at home through a web browser, using only a camera and microphone. Our system uses multimodal image, speech and natural language processing algorithms to detect signs of abnormalities in mental functioning and recommend them for further examination by a mental health specialist.

Publications: [Pervasive Computing 2022]

Knowing the amount of perspiration we produce throughout the day is important for maintaining health. Existing mobile solutions for measuring perspiration often require you place a wearable device directly in contact with your skin, which may produce discomfort and can only measure the sweat at a specific location. We create a novel wearable sensor system that measures an individual’s sweat level, consisting of multiple braided cotton-covered conductive threads, that is robust to sensor distortion or the motion of the person. Because our system is made from threads, it can be easily embedded into clothing, removing the need for users to attach an additional wearable device.

Publications: [Sensors 2018], [CPD 2018],

Sensing a person's inner emotions is challenging, often requiring users to wear bulky ECG caps in laboratory settings. Outside of laboratory settings, there exist no portable and robust solutions for monitoring emotions. We create a a low-cost, long-lasting, wireless, glasses-based platform for continuous monitoring of user's facial expressions and emotions in daily life. Our system leverages a wide range of non-contact sensors (e.g., proximity sensor and thermal camera), which reduces discomfort and allows us to sense emotions even as users move about their daily lives.

Publications: [IoTDI 2020], [IPSN 2020 (Demo)], [Demo]

Leg shaking is something all of us can relate to; when you or someone near you sits down and unconciously begins to shake his/her leg, the generated noise and vibrations causes a disturbance. Intuition says that this problem can be resolved using the smartphone's accelerometer combined with simple processing techniques, but show that if the smartphone is not directly located on the person's shaking leg, the accelerometer can measure patterns that look like other common movements. To solve this challenge, we develop a novel PCA-based filtering algorithm that retains only the fundamental frequencies of leg shaking and incorporate this method into a smartphone-based system that leverages user feedback to improve detection performance over time by personalizing our novel filtering method to the phone owner's unique leg shaking characteristics.

Publications: [CHASE/Smart Health 2018], [HASCA 2017]

Ensuring patients take their medication correctly and on time is very challenging, as healthcare professionals often need to rely on patients to hold themselves accountable. Existing smart pillbox systems have been successful in notifying, tracking and reporting medication adherence, but do not provide sufficient security and privacy for the patients’ medication adherence records. We create an intelligent privacy preserving pillbox that by ensuring that medication adherence records can only be decrypted by the doctor, authorized by a face-to-face key exchange process through out-of-band communication channels. Our system gives patients fine-grained control over how their information is transmitted and shared, which has been a major privacy concern for many patients. Additionally, we develop a novel light-based sensor for detecting when a patient has taken his/her pill, which has improved performance over existing modalities (e.g., weight sensing).

Publications: [CHASE/Smart Health 2018]