UW scientists turn Amazon’s Alexa into heart monitoring device using sound waves

Researchers at the University of Washington have figured out a way to use machine-learning algorithms to turn smart speakers into sensitive medical devices that can detect irregular heartbeats.

The scientists use smart speakers like Amazon Echo or Google Home to send out an inaudible sound that bounces off a person’s chest and returns to the device, reshaped in a way that reveals the heartbeat. An uneven cardiac rhythm can be associated with ailments including strokes or sleep apnea.

The researchers employed a machine-learning algorithm to tease out the heartbeats from other sounds and signals such as breathing, which is easier to detect because it involves a much larger motion. The algorithm was also needed to zero in on erratic heart rhythms — which from a health perspective are generally more important to identify than a steady “lub-dub.”

“If you have a regular pattern, it is easy to find,” said Dr. Arun Sridhar, assistant professor of cardiology at the UW School of Medicine. “If it’s all over the place, it’s challenging.”

Here’s how it works:

• The smart speaker sends out a continuous inaudible signal that strikes a person who is sitting 1-2 feet away from the device.
• The sonar signal bounces back, reflecting the small movements of the chest that come from the motions of a beating heart.
• That signal is captured by the smart speaker’s multiple microphones. (The microphones are what allow Alexa or Google Assistant to tune into a specific voice that’s giving directions, separating it from background chatter).
• A “beam-forming algorithm” written by the researchers allows the speaker to use input from the microphones to isolate the heartbeat signal from other noise.

The non-invasive technology could be used in home or clinical settings to check on patients remotely, to diagnose illness or to monitor someone as a person sleeps. But there are limitations. The monitoring worked less well with overweight people and in participants wearing multiple layers of clothing.

Privacy advocates often raise concerns about the ability of smart speakers to listen in on people and collect personal information. This new skill does not use frequencies that have audible data, helping preserve privacy. It also works in a limited range, only monitoring someone within a couple of feet of the device.

The scientists on Tuesday published a proof-of-concept paper on the technology in the journal Communications Biology. They tested the approach on 26 healthy people and 24 cardiac patients who were hospitalized with conditions such as atrial fibrillation and heart failure. The researchers compared the results collected by the smart speakers with those generated by a conventional heartbeat monitor and found there was little difference.

Sridhar was co-senior author on the research along with Shyam Gollakota, a UW associate professor in the Paul G. Allen School of Computer Science and Engineering. Other authors are Anran Wang, a doctoral student in the Allen School, and Dr. Dan Nguyen, a clinical instructor at the UW School of Medicine.

“If you have a device like this, you can monitor a patient on an extended basis and define patterns that are individualized for the patient. For example, we can figure out when arrhythmias are happening for each specific patient and then develop corresponding care plans that are tailored for when the patients actually need them,” Sridhar said. “This is the future of cardiology. And the beauty of using these kinds of devices is that they are already in people’s homes.”

Gollakota is also the co-founder of Sound Life Sciences, a Seattle-based startup that spun out of the UW. The company is commercializing smart speaker-based health tools that can monitor motion and vital signs like breathing and heart rate for remote monitoring and telemedicine. The 2-year-old company holds the patent for this new heart-rhythm technology as well.

Wavely Diagnostics, another startup co-founded by Gollakota, is building a commercial app that uses software running on smartphones to detect earaches.

The original article can be found at: GeekWire