Artificial Intelligence in Health

DISCLAIMER: The technologies discussed in this article should not be considered a replacement for a physician-made diagnosis.

Smart phones are a central part of our daily lives with applications, or apps, to keep us entertained, to be productive and even to improve our health. In terms of health, we are just beginning to scratch the surface of what might be possible. We can track our steps, encourage healthy habits, and research conditions, but we are a long way from pulling out a medical tricorder (the handheld device from Star Trek used to help diagnose diseases and collect bodily information about a patient) and instantly diagnosing health problems. Advances in technology are bringing us ever closer, however.

Suppose your next phone call to a friend could alert you to a concussion, identify you as being high risk for coronary heart disease, reveal Parkinson’s disease, or recognize early signs of Alzheimer’s disease. What if the photos you upload to social media can also be used to spot genetic conditions?

Through improvements in artificial intelligence (AI) and technology, computers are able to process large quantities of data -- analyzing, sorting and matching patterns to do just that. By comparing audio and/or visual input with known biomarkers, machines are able to provide physicians with additional data to help in diagnosis.

Biomarkers, or biological markers, are objective indications which help determine the medical state of a patient. Unlike medical symptoms perceived by the patient, biomarkers are observed from outside the patient and are used as indicators of normal biological processes.

For example, take a situation where someone tells you he hates you. Sounds bad, right? But when you factor in a good-natured tone, a smile on his face, and a longstanding friendship, maybe he is simply congratulating you on winning a new car and not expressing actual hate. Beyond the statement itself, we compare the audio and visual cues against past experiences to determine the intent of the message.

Advances in AI are allowing medical professionals to identify objective, quantifiable biomarkers with speed and accuracy to aid in the diagnosis of a patient. As AI and processing power have improved, machines have become more adept at detecting subtle indications that something is wrong and matching that to a set biomarker. Many of these tests deal with audio and visual cues that physicians are either unable to pick up on or are unable to do so quickly.

Audio Biomarkers

Audio or vocal biomarkers are medical signs detected in your voice. By mapping these voice characteristics to specific medical conditions, physicians gain additional insight into his or her patient’s health. Along these lines, researchers are beginning to use speech patterns to help predict whether patients have or are likely to develop certain conditions.

Audio wave.

The United States Army Medical Material Agency (USAMMA), in conjunction with the Massachusetts Institute of Technology (MIT), is working on a device that uses audio biomarkers to help detect concussions, or other mild traumatic brain injuries (TBI). The device will be programmed to look for elongated syllables and vowel sounds, difficulty in pronouncing phrases that require complex facial muscles, general voice quality, and other audio biomarkers that show a strong correlation with concussion. Although the Army intends to use this device in training or on the battlefield, the ultimate goal of a real-time screening app that can be used anywhere is not all that far-fetched.

In December 2016, the Mayo Clinic released a study showing how voice analysis can be used to help detect coronary artery disease (CAD). The study identified 13 voice characteristics that may be used by physicians to determine the probability of CAD for patients with chest pains prior to a coronary angiograph. In no way does this analysis replace the physician, but it does provide him or her with another tool to help make better decisions and provide better patient care.

View Beyond Verbal and Mayo Clinic CAD Poster

Winterlight Labs is developing a tool (currently in beta testing) to help with the early detection of Alzheimer’s disease, the sixth leading cause of death in America. By analyzing speech patterns outside of the range of human hearing, Winterlight Labs hopes to diagnose the disease in its early stages when the few known interventions are still available. In addition to being much more sensitive, this tool has an advantage in that it can be taken multiple times a week where traditional tests are limited to every 6 months. Frequent testing not only allows for better tracking over time but also helps to eliminate inaccuracy due to taking the test on a good day or bad day.

Parkinson’s disease, a progressive disorder of the nervous system that affects movement. Tremors are the most easily recognized of the early signs of Parkinson’s but other symptoms include slowed movement, rigid muscles, loss of automatic movements and speech changes. Parkinson’s may cause you to speak softly, quickly, slur or hesitate before talking.  No specific tests exist to diagnose Parkinson’s and physicians (often neurologists) diagnose Parkinson’s based on medical history, a review of signs and symptoms, and by eliminating other possibilities. This can be time consuming and expensive to diagnose.

Using AI to analyze voice samples could save both time and expense if researchers are able to develop reliable biomarkers. The Parkinson’s Voice Initiative has been established to search for these early biomarkers and in turn develop high speed, voice analysis, tests (less than 30 seconds) that can be taken over the phone or used in screening programs. In early testing, researchers were able to achieve as high as 98.6% detection accuracy under lab conditions (without line static and background noise).

Visual Biomarkers

Improvements in technology have also expanded new possibilities for using visual biomarkers to aid in diagnosis and prevention of genetic disorders. Research shows that many genetic conditions have a unique set of facial features, a tell-tale “face,” that can aid in diagnosis. Artificial intelligence can be used to quickly analyze facial features and recognize patterns to diagnose conditions, the same way facial recognition identifies people.

Visual Biomarkers

Last October, FDNA Inc. announced the launch of a set of apps, called Face2Gene, to aid in the identification and evaluation of rare genetic disorders. The Face2Gene apps take a patient photo, extracts data points from the photo and compares them with a database of similar data points associated with real-world patients that have been diagnosed with these rare genetic disorders.

The RightEye GeoPref Autism Test™ helps doctors identify early stages of autism spectrum disorder (ASD, a group of developmental disorders with a wide range of symptoms and levels of disability) in children 12 to 40 months old by using eye-tracking technology. The test uses infrared sensors to test the child’s eye movement as he or she watches split-screen video (one side with people and faces and the other with moving geometric shapes). At this age, children with healthy brains focus more on faces than shapes. The amount of time they spend looking at each screen can help predict where the child might fall on spectrum. Early tests have shown an 86% success rate in correctly predicting ASD.

Additional Applications

These are just a few examples of technologies that might be possible in the future. As research and technology advances, it would not be out of the question to include a quick voice screening as part of the procedure when visiting the doctor (many already check blood pressure and temperature). Any red flags would be brought to the doctor’s attention and included in the patient’s medical records.

Schools routinely screen children with hearing and vision tests and it would be easy to record a voice screening at the same time, assuming that a standardized voice sample could be used to screen for multiple conditions. Parents could decide if they want their children screened or not.

We already have medical kiosks in places like drugstores that check blood pressure, heart rate, and vision. Adding recorders for voice screening, a camera for facial recognition tests, or an infrared sensor to track eye movements could have a tremendous impact on early detection.

Smartphone Applications

In fact, why wait for a doctor visit if an app on your smartphone (that can be turned on or off or be given permission) can analyze your voice as you make everyday phone calls and alert you to any significant findings?

Facebook, a popular social media platform with more than 1.65 billion users, already uses facial recognition software to identify people in photos by comparing facial features from new photos with those in its database that have previously been identified (or tagged). Could it just as easily keep a database of visual biomarkers that are unique to certain genetic conditions? Sure it could.

In addition, thanks to its enormous database of tagged images and the associated data found in user profiles and posts, Facebook would present a tremendous research opportunity. By correlating additional data points derived from posts about diet and exercise, location of residence and recent trips,  family history (pulled from listed family member profiles) or any number of other data points, researchers may be able to learn more about causes, symptoms, and treatments.

Although it is not likely to happen on a social media site like Facebook, a group called Beyond Verbal, through the Beyond mHealth Research Platform, is working to collaborate with researchers, hospitals, universities, and businesses to collect voice samples and discover new biomarkers. Beyond Verbal already has well over 2.5 million voice samples in more than 40 languages and is continuing to grow. With virtual assistants like Siri and Alexa, smart cars, and smart phones, the opportunities are endless.


As AI improves in both speed and accuracy, healthcare professionals will see greater benefit from the use of audio and visual biomarkers in diagnosing patients. New devices and apps are just around the corner for both patients and physicians that will go a long way toward helping improve health, possibly even moving from diagnosis to prevention.

Naomi Lachance. NPR. Facebook's Facial Recognition Software Is Different From The FBI's. Here's Why. Published 5/18/2016. Accessed 1/17/2017.

Ellen Crown. U.S. Army Medical Materiel Agency Public Affairs. Army Partners with MIT Lincoln Lab on Voice Analysis Program to Detect Brain Injury. Published 6/6/2016. Accessed 1/10/2017.

The Medical Futurist. Vocal Biomarkers: New Opportunities in Prevention. Accessed 1/10/2017.

Megan Molteni. Wired. Thanks to AI, Computers Can Now See Your Health. Published 1/9/2017. Accessed 1/10/2017.

Karen Pierce, Steven Marinero, Roxana Hazin, Benjamin McKenna, Cynthia Carter Barnes, and Ajith Malige. Society of Biological Psychiatry. Eye Tracking Reveals Abnormal Visual Preference for Geometric Images as an Early Biomarker of an Autism Spectrum Disorder Subtype Associated with Increased Symptom Severity. Published 2015. Accessed 1/10/2017.

Kyle Strimbu and Jorge A. Travel, MD. What are Biomarkers? Published 2010. Accessed 1/10/2017.

Heather Mack. Mobi Health News. Mayo Clinic study shows voice-analyzing app may be useful in heart disease diagnosis. Published 11/14/2016. Accessed 1/10/2017.

Wikipedia. Tricorder. Accessed 1/17/2017.

Mayo Clinic. Parkinson’s Disease. Accessed 1/18/2017.

Parkinson’s Voice Initiative. Accessed 1/18/2017.

Face2Gene. Accessed 1/18/2017.

Leave a Reply