This is an excerpt from a story delivered exclusively to Business Insider Intelligence Digital Health Briefing subscribers. To receive the full story plus other insights each morning, click here.
AI has made a splash in the healthcare industry, with a flurry of hospitals rushing to deploy AI technologies.
But the tech’s also littered with potential pitfalls, the consequences of which could have adverse effects on all players in the healthcare system, per research cited in The New York Times.
Here’s what it means: AI is susceptible to manipulation and may generate inaccurate diagnoses.
- Slight tweaks can dupe AI systems into seeing something that isn’t there. Researchers revealed that a quick alteration of a few pixels in an image of a benign skin lesion led to a misdiagnosis that the lesion was malignant, for example. The AI system also incorrectly diagnosed when the scan was rotated.
- And AI produces different diagnoses in response to synonymous symptom descriptions. For example, the tech spat out different diagnoses when doctors defined a symptom as “back pain” versus “lumbago” — despite the terms’ unvarying meanings.
The bigger picture: AI’s shortcomings could lead to shady behavior and costly medical errors.
- Bad actors could game AI for financial gain. For example, hospitals could make alterations to patient scans in order to generate diagnoses that drum up higher payouts from payers. Or doctors could tweak their language to produce an intended diagnosis, whether or not it's accurate.
- Unintended misdiagnoses generated by AI could lead to costly medical errors. Medical errors cost the US health system about $20 billion dollars annually and account for 100,000 to 200,000 patient deaths each year. Errors brought about by ill-trained AI tools could add to these losses, with hospitals bearing the brunt of the consequences.
AI's use in healthcare should be regulated to mitigate manipulation. While the US Food and Drug Administration (FDA) announced last year that the agency would be developing “a regulatory framework to promote innovation” of AI in the health space, regulations and standards around AI in healthcare are still ambiguous, Forbes notes.
A formal regulatory process and monitoring framework around AI could curb misuse, negative outcomes, and bad press that could otherwise delegitimize AI and undermine the tech's impact on healthcare.
More to Learn
In Digital Health Startups to Watch, Business Insider Intelligence looks at the top digital health companies disrupting US healthcare in four key areas: artificial intelligence (AI), digital therapeutics, health insurance, and genomics. Startups in this report were selected based on the funding they've received over the past year, notable investors, the products they offer, and leadership in their functional area.
In full, the report:
- Details the areas of the US health industry that show the greatest potential for disruption.
- Forecasts the industry adoption of bleeding edge technology and how it will transform how healthcare organizations operate.
- Unveils the top five startups in AI, digital therapeutics, health insurance, and genomics, and how they're positioned to solve big issues that key players in healthcare face.
- Explores what's next for the leading startups, providing a glimpse into the future of the healthcare space and demonstrating how we'll get there.