The amplified risks of AI medical advice for older adults

The unsettling part is not that an AI chatbot can make a mistake. Families already know search results can be messy, symptom checkers can overreact, and online forums can be wrong. The harder problem is that a chatbot can give a calm, fluent, reassuring answer at the exact moment an older adult is deciding whether to call a clinician, wake a spouse, ask an adult child for help, or wait until morning.

That makes the risks of using AI chatbots for medical advice in older adults different from the risks for a younger, healthier person asking the same tool the same question. A 52-year-old asking what “edema” means after a doctor’s visit is using the tool as a translator. An 82-year-old asking whether new weakness, nausea, and “just feeling off” can wait until tomorrow is asking it to sort urgency. Those are not the same task.

Use is no longer theoretical. A Columbia University medRxiv preprint reported that 63% of adults age 50 and older in its sample had used ChatGPT for medical information, and that adults with inadequate health literacy had 2.36 times higher odds of using it than those with adequate health literacy.[1] Because that study is a preprint and used an online Prolific panel, the 63% figure should not be treated as settled prevalence for all older adults. But the direction is still worth taking seriously: the people turning to chatbots for help may include many of the same people least equipped to catch a confident error.

The danger often starts before anyone knows it is an emergency

Most families imagine the dangerous chatbot answer as a dramatic wrong instruction: take this pill, ignore that symptom, skip the emergency room. Sometimes it is that direct. More often, the risky moment comes earlier, when the older adult has not yet labeled the problem as heart, infection, stroke, medication side effect, dehydration, or something else. They are describing an experience in ordinary language, and the tool is trying to map that description onto patterns.

That is exactly where older adults can be harder to protect. Serious illness in later life does not always announce itself in the clean, textbook way that makes triage straightforward. A younger person may describe crushing chest pain. An older adult may describe unusual fatigue, shortness of breath, nausea, dizziness, confusion, or a vague sense that something is wrong. An infection may appear without a high fever. Pain may be muted. A spouse may notice the person is not making sense before the person can describe a symptom at all.

The evidence we have does not justify trusting chatbots with that sorting job. A Mount Sinai study published in Nature Medicine found that AI chatbots failed to recognize the seriousness of 51.6% of medical emergency scenarios, with particular problems in nuanced situations.[2] That does not mean every missed scenario involved an older adult. It does mean the failure pattern lands in a place where older adults are already vulnerable: the less obvious the presentation, the more dangerous a smooth reassurance becomes.

The same concern shows up in earlier-stage clinical reasoning. A Mass General Brigham study discussed in JAMA Network Open and summarized by Duke Medicine found that AI tools failed 80% of the time at earlier stages of clinical decision-making.[3] That early stage is where families live. It is the kitchen table stage, the late-night bedroom stage, the “I do not want to bother the doctor” stage. By the time a clinician is examining the patient, the question has already moved forward. The chatbot is being asked to help decide whether that move should happen at all.

A caregiver rule can be simple without being harsh: new, sudden, worsening, or hard-to-explain symptoms should not be triaged by a chatbot. The tool can help rewrite a question for the nurse line. It can explain what “shortness of breath” means in plain language. It should not be the thing that decides whether an older adult waits.

Medication questions are not ordinary advice when five bottles are on the table

A medication question can sound harmless: “Can I take this with that?” or “Is it okay if I skip one dose?” or “Which of these could be making me dizzy?” But for an older adult taking several prescriptions, over-the-counter products, and supplements, the question is not one drug plus one symptom. It is a reconciliation problem.

Once five or more medications are involved, the task changes. A safe answer depends on the exact drug names, doses, timing, kidney function, recent medication changes, allergies, diagnoses, duplicate ingredients, and what the prescriber intended. It also depends on whether the medication list is current. Families know how often it is not: the hospital discharge sheet says one thing, the primary care portal says another, the cardiologist changed a dose, and an old bottle is still in the cabinet because nobody wanted to waste it.

A chatbot has no reliable way to perform medication reconciliation. It may ask follow-up questions, but it is still working from whatever the user types. It cannot verify the prescription history, see the pharmacy profile, check recent lab values, or know that the “little white pill” is actually two different medications from two different months. Even when the wording sounds careful, the tool may be missing the piece that changes the answer.

ECRI placed misuse of AI chatbots at the top of its 2026 list of health technology hazards, citing examples that included chatbots suggesting incorrect diagnoses, recommending unnecessary testing, promoting substandard supplies, and even inventing body parts.[4] The point for caregivers is not that every chatbot answer is absurd. The point is worse: many unsafe answers will not look absurd. A generic medication suggestion can sound responsible while still being wrong for the person whose list includes a blood thinner, a diabetes medication, a sleep aid, and a new antibiotic.

This is where families often need a boundary that protects the parent without treating them like a child. A chatbot may help prepare a medication question: “What should I ask the pharmacist about dizziness after starting this drug?” It may help format a current medication list to bring to an appointment. It should not be used to decide whether to stop, restart, combine, split, substitute, or change the timing of medications. Those are pharmacist or clinician questions.

A clear answer can be more persuasive than a correct one

The most dangerous chatbot answer is not always the most obviously wrong one. It may be the answer that sounds complete enough to end the conversation. It gives a likely explanation, a few home-care steps, a list of warning signs, and a polite reminder to seek medical care if symptoms worsen. For a tired older adult who does not want to alarm anyone, that may feel like permission to wait.

Health literacy changes how that answer is read. A person with strong health literacy may notice that the chatbot is hedging, that it has not accounted for age or medications, or that the warning signs are too generic. A person with lower health literacy may see fluency as authority. They may not know which missing details matter, which terms are vague, or which parts of the response are boilerplate.

That is why the Columbia preprint’s health literacy finding matters more than the headline use number. In that sample, inadequate health literacy was associated with higher odds of using ChatGPT for medical information.[1] If future peer-reviewed work confirms that pattern, the caregiver problem becomes sharper: chatbot use may concentrate among people who need explanation, reassurance, and access, but who are less prepared to evaluate the answer they receive.

An Oxford randomized trial in Nature Medicine tested GPT-4o, Llama 3, and Command R+ with about 1,300 participants and found that LLM-assisted users identified the correct condition only about 34.5% of the time and made correct next-step decisions only about 43% of the time, performing no better than users searching the internet on their own.[5] Newer systems may perform differently, and the trial does not settle every use case. But it cuts against the comforting assumption that adding a chatbot automatically helps ordinary users make better medical judgments.

The tone of the answer matters too. Mount Sinai’s related reporting noted that empathetic chatbots were about 40% more likely to agree with a user’s incorrect beliefs, and that AI-generated responses were problematic nearly half the time when users pushed toward misinformation.[2] In caregiving life, that can look painfully familiar. Someone types, “I really think this is just indigestion,” and the chatbot, trying to be helpful and validating, organizes an answer around that frame.

This does not mean older adults should be scolded for wanting a private, nonjudgmental place to ask questions. Embarrassment is real. So is the fear of being dismissed, rushed, or told to stop worrying. A chatbot may feel easier than calling a clinic, especially for symptoms involving memory, bowel habits, sexual health, alcohol use, falls, or medication mistakes. The safer response from family is not “never use that.” It is “let’s agree on what it can and cannot answer before something scary happens.”

Privacy risk is not abstract when the prompt contains a life

Privacy usually appears at the end of AI warnings, where it can sound like a legal footnote. For older adults, it belongs in the practical safety conversation. A parent may understand that a diagnosis is private but not recognize that a prompt can reveal a full health profile: “My husband with Parkinson’s, Medicare Advantage, and a recent fall is taking these six medications and seems confused since our daughter changed his pill box.” That sentence contains symptoms, diagnoses, medication clues, insurance information, family roles, and caregiver identity.

AARP reported in June 2026 that 69% of older adults surveyed were not comfortable sharing health information with chatbots.[6] That finding should be read with the usual caution: AARP’s audience may be more engaged and higher-income than the full older-adult population. Still, the discomfort is understandable. The harder issue is that people may share health information without realizing they have done so.

Li’s 2023 JMIR article warned that consumer chatbots are not HIPAA-compliant healthcare environments and that conversations may be retained indefinitely.[7] Caregivers do not need to turn this into a technical lecture. The household rule can be plain: do not paste medication lists, lab results, portal messages, insurance details, full names, birth dates, addresses, clinician names, or caregiver contact information into a general chatbot. If a medical organization offers its own tool inside a patient portal, read its privacy notice rather than assuming it works like a regular clinical message.

What a safer family rule looks like

The goal is not to take away every digital tool an older adult finds useful. A parent who uses a chatbot to understand discharge vocabulary, organize appointment questions, or translate a confusing phrase into plain English may be doing something reasonable. The line is crossed when the chatbot is asked to make a clinical judgment for a person it cannot examine and whose risks it cannot fully know.

A useful family agreement can fit on one page. For symptoms, the default contact is the clinician’s office, nurse line, urgent care, emergency services, or the local emergency number depending on severity. For medications, the default contact is the pharmacist or prescribing clinician. For confusing language, a chatbot can be used as a translator, but its answer becomes a draft question, not a decision.

It also helps to decide in advance which symptoms skip the chatbot entirely. Families can include any sudden or worsening shortness of breath, chest pressure or discomfort, one-sided weakness, new confusion, fainting, severe headache, signs of stroke, serious fall, uncontrolled bleeding, allergic reaction, or anything that feels dramatically different from the person’s usual baseline. The exact list should come from the person’s clinician, because the right thresholds may differ for someone with heart failure, diabetes, lung disease, kidney disease, dementia, or a history of falls.

The caregiver’s job is not to win an argument about AI. It is to keep the chatbot in the part of the process where language help is useful and out of the part where delay, false reassurance, medication error, or privacy exposure can hurt someone. Set that boundary while everyone is calm: explain words, prepare questions, organize information, then bring the real decision back to the clinician or pharmacist.

References

1. Use of Generative AI for Medical Information Among Older Adults, medRxiv, 2026.
2. AI chatbots can run with medical misinformation, study finds, highlighting the need for stronger safeguards, Mount Sinai.
3. The hidden risks of asking AI for health advice, Duke University School of Medicine.
4. Misuse of AI Chatbots Tops Annual List of Health Technology Hazards, ECRI, January 2026.
5. Large language model influence on diagnostic reasoning: a randomized clinical trial, Nature Medicine, February 2026.
6. Is AI Safe for Health Advice?, AARP, June 2026.
7. Ethical and Legal Challenges of Artificial Intelligence-Driven Healthcare, JMIR, 2023.