Can an AI risk message raise flu vaccination?
Three preregistered US field trials found that telling algorithm-identified patients they were at high risk produced a small absolute rise in flu vaccination. Naming the algorithm or explaining its reasons did not add a measurable benefit.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1Three preregistered randomized trials at a US health system generated 116,811 trial assignments across three flu seasons and included more than 90,000 unique patients; some people could appear in more than one season.
- 2Compared with a simple vaccine reminder, a high-risk message increased vaccination by 1.1–1.4 percentage points, a relative rise of 3.3–5.4%. Compared with no study message, the increase was 1.7–3.5 points, or 3.3–14.7% relative.
- 3Saying that an algorithm was involved, saying medical records were analysed, or providing personalized reasons for the risk prediction did not measurably improve vaccination beyond communicating high risk itself.
Research topic
Whether messages telling algorithm-identified patients that they faced high influenza-complication risk increased vaccination, and whether disclosing or explaining the algorithm changed the effect
The evidence comes from three real-world randomized trials
Gail Rosenbaum and colleagues tested a practical question rather than a laboratory attitude: when a health system's machine-learning model identifies people at high risk of influenza complications, does telling them that risk change whether they receive a flu vaccine? The team ran three preregistered parallel-group field trials at Geisinger, a health system serving patients in Pennsylvania and nearby areas, across the 2020–21, 2021–22 and 2022–23 flu seasons. Vaccination recorded in the health system was the primary behavioural outcome; it was not a survey intention or a model benchmark.
The published extended-data denominators total 116,811 randomized assignments: 35,201 in study 1, 42,481 in study 2 and 39,129 in study 3. The article reports more than 90,000 unique patients because people could remain eligible and be randomized again in a later season. That distinction matters. The trial unit within each season was a patient assignment, but the three-study total is not 116,811 different people. The repeated-season design strengthens replication across time while preventing a simple interpretation of the pooled assignment count as an independent headcount.[1][2][3][4]
The Impact Brief · Free
Follow the evidence in health & life sciences.
Get a five-minute weekday briefing on what changed, why it matters and where the evidence comes from. Choose the topics you care about.
Patients were randomized to different explanations of the same risk flag
The health system had partnered with Medial EarlySign to score the risk of influenza complications from coded electronic-health-record data. In the first registration, eligible patients were in the model's top 10% of predicted risk, were at least 17 years old and had no recorded vaccine contraindication. Patients were randomized to receive no extra study message, a message saying they were at high risk, a high-risk message saying their medical records had been analysed, or a message adding that an algorithm or machine-learning analysis had produced the result. Later trials expanded the message comparison and used five arms, including an ordinary vaccination reminder as an active control.
The messages were delivered through the channels already available to the health system, including letters, text messages and secure patient-portal messages. Study 1's registered design masked patients to the existence of alternative study arms and did not tell care providers which patients were assigned to which communication. Studies 2 and 3 each split roughly 42,000 and 39,000 assignments across five similarly sized arms. The design therefore isolates small wording differences inside a live outreach programme, but it cannot isolate every surrounding influence: the pandemic, seasonal vaccine campaigns, prior contact with the system and changing baseline uptake all differed across years.[1][2][3][4]
The high-risk message produced a small but measurable lift
Across the trials, telling patients that they were at high risk increased recorded vaccination. Relative to patients who received a simple reminder to get vaccinated, the high-risk message added 1.1–1.4 percentage points, equivalent to a 3.3–5.4% relative increase. Relative to patients who received no study message, the gain ranged from 1.7 to 3.5 percentage points, or 3.3–14.7% relative. The two percentage forms answer different questions: percentage points describe the change in the share vaccinated, while the relative percentage depends heavily on the control rate.
Baseline vaccination also varied sharply by season. The passive-control rates shown in the paper's extended data were 50.8% in study 1, 31.5% in study 2 and 23.8% in study 3. A similar absolute gain therefore becomes a larger relative percentage when the starting rate is lower. For a health system sending tens of thousands of low-cost messages, one or two additional vaccinations per hundred contacted people may still be operationally useful. It is not evidence that personalized risk communication overcomes all access barriers, distrust, contraindications or hesitancy, and it should not be advertised as a dramatic individual effect.[1]
Naming the algorithm and explaining it did not add a detectable benefit
The most directly AI-related result is a null comparison. Vaccination was similar whether the message disclosed that an algorithm helped determine the risk, described an analysis of medical records without the algorithm label, or supplied personalized reasons for the prediction. In this setting, patients were neither measurably discouraged by the algorithm disclosure nor more persuaded by it. The operative content appears to have been the risk signal itself: the health system believed the patient faced elevated danger from influenza complications.
A null result is not proof that explanations never matter. These were brief outreach messages for a familiar preventive action already recommended for most people, not a consent conversation about diagnosis, treatment or denial of care. Patients could ignore the message without delegating a clinical decision to software. The study was powered for practical differences around two percentage points; smaller effects, subgroup effects or changes in trust may remain undetected. It also did not compare technically detailed explanations, interactive contestability or disclosure of model uncertainty.[1]
The study measures vaccination, not model accuracy or clinical benefit
The primary endpoint is consequential because it records behaviour, but it leaves two separate questions unanswered. First, the trials do not validate the machine-learning risk model in this article. Eligibility depended on a previously developed model, and the intervention tests communication among people it flagged. If calibration, subgroup performance or data quality were poor, a successful message could still target the wrong people. Second, an increase in vaccination is not the same as a demonstrated reduction in influenza, hospitalization or death; those events are rarer and need much larger or separately powered analyses.
The study also took place within one integrated US health system. Its patients had known contact routes and electronic records rich enough to run the model. Effects may differ where vaccination is paid for differently, delivered outside the originating health system, recorded incompletely, or constrained by supply, appointment access and language. People with limited portal use, unstable addresses or distrust of record-based profiling may receive or interpret messages differently. The paper's competing-interest statement notes that Geisinger paid Medial EarlySign to generate the risk scores and reasons; the authors report no other competing interests.[1][2][3][4]
The practical lesson is to separate the nudge from the model
For health services, the result supports a narrow implementation claim: a concise, credible high-risk message can add a modest amount of vaccination uptake among patients already selected by an algorithm. Teams should evaluate the whole pipeline separately. Model performance determines who is selected; message delivery determines who sees the intervention; wording affects response; and clinical operations determine whether an interested person can actually obtain a vaccine. A positive final uptake effect cannot reveal which earlier stage failed for everyone else.
The intervention should therefore be monitored with absolute numbers and denominators. Systems need delivery rates by channel, vaccination capture inside and outside the network, model calibration across demographic and clinical groups, opt-out rates, complaints and missed high-risk patients. Because the article found no added uptake from naming the algorithm or listing reasons, disclosure should not be optimized solely as a persuasion tactic. Transparency, lawful data use, the ability to ask questions and a route to correct records remain governance duties even when they do not increase clicks or vaccinations.[1][2][5]
What would change the assessment
Confidence in transferability would rise with preregistered replication in other health systems and countries, including safety-net settings, different languages and services where vaccination data arrive from pharmacies and public clinics. Reporting effects for delivery channel, age, deprivation, race and ethnicity, prior vaccine behaviour and baseline model risk would show who benefits and who is missed. A design that randomizes both message and access support—for example, a scheduling link, transport help or a no-cost walk-in option—could distinguish informational barriers from practical ones.
The model side also needs contemporaneous external validation and fairness analysis, with calibration and error rates reported for every group used operationally. Longer follow-up should test laboratory-confirmed influenza and serious complications rather than assuming that a small uptake gain produces an equal clinical benefit. Until then, the best-supported conclusion is modest: communicating high algorithm-estimated risk raised vaccination a little in three US field trials, while algorithm disclosure and personalized reasons did not measurably add to that effect. The work supports careful experimentation with outreach, not a blanket case for AI-driven medical messaging.[1][2][3][4][5]
What this means for people
- A brief high-risk message may prompt a small additional share of selected patients to receive a recommended vaccine.
- People may be targeted using health-record risk scores without seeing how selection errors or uncertainty were handled.
- Naming the algorithm did not change uptake here, but patients still need understandable disclosure and a way to correct data or ask questions.
Global context
The trials were run in one US health system and depend on its records, outreach channels and vaccine-delivery arrangements. Other countries may have more complete national vaccination records, different consent rules and stronger or weaker primary-care access. Transfer should therefore start with local validation of both the risk model and the service pathway, not with a copy of the message alone.
What the evidence does not yet show
- The three seasons produced 116,811 randomized assignments but more than 90,000 unique patients; some people could contribute in more than one season.
- All trials were conducted within one integrated US health system, limiting geographic, financing and delivery-system generalization.
- The article tests communication after algorithmic selection; it does not independently establish the risk model's calibration, fairness or clinical validity.
- Absolute vaccination gains were small, and a larger relative percentage in a low-baseline season should not be confused with a large individual effect.
- No added vaccination effect from algorithm disclosure or explanation does not establish that transparency is unimportant for trust, consent, legality or higher-stakes clinical decisions.
- Recorded vaccination is a behavioural endpoint, not direct evidence of fewer influenza infections, hospitalizations or deaths.
What to watch next
- Replication outside Geisinger, in other countries and in multilingual or lower-access populations.
- Independent validation of the risk model's calibration and error rates across demographic and clinical groups.
- Trials that pair risk messages with access support and capture vaccinations delivered outside the originating health system.
- Clinical outcomes and trust measures, not vaccination uptake alone.
Living evidence record
Impact record IAI-1R4WM2Y
Evidence stage
Studied
Confidence
Corroborated
Reporting basis
Multi-source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
5 October 2026
Source trail
5 direct sources across 2 source types.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Evidence trail
Sources used for this report
Links checked 5 October 2026
This report is labelled multi-source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
Health & Life Sciences
Can an ECG AI explain its risk predictions?
A language-model framework reached AUROCs of 0.98 for a composite cardiovascular outcome and 0.86 for mortality in an internal 10,000-patient holdout. The evidence comes from one Ohio health system, and its explanations were judged for plausibility rather than proved correct.
10 min · 1 source
Health & Life Sciences
Can AI narrow the search for a phage—and what would prove a patient benefit?
A new strain-level prediction study offers a way to rank laboratory candidates. We compare its evidence stage with clinical-trial requirements and the wider antimicrobial-resistance challenge.
6 min · 3 sources
Health & Life Sciences
Did web search make GPT-5 a better diagnostician?
Not reliably. In a peer-reviewed benchmark of 161 image-based clinical cases, the strongest reasoning configuration reached 46.0% diagnostic accuracy. Adding web search cut the simplest configuration from 38.5% to 26.7%, while agent and deep-research modes also trailed that baseline.
11 min · 1 source
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.