Listen to the audio version of this article (generated by AI).
Artificial intelligence (AI) is no longer a distant prospect in diabetes care. Algorithms already screen retinal images, predict changes in glucose, adjust insulin delivery and sift through expanding stores of health data. Newer systems promise to support education, clinical decisions and communication between people living with diabetes and their care teams. AI could ease the relentless work of living with diabetes. But that promise means little unless systems are designed with people with diabetes in mind, tested in real care settings and held to account when they fail.
The harder question is no longer simply what AI can do. It is who decides how it is built, how carefully it is introduced, and who watches it once it is in use and who answers when it causes harm.
In a recent International Diabetes Federation (IDF) D-Talk podcast on responsible AI, Dr Amit Kumar Dey and Dr Elaine Chao, chair and member respectively of the IDF Technology and AI Working Group, argued for progress without complacency. Their central point was straightforward: AI must serve people living with diabetes, work within real lives and health systems, and remain open to scrutiny long after launch.
“Artificial intelligence must be used in the most seamless manner. It must fit into the workflow of the individual, and not the individual that has to fit into the workflow of the AI tool” said Dr Amit Kumar Dey.
The harder question is no longer simply what AI can do. It is who decides how it is built, how carefully it is introduced, who watches it
AI is routinely sold through superlatives: faster, smarter, more accurate and more personalised. Healthcare, however, is not an ordinary technology market. A confident but wrong restaurant suggestion is irritating. A confident but wrong diabetes management recommendation can be dangerous.
That is why the idea of “responsible and humble AI” matters. It asks developers and health systems to under-promise, resist hype and be candid about what a system cannot do. Humility means stating where and with whom a tool was tested, showing uncertainty rather than concealing it, and recognising when human judgement must take over. It is, in essence, a technological version of medicine’s duty to do no harm.
This approach is consistent with World Health Organization guidance, which says health AI should protect human autonomy, promote safety and well-being, be transparent and accountable, advance inclusion and equity, and remain responsive and sustainable. Those principles should govern the whole process: which problems are chosen, whose data are used, how products are tested and what counts as success.
For diabetes management, success cannot be reduced to a model’s accuracy score. The real measure is whether a tool makes life safer, easier and less mentally exhausting. Diabetes self-management involves an unending stream of decisions about food, medication, movement, sleep, illness and glucose. Dr Chao described responsible AI as technology that improves health while easing the cognitive load. The human outcome, not the technical spectacle, must come first.
“AI in diabetes care should be serving people with diabetes, number one. It should be improving their health—or making their lives better, healthier, safer, reducing the mental burden,” she argued.
For diabetes management, success cannot be reduced to a model’s accuracy score. The real measure is whether a tool makes life safer, easier and less mentally exhausting
A digital tool can work in a trial and still be miserable to use. It may require repeated data entry, trigger constant notifications, or force clinicians to switch between systems that do not communicate with one another. Every extra click, alarm and dashboard risk becoming another piece of unpaid work.
Technology can contribute to burnout among healthcare professionals and people living with diabetes alike. The concern is not theoretical: studies have linked poor electronic health record design and usability with clinician stress and burnout. AI that is poorly integrated may simply pile more alerts, documentation and checks onto an already fragmented workday.
The same problem appears in diabetes devices. Glucose alerts can be vital, especially in warning of hypoglycaemia. But excessive or poorly personalised alarms can interrupt sleep, increase anxiety and eventually lead people to ignore the technology. Responsible AI should cut through noise, separate urgent warnings from routine information and adapt to individual needs and preferences.
User interface and user experience design are therefore matters of safety, not decoration. The discussion calls for AI that works “ambiently”: it should fit around the routines of a person or care team, rather than forcing them to reorganise their lives around a machine. A good system delivers useful information at the right moment and stays quiet when it has nothing worthwhile to add.
All too often, developers build health technologies almost to completion before inviting the people expected to use them into the room. By that stage, consultation may be limited to choosing colours, changing labels or testing navigation, even though the product’s purpose and underlying assumptions have already been set.
Developers should involve people living with diabetes earlier in the process. Working alongside caregivers and healthcare professionals, they should help define the problem, shape the technology, test it, and evaluate its performance in real-world care.
This participatory design encourages sharing influence rather than merely requesting feedback. Human-centred design research suggests that involving users can help teams identify hidden biases, contextual risks and workflow failures that technical testing may miss. In diabetes care, lived experience helps determine whether a system is practical, respectful and suitable for the diverse people and settings in which it will be used.
Participation also has to be broad and properly supported. A single advisory panel cannot stand in for people with different types of diabetes living in radically different health systems. Developers should include underrepresented groups, value their expertise and show how their contributions shaped the technology.
In diabetes care, lived experience helps determine whether a system is practical, respectful and suitable for the diverse people and settings in which it will be used
Many AI systems look impressive when tested retrospectively on existing datasets. That is useful but limited. It does not show how the system will alter decisions, workload, safety or health outcomes when people begin relying on it in the real world.
AI systems should therefore undergo prospective clinical evaluation, including randomised trials where appropriate, and be tested in the populations and settings for which they are ultimately intended.
Established reporting frameworks point in the same direction:
Evaluation must ask more than whether a prediction was correct. It should assess whether the system improves care without creating new risks, burdens or inequalities. It should also expose the system to the conditions that polished datasets omit: missing information, device failures, weak connectivity and unexpected human behaviour.
Explainability matters, but it is not a magic solution. Complex models cannot always offer a neat physiological account of every output. Responsible use requires clear limits, transparent evidence, human oversight and a safe fallback when the system fails or produces questionable advice.
Evaluation must ask more than whether a prediction was correct. It should assess whether the system improves care without creating new risks, burdens or inequalities
A medicine is not considered permanently safe just because it passed one trial. Side effects are tracked after approval; new evidence is reviewed; and action can be taken when harm emerges. AI needs the same culture of vigilance.
That is particularly important because an AI system can become less reliable even when the software appears unchanged. The population may shift, clinical practice may evolve, sensors may be updated or incoming data may no longer resemble the information used to train the model. This “drift” can gradually reduce performance and may affect some groups more severely than others.
This ongoing duty is known as lifecycle accountability. Once an AI system is in use, its recommendations, errors and adverse events should be recorded and audited. Serious failures should be investigated, with clear authority to modify, suspend or withdraw the system when necessary.
Regulators are beginning to adopt this lifecycle view. WHO’s regulatory considerations call for health AI to be well-documented, independently tested, and governed through shared responsibility across the healthcare system.
For healthcare organisations, buying an AI product should mark the start of governance, not its conclusion. Every deployment needs named owners, agreed monitoring measures, regular reviews, incident reporting, update procedures and a clear point at which use will be stopped.
When AI contributes to a harmful decision, responsibility can quickly become blurred. The developer points to the clinician, the clinician points to the software, and the health system points to regulatory approval. That circle of blame is not accountability.
Healthcare professionals using AI-supported recommendations must retain responsibility for clinical decisions, particularly while the limits of these systems remain poorly understood. But responsibility is also shared among developers, health services, regulators, policymakers and users. Shared responsibility must not become “nobody’s responsibility”.
The duties of each group need to be explicit. A clinician may answer for an individual care decision but cannot reasonably be expected to uncover a concealed flaw in training data or track model drift across an entire health service. Developers must demonstrate performance and disclose limitations. Health services must assess local suitability, train staff, and monitor use. Regulators must set enforceable rules. Policymakers must provide routes for redress and prevent commercial secrecy from obstructing safety scrutiny.
AI literacy is part of this bargain. Healthcare professionals do not need to become software engineers, but they do need to understand a system’s intended use, data limitations, common failure modes and uncertainty – and know when to reject or escalate an output. People living with diabetes deserve equally clear information about when AI is being used, what data it processes and how they can question a recommendation.
AI literacy is part of this bargain. Healthcare professionals do not need to become software engineers, but they do need to understand a system’s intended use, data limitations, common failure modes and uncertainty
Responsible AI is not just a model. It depends on reliable data, interoperable systems, privacy safeguards, connectivity and people with the time and skills to use it. Without those foundations, AI may widen the inequalities it is often advertised as solving.
Interoperability – the ability of devices and information systems to exchange and use data – is essential if AI is to support care without creating yet another set of fragmented records. Standards such as HL7 FHIR provide a common framework for electronic health information exchange. But technical compatibility is only part of the task. Data must also be meaningful, complete and governed with proper respect for consent and local law.
Federated and swarm learning offer possible ways to learn from data held in different places while keeping raw records closer to where they were collected. Federated learning can enable institutions to collaborate without storing every record in a single central database. It does not, however, eliminate privacy, security or bias risks. Used with care, such methods could help make diabetes datasets more representative. Swarm learning enables several hospitals, clinics, or devices to train a shared AI model without sending all their raw data to a single central database.
Equity also means choosing the right technology, not automatically the most sophisticated one. A lightweight tool that runs on an affordable phone, works in a local language, and tolerates intermittent connectivity may help far more people than a powerful system that depends on costly devices and specialist infrastructure. As Dr Dey put it, the goal is the right care for the right person at the right time, using the right technology – not necessarily the technology with the grandest claims.
Some fragmentation is inevitable, as countries are writing AI rules at different speeds and through different legal systems. But the basic expectations for health AI should not change at borders. People living with diabetes everywhere deserve systems that are safe, transparent, clinically validated, inclusive and monitored after deployment.
International organisations, professional bodies and diabetes communities can help establish those shared expectations. The International Diabetes Federation has a particular responsibility to ensure that global policy is shaped not only by governments and technology companies, but by people living with diabetes and those delivering care across widely different settings.
Five years from now, progress should not be judged by whether the newest model appears more autonomous or more impressive. A better test would be whether there is a trustworthy tool, co-designed with its users, prospectively validated, woven into care without adding burden, monitored throughout its life and accessible beyond the world’s richest clinics.
The final question is disarmingly simple: does the technology improve quality of life for the person with diabetes? If the answer is uncertain, the system is not ready – however advanced its developers say it is.
A position paper developed by the IDF AI & Technology Working Group, the Geneva Review posits AI could improve diabetes prevention, diagnosis, monitoring, treatment and self-management. However, such systems must be designed around the needs and lived experiences of people with diabetes and evaluated in real-world clinical settings. The report calls for accountable and “humble” AI. This means using high-quality, interoperable data. It also means reducing bias, explaining decisions clearly and involving patients in design. AI tools should be affordable and work in low-resource settings. They should also be monitored after launch for errors, unfair outcomes and harm. The report’s main message is simple: AI should be judged by whether it improves care safely, fairly and transparently.
Download The Geneva Review and discover what accountable AI in diabetes care should look like.
In this D-Talk podcast, we explore how technology and artificial intelligence are reshaping diabetes care, from earlier detection of complications to more personalised support for people living with diabetes. Our guests, Dr Elaine Chow and Dr Amit Dey, discuss the promise of AI and the responsibility that comes with it: ensuring tools are safe, transparent, equitable and designed around real patient needs. They explain why people living with diabetes must be involved in shaping these technologies and why stronger governance, better data systems and inclusive design will be essential as AI moves from innovation to real-world impact.
