By Leigh Kamore Haynes
Introduction
As large language models (LLMs) and generative artificial intelligence (AI) tools have improved and become widely available, they have sprawled into nearly every facet of people’s lives, including our health care and social systems. While AI tools and AI-powered platforms offer promising opportunities to enhance the effectiveness and efficiency of programs and processes, it is important that their implementation and use do not compromise human rights or exacerbate existing health inequities. Practitioners and advocates in the field must anticipate potential threats and call attention to them to ensure that any harm or injustice is prevented.
OpenAI, the creator of ChatGPT, envisions AI as part of everyone’s daily activities, like electricity. It imagines AI operating across the breadth of life saying that AI “can help someone navigate a medical bill, learn a new skill, start a small business, care for an aging parent, … or make a scientific discovery.” The really wonderful thing about AI tools is that it brings the possibility of imagining the multitude of ways they may be deployed, as OpenAI has done, to save time, improve workflows, and catalyse discovery. What more often goes neglected or disregarded are the challenges of carrying out implementation in safe, unbiased ways that do not diminish or restrict people’s rights, including the right to health. Human rights researchers, practitioners, and advocates should remain actively engaged in the discourse around AI, the policy discourse in particular, to call out risks–based on evidence and lived experience–and propose solutions that protect human rights and promote justice.
Situational AI implementations: Imagining the risks alongside the opportunities
Alongside the imagining the opportunities and potential that AI brings to everyday life and work, we must similarly imagine a protective environment in which it can function. This creates an imperative to proactively consider how straightforward, practical, and well-intended implementations of AI tools may cause harm if protective measures are not put in place. The following anecdotal examples aim to demonstrate the type of thought process that should accompany the imagining, development, and implementation of AI tools.
AI-powered predictive analytics are used in health care settings with the aim of improving delivery of care and patient outcomes. For example, a predictive AI model used in a hospital can scan electronic medical records to identify trends in conditions patients present with across time (infectious, chronic, emergent). This data is then used to predict staffing, number of beds needed, pharmaceutical orders, and other needs deemed to improve efficiency and responsiveness of the facility. These products are marketed and sold for use at any hospital. It is necessary to consider, though, that a model trained on one hospital's population may not be adequate for deployment elsewhere. Predictions the AI makes may lead to understaffing, misallocation of beds, or a lack of supplies needed for a facility that may not have the funding or technology support to validate the AI tool locally for its own patient population. This potentially leaves poorer, under-resourced hospitals, which most often serve those already facing worse health outcomes, with a tool that underperforms and may deepen health inequities, especially with regard to quality, accessibility, and equity, including in patient outcomes.
These risks should also be examined as AI is deployed in settings that may affect the broader determinants of health. States have adopted automated decision-making systems which screen and review applications for benefits to “streamline bureaucratic processes and address … rising welfare needs and inefficiencies in benefit delivery.” Many of these systems can be (and are being) bolstered by AI which may be able to cross-reference an application with a list of keywords the state agency has provided and score the application based on pre-determined weights assigned to each keyword and make a rapid decision. If the application scores below a certain threshold, the AI automates denial a letter to the applicant. However, it is important for those concerned with the right to health and access to social determinants of health to reflect on the rights implications of such an AI tool. In practice applicants with complex circumstances that should receive human review or those who simply have incomplete files, risk being disproportionately screened out. As a result of a denial based on these circumstances, these individuals may face increasing debt, housing or food insecurity, and other challenges that have severe consequences for physical and mental health. Similar risks have previously surfaced in the Dutch ‘SyRI' case in which algorithmic detection of social welfare fraud was found to violate the right to exercise private life as provided for in the European Convention on Human Rights.
Finally, AI can have further impacts at the population level. Public health agencies rely on wastewater-based epidemiology, powered by machine learning AI, as a method for early detection and monitoring of the spread of infectious diseases. This practice has expanded to monitoring the consumption, use, and exposure to illicit substances. This is important public health data to collect, but those concerned with protecting and promoting human rights should consider the myriad ways that data may be used. For example, the data may be collected for public health, but if it is entered automatically into a database that is accessible to other “public safety” agencies, such as police, a problem arises. This public health tool risks morphing into an instrument of surveillance allowing for neighbourhoods with higher detected traces of illicit substances being stigmatised and targeted for policing – rather than a public health intervention, such as harm reduction – while chilling effects make communities and researchers wary of participating in surveillance that truly serves public health.
Policy approaches to minimise risks can be imagined as AI scales up
The aforementioned scenarios indicate how integration of artificial intelligence into health and social systems presents new challenges for sustaining action at all levels required to promote health equity and uphold the right to health. As AI proliferates, it becomes imperative to build guardrails that protect human rights and the right to health. Anchoring practice in human rights principles – namely, accountability, indivisibility, non-discrimination, participation, transparency, and universality – can promote transparency around where and how AI is used. Adhering to these principles may also promote meaningful participation of affected communities in design and deployment as well as prioritising human review of any automated decisions or outcomes.
Moreover, responsible AI principles provide an additional framework by which to prevent any human rights harms that may collaterally occur due to use of AI tools. Responsible AI principles include accountability; diversity, non-discrimination, and fairness. They also demand human agency and oversight, privacy and data governance, technical robustness and safety, and transparency. Finally these principles look further afield to social and environmental wellbeing, including environmental and labour impacts throughout the entire AI supply chain.
AI impact assessments may also provide an indication of how AI may affect people’s health and lives, including enjoying human rights. For example, under Article 27 of the EU AI Act, certain deployers of high-risk AI systems must conduct a fundamental rights impact assessment before putting those systems into use. But institutions should certainly go further, implementing expansive assessment procedures that also cover general-purpose AI systems and broader environmental and social impacts before deployment.
Finally policy and processes for the regulation and oversight of AI should aim to guarantee equitable access so that AI tools and their benefits are fairly available to all communities. Where barriers to access are identified, companies and governments should make meaningful and concerted efforts to address the underlying social injustices and other causes that impede equity.
Conclusion
AI is rapidly becoming part of systems that shape people’s everyday lives and well-being, from how we experience the health care system to how authorities make decisions to profile entire communities. In their promotion of the technology’s promise AI company founders and leaders in tech neglect to raise awareness of the risks of AI. These risks pose threats to the right to health and human rights more broadly. Accordingly, it is critical for researchers, practitioners, and advocates to call attention to these risks and propose solutions that promote equity and justice as AI continues its rapid scale-up.
Bio
Leigh Kamore Haynes, JD, MPH, LLM is an Associate Professor of Practice in the Department of Public Health at Simmons University and PhD Fellow in the Faculty of Political and Social Sciences at Ghent University.