The autonomous capabilities of emerging AI systems pose societal risks if consequential decisions are based on flawed information and are unguided by appropriate ethical parameters. This essay proposes a process for determining the level of oversight – informed by ethical considerations – needed for safer AI.
This essay was 100 per cent written by a human being.
Not long ago, starting an essay with this proposition might have been seen as very odd. Today, the fact that this seems an increasingly reasonable warranty is testament to the meteoric rise of generative artificial intelligence (GAI). In this new paradigm, AI models are able to create information by learning how to mimic the data in their training datasets, generating convincing examples of photos, computer code, music – or essays like this one.
As humans, our learning processes and definitions of truth and reality are connected to empirical observation. For decades, computational models have manipulated who sees what. With its newfound generative capabilities, GAI is taking this manipulation to the next level. When creating synthetic data, GAI is not just redefining creativity but also convincingly challenging our natural trust in the information we perceive through our senses. Is the politician’s speech that is going viral on social media real or fake? Am I talking to a human customer representative, or is a polite bot replying to my chats? How can we be sure what is real, ever again?
At present, the implications of GAI for the future of democracy, the economy, labour and education are unfathomable, in terms of both the thrilling possibilities and chilling risks. This essay explores some of the new challenges that this generative future presents to the already-complex landscape of AI ethics, underscoring how critical it is for society to reach timely agreement on policy frameworks for ethical innovation.
AI garbage in, AI garbage out
Despite the attention it is now attracting, the field of AI is not new. It has been around for decades, and has seen an evolution in approaches and techniques. The latest advancements arise in part from developments in machine learning models, enabled by the conjunction of data availability (thanks to the ubiquity of the internet and smart devices), mathematical research, increasing computing power, and lower costs for data storage and retrieval.
While previous approaches such as expert systems were about codifying logic and rules of knowledge, machine learning can be thought of as ‘learning by example’. When you have seen enough pictures of cats, you can recognize one in new information, even if you cannot explain the rationale behind how you know it to be a cat.
At its core, an AI model is a stack of algorithms. An algorithm is a sequence of instructions to transform an input into an output. In a way, we can think of it as a recipe: ingredients (input) are processed according to prescribed steps to achieve a result (output). Crucially, the final product depends both on the quality of the ingredients and on using the right recipe. It is impossible to bake an apple pie if bananas are the input, or if the recipe was created by a system trained with oranges. The same goes for AI models: biased data lead to biased results. If garbage goes in, garbage comes out.
Machine learning makes AI models extremely dependent on data collection, which in the context of our current internet landscape has contributed to the rise of ‘surveillance capitalism’.
Machine learning makes AI models extremely dependent on data collection, which in the context of our current internet landscape has contributed to the rise of what Shoshana Zuboff has called ‘surveillance capitalism’. Targeted advertising has become the lifeblood of digital economies, shaping both how digital platforms and its products are designed and how we interact with them.
Before the current wave of GAI, AI models were focused on prediction and classification tasks, like recommendation engines suggesting the next video to watch, or which products might be appealing for specific consumers based on their purchase history. More worryingly, such systems started to be deployed to predict future behaviours in delicate and highly contextual matters, such as the likelihood of someone defaulting on a mortgage payment or committing a crime.
These types of statistical systems look into the past to try to predict the future. They extract features from data to build a model that interprets the world and aims to predict future outcomes to similar problems. While mathematically sound, this proposition has a fundamental problem: as they aim to capture and describe a situation in the real world, AI models replicate the biases and inequalities in the reality that they purport to observe. Consequently, those biases and inequalities are perpetuated in the outputs and projected into the very same future AI tries to predict. This creates a vicious cycle and makes AI systems conservative and risk-averse, biased towards the status quo. It also risks history repeating itself: if data show that some jobs, roles and industries have been male-dominated, a model developed to extract the best candidates in the same fields will tend to pick people with similar profiles. Early recruitment models for traditionally male-dominated roles, for instance, discriminated against female applicants.
Furthermore, even if AI models can simulate intelligent results, they lack contextual awareness and common sense, which makes them unsuitable for dealing with nuanced linguistic tasks, pondering values or moderating content.
Foreseeable challenges posed by GAI: beauty, truth, hallucinations and anthropomorphization
Understanding AI blind spots and ethical problems is critical because, despite mitigation strategies and technical safeguards, those flaws are being carried on into GAI and will be true for whatever AI breakthroughs come next.
Historically, dealing with bias has been challenging for AI. This is especially the case for generative models because the biases embedded in the content they produce create a new layer of digital reality in terms of visual languages or factoids. By reflecting reality through distorted lenses, and releasing back into the world content according to that point of view, GAI is creating ontological aesthetics and semantics, producing new cultural signifiers. As the internet becomes flooded with synthetic content, the stereotypes, misconceptions and falsehoods produced by AI systems spill over into people’s actual beliefs and perceptions, effectively becoming ‘real’ as they are assimilated by society. This creates another vicious cycle, as today’s synthetic content becomes tomorrow’s training data, perpetuating bias into the future. For example, GAI image generators prompted to create an image of a doctor are likely to produce an image of a male doctor. AI-generated images are also redefining ideals of beauty by defaulting to hegemonic, unattainable and synthetic standards of physical perfection that risk exacerbating body dysmorphia rampant among vulnerable social media users.
AIs’ blurring of fact with fiction is creating novel problems for users, including legal risks associated with reliance on AI-generated text and ‘analysis’ in situations where real-life accuracy is demanded. This is not necessarily the result of bad actors. Large language models (LLMs) are complex pieces of intellectual machinery that can fail, providing confident-sounding but spectacularly wrong answers. In computer science, these faux pas have been christened AI ‘hallucinations’ or ‘delusions’ – instances in which AI models fabricate information entirely, while confidently behaving as if they are stating facts.
One of the education challenges ahead is to strengthen humanity’s critical thinking skills in relation to confidently presented errors or misrepresentations of fact.
This connects with another rising trend, unwarranted human reliance on AI systems as oracles and authoritative sources. Just as people tend to trust confident-sounding speakers, we should consider carefully the bond of ‘epistemic trust’ that is developing between humans and LLMs. When one considers also how the replies provided by LLMs are generally detached from sources, one of the education challenges ahead is to strengthen humanity’s critical thinking skills in relation to confidently presented errors or misrepresentations of fact.
GAI models are also susceptible to specific security risks. In a sort of AI hypnotic suggestion, attackers can manipulate the prompts given to AI models to induce forced answers. These ‘prompt injections’ can hijack and override safeguards, poisoning the resultant output, which will be provided according to the new instructions, inadvertently to the user. Other vectors attack GAI models in a way similar to social engineering, by persuading or confusing the model, tricking it into providing answers or overriding safety guardrails.
Considering how internet algorithms rank and position information according to relevance, if false information is disseminated and repeated enough by the right sources, it will find its way to the front page of search results. And from then, it is a short data-mining step away from that information ending up in training datasets. As with propaganda, synthetic truths can crystallize into ideas systemically presented as real in a sort of self-fulfilling prophecy.
Another looming problem is emotional manipulation and attachment. As GAI creates agents able to interact in real time in a way that can be tailored to specific users, there are already reported cases of emotional bonding of humans with AI bots. Conversely, a journalist’s conversation with Microsoft Bing’s chatbot took a bizarre turn when the system declared its love for him and suggested that he break up with his wife.
A related problem is anthropomorphization, in which human qualities, emotions or intentions are attributed to AI systems. The conversational nature of people’s interaction with LLMs, as well as the fact that AIs are sometimes embodied in human-looking robots or avatars, contributes to the confusion. Sometimes there are legitimate debates about the consciousness and personhood of AI systems, but more frequently than not the question is raised for shock value or the sake of marketing. What is certain, however, is that anthropomorphization can be a distraction from pressing and practical ethical concerns about the use of AI, and that this can contribute to public misrepresentation of AI’s capabilities and limits.
Understanding the different dimensions of AI ethics
Despite the challenges outlined above, the temptation to implement AI for the sake of automation is high. Public institutions are particularly suggestible to the promises of modernization, security and efficiency. AI systems are already being deployed in areas like justice and surveillance without proper safeguards, assessments, or possibilities for recourse in the event of error. Ethically, determining when and where to implement automated decision-making systems and how they affect society is complex. Not every problem can or should be the subject of an automated solution, certainly not now and perhaps not ever.
Ethically, determining when and where to implement automated decision-making systems and how they affect society is complex. Not every problem can or should be the subject of an automated solution, certainly not now and perhaps not ever.
These are not decisions reserved solely for big corporations or governments. Today, everyone interacts with automated systems in different capacities, in the workplace, in public spaces or in private life. Some people may have room to choose to avoid AI tools, but others have no such choice. There are, however, some principles that might inform a more responsible approach to the use of these models, and some considerations that should be taken into account. To help academics, policymakers or concerned citizens navigate this issue in an informed manner, this essay presents a framework that breaks these complex problems into a sequence of analytical steps that can be adapted as needed to different situations.
The first step is to create an ethical AI matrix weighing the context, potential harms and potential gains so that a user can determine whether implementing an AI system instead of other solutions is justified, and the extent of automation that might be appropriate in a given case (this would cover a spectrum – from fully automated decision to requiring human oversight to unsuited for automation). To assess this, the matrix considers three vectors, outlined in Table 1.
Table 1. Ethical AI matrix
