Describing AI integration as a race may be an acceptable turn-of-phrase at the strategy stage of policymaking, but it could lead to trouble when it comes to implementing the kinds of structural measures that AI ethics likely require. An aggressive race-like posture could even lead states to side-step certain ethical precepts entirely, if securing an advantage in a particular domain is deemed to be sufficiently urgent. (Or, at a minimum, it risks reframing the application of AI ethical principles as a necessary stepping-stone to get ahead in the AI race – rather than an end in itself.)
Running, not talking
The race for AI emphasizes a narrow, predominantly Western vision of technological progress that valorizes raw scale, and disregards other indicators that might be more accurately indicative of a state’s capacity to adopt AI in a way that truly serves the common good. In an alternative framing of the AI race, critical factors may include openness and transparency of institutions and freedom of civil society and the press (which would be key for the implementation of ‘accountable’ AI), rule of law and economic equality (which are necessary for AI projects to successfully engender fair outcomes), and educational attainment in other fields outside of science, technology, engineering and mathematics (which would bolster a state’s capacity to untangle the complex philosophical and legal challenges posed by the automation of critical functions).
And yet, it is difficult to have this type of conversation because the notion itself of an AI race stands at odds with the kind of truly inclusive discursive process that is likely to be warranted for the responsible implementation of technology that serves everyone equally. The assumption that AI development is a race leaves little breathing room for any arguments that question the race itself – or arguments that question, for example, whether winning the race in this particular way would be a universal good.
A closer look at AI indexes
When states want to see their progress in the AI race, they often turn to an AI index. Over the past several years, a number of indexes, which rank states against one another in terms of their relative ‘readiness’ or ‘capacity’ for AI development and adoption, have become a fixture in the AI governance sphere. For many states, rising in the rankings has even become a matter of policy (some of the indexes themselves actively encourage a race-like approach to AI policy). Yet there is evidence to suggest that these indexes are an imperfect scorecard of relative national progress in AI.
First, AI ‘readiness’ or ‘capacity’ are ambiguous concepts that can only be gauged by inconsistent, divergent proxy indicators. The Oxford Insights index ranks countries based on just 10 indicators that are given equal weight despite representing vastly different national characteristics. For example, whether a country has an AI strategy is weighted equally to the national rate of internet usage. Seven of these indicators are derived directly from other indexes, which further confounds evaluation.
Meanwhile, some of the individual indicators commonly used in indexes are potentially inappropriate for any kind of like-for-like comparison between states. For example, a metric of each country’s public investments in AI cannot account for the potentially very significant differences in how those investments are spent. Indexes that rank states by their number of ‘AI players’ do not distinguish between very large and prolific organizations such as a leading research university and small ventures that are much less likely to produce novel AI. Similarly, metrics relating to the number of ‘AI projects’ or ‘AI services’ (for example, public administration functions or private sector products that involve AI) do not always differentiate between types of AI. Indexes may, for example, count a large search service that uses cutting-edge natural language processing the same way they count a city council that uses software with a decades-old algorithm.
Some of the key metrics used in these rankings also appear to draw from unreliable data. The Stanford Human-centred Artificial Intelligence (HAI) Index’s ranking of ‘Relative AI Skill Penetration Rate by Geographical Area’ and more than a dozen indicators used in Tortoise Media’s Global AI Index rely on a LinkedIn dataset derived largely from information that is self-added by LinkedIn users, some of whom inevitably make distorted claims about their AI-relevant skills. Indicators of AI adoption by companies also rely on self-reporting that might be prone to inaccuracies. Stanford’s ranking of ‘AI Adoption by Organizations in the World’, is derived from a voluntary online survey of several thousand executives conducted by McKinsey & Company – a survey whose detailed methodology and raw data are not publicly released.
There is also evidence of potential regional and demographic biases in both index data and development. For example, raw data on journal citations, a key indicator in several indexes, can under-represent the academic output of institutions and practitioners from the Global South. (There is evidence to suggest that biases can be a major factor contributing to the under-representation of Global South authors, articles and journals in citation counts.) LinkedIn, a key source of data, does not have the same user rates all over the world, meaning that it may under-represent AI penetration in certain regions. Meanwhile, a disproportionate number of these data and indicators are compiled, sorted and presented by Western institutions, and non-male experts are under-represented in some indexes’ staffing and advisory bodies.
