Getting Started with Critical AI and Data Ethics: A Reading List

Critical AI literacy and data ethics literacy are more important than ever. And I think a lot of people have picked up on that and want to read up on these topics. But it’s also challenging to know where to start. So I thought I’d do a little public service!

I’m teaching a class on critical AI and data ethics, Critical AI and Data Ethics. From Data (Feminism|Gaps|Work) to Critical (Archival|Data|Code) Studies, at Humboldt-Universität zu Berlin this fall term. For the course, I’ve put together a syllabus and reading list that I’ve now shared on GitHub so that everybody else can benefit from it as well. It’s basically the reading list I would have loved to have had when I first started getting into these topics a few years ago.

One important thing I’ve learned along the way is that AI ethics is a rather fragmented field (see the disclaimer section in the reading list). There are many different disciplinary perspectives on AI ethics, critical data studies, and related topics, and they often exist in their own disciplinary silos. There are huge AI ethics subfields in medicine, law, philosophy, and computer science, but also in what I would call the critical disciplines: the many critical turns that have emerged in relation to critical theory, such as critical archival studies, critical data studies, critical code studies, critical infrastructure studies, critical digital humanities, and, as I’ve recently argued, possibly also critical computational humanities. So where do you even begin?

Positionalities

First, I should say that this reading list is very much shaped by my own positionality. My background is in the history of science and the history of knowledge, which also connects me to science and technology studies (STS). I work in digital humanities, and the approach to AI ethics that I personally find most useful is one grounded in critical theory and historical perspectives or those most directly applicable to my work more broadly. And I think this “critical” approach is particularly applicable to humanities disciplines working historically.

But what’s important to understand about critical theory, very briefly and somewhat simplistically, is that here, critique isn’t just about identifying problems. It’s about diagnosing them with the intention of actually making things better. Critical doesn’t necessarily mean refusal. It can (and in my opinions should) also motivate action!

Why I’m Not Against AI as Such

Of course, there are people who reject AI entirely. And I should say upfront that I’m not one of them. Personally, I think it’s more important to keep lines of communication open and ensure that people with both critical and technical literacy are part of the conversation. We need a nuanced discourse that differentiates between different technologies, applications, and contexts.

Because the thing that recently bothers me most about the complete refusal of AI is that it can obscure just how many different technologies and methods are subsumed under that term. AI isn’t just generative AI. And it isn’t just the technologies currently being developed by some rather awful companies. Machine learning methods have been used in digital humanities for a long time, often in rather unspectacular ways. Many of these are small models that have very little to do with the generative AI systems dominating the current debate.

This is actually one of the key points I want students to take away from the first session of my course, which focuses on definitions of AI. One of our readings is an introduction from AI Snake Oil, which makes a great analogy: if we called every means of transportation simply a “vehicle,” we would have no way to distinguish between the environmental impacts of a bicycle, a truck, and an airplane. And that’s essentially what’s happening with AI.

There are also more personal reasons why I don’t think complete refusal is particularly helpful. For example, I often see people saying that they would never use AI in their writing. And when I look at who’s making these statements, I notice that many of them are native English speakers. And I think: well, perhaps you’ve also never been discriminated against in academic publishing because your English wasn’t quite perfect. Because I have. Since I’ve started using generative AI to help with my English, I haven’t received that kind of feedback anymore. Of course, that could be a coincidence. Maybe I’ve simply become a better writer. But I do think AI probably has something to do with it: it helps me eliminate Germanisms from my writing, including those that are so deeply embedded in the sentence structure that I find them almost impossible to edit out myself.

Of course, there are now also studies suggesting that existing power structures are being reproduced in how people judge AI-assisted writing. Non-native English speakers, particularly those from the Global South / Majority World, and women face more negative reactions when their use of AI is detected, while (white) men’s use is often viewed positively. So, once again, we’re dealing with power structures. And that’s precisely why so much of this reading list focuses on disciplines that study power.

But I don’t think the technology itself, insofar as we can separate it from the companies currently dominating it, is necessarily fundamentally flawed. Take large language models, for example. Yes, many existing models have been trained on problematic datasets and under exploitative working conditions. But technically, we could create models with better datasets and better working conditions. It’s possible. It’s just expensive and requires a lot of work. Why should we discount that possibility if the technology itself can be useful? (And my stance is definitely not just an utilitarian “everything goes as long as it’s useful to some privileged person.”)

Projects such as the Swiss open model Apertus or the BLOOM model show that there are alternative approaches. BLOOM, for example, made efforts to be more transparent about its environmental impacts, helping us better understand and, thus, potentially reduce the costs involved.

And speaking of environmental impacts, I think we also need more nuance here (read more about my own efforts in Guesstimating Environmental Impacts of LLM Workflows). A substantial part of the environmental costs of developing large language models comes from training them. Companies are constantly producing new, supposedly better models to maintain their position at the cutting edge of the market and satisfy investors. But is that really necessary? Personally, I found GPT-4o perfectly well for many of my writing tasks. I don’t need a new, more powerful model every few months. Inference, the process of actually using a trained model, is considerably less energy-intensive than training. Of course, inference at enormous scales also adds up, but this distinction still matters.

In our recent analysis of the environmental impacts of using LLMs for quantitative analysis in digital humanities, we found that our particular research application consumed relatively little energy. The resulting emissions were comparable to fairly mundane everyday activities and substantially lower than those associated with my conference travel. That doesn’t mean large-scale applications, for example in libraries, don’t require careful consideration. But it does mean that using a model in a constrained everyday research context isn’t necessarily the environmental catastrophe that some discussions make it out to be.

Ultimately, AI is a sociotechnical phenomenon. It’s embedded in structures of money, power and labour that make its current applications problematic. But I think we should distinguish between the technologies, the people and institutions developing them, and the conditions under which they’re being used. To do that, we need critical AI literacy.

So What’s Actually in the Reading List?

The list contains the articles and books I wish somebody had handed me when I first started exploring these topics.

Because it’s based on an actual course, I couldn’t include everything. I’ve selected two core readings for each of the ten thematic sessions, supplemented by optional readings for anyone who wants to explore a topic further.

The idea is to provide a manageable starting point while also offering good jumping-off points for further reading and introducing some of the important authors in the field.

Here’s a quick overview of what we’ll be covering.

1. Definitions of AI

We start with the basics: what actually is AI? We, thus, start with definitions of AI, looking at two quite different introductions from different disciplinary backgrounds that show just how complex the phenomenon actually is.

One of these is from AI Snake Oil, which starts by pointing out the terminological problems surrounding AI. As I mentioned earlier, if we called all forms of transportation simply “vehicles,” we wouldn’t be able to distinguish between the environmental impacts of a bicycle, a truck, or an airplane. But this is exactly what is happening with AI.

I’ve also included some video tutorials explaining the technical foundations of AI and machine learning. I think this is particularly important because I’ve noticed that some people who are very critical of AI don’t necessarily understand how these technologies work mechanically. Understanding the technology (at least a little bit) is an important foundation for criticising it meaningfully.

2. The Critical Turns

Next, we look at the many critical turns that have emerged across different disciplines. There are critical data studies, critical code studies, critical archival studies, critical infrastructure studies, critical digital humanities, and several others. These approaches are related to critical theory, but they also each have their own disciplinary traditions and concerns. We read texts representing two of these critical turns, while the others are included as further readings. The idea is to give an introduction to this broader landscape and show how these different approaches can help us think about AI and data.

3. Foundational Texts and Historicising AI and Data

For the third session, we read two classics: Raw Data Is an Oxymoron and Sorting Things Out: Classification and Its Consequences. Both are foundational texts for understanding data and classification, and particularly the idea that data is never simply raw or neutral.

As optional readings, I’ve included some work by historians who have written histories of data, AI, and information practices more broadly. Coming from the history of science and knowledge myself, I find this historical perspective particularly relevant.

4. Data Feminism and Gender Issues

The fourth session is about data feminism and gender issues. Here, we obviously read Data Feminism by Catherine D’Ignazio and Lauren F. Klein, which is one of the central texts in this field. I’ve also linked a number of other relevant readings on gender, feminist approaches to data, including related work from digital humanities.

5. Activism and Auditing

In the fifth session, we look at activism and auditing as examples of how critical approaches can translate into practice. This session, thus, asks what we can actually do about the problems identified by critical scholarship.

One of our examples is counting feminicide, which grew out of data feminist approaches. We also look at auditing practices and how they can be used to examine datasets and computational systems critically.

I think this is particularly interesting because it shows how identifying problems with data can lead to practical interventions that actually prove outcomes. It demonstrates well how critique can translate into practical action.

6. Focusing on Data: Critical Dataset Studies

A growing body of scholarship argues that many problems associated with AI originate in the datasets on which computational systems depend. Rather than focusing exclusively on fixing models, this approach asks us to examine the data itself: where it comes from, how it was constructed, what’s missing, and what kinds of biases and power structures it reproduces. This is an approach I personally identify with quite strongly in my own research.

7. Large Language Models and Big Scales

Of course, we also need to talk about LLMs. The seventh session, thus, focuses on large language models and the implications of working at increasingly large scales.

We read the famous Stochastic Parrots paper, alongside a less well-known paper that I particularly like for understanding how the training material of language models influences their downstream outputs. I’ve also included a suggested reading on the dark sides of dataset scaling.

8. Data Work

The eighth session is about data work and the labour behind AI and computational research. We read Ghost Work and selections from Karen Hao’s Empire of AI, which help us understand the often invisible human labour involved in building and maintaining these technologies.

I’ve also linked several related readings, including Steven Shapin’s Invisible Technician, which connects these questions to a much longer discussion in the history of science about invisible and undervalued labour. And then there’s the whole question of infrastructural invisibility, associated with Susan Leigh Star’s work. I think these connections are fascinating because they show that many of the problems we’re currently discussing in relation to AI have much longer histories. The invisibility and undervaluation of technical and data-related labour are certainly not new phenomena.

9. AI and the Environment

The ninth session focuses on AI and its environmental impacts. We read the relevant chapter from Kate Crawford’s Atlas of AI and Karen Hao’s discussion of data centres. The further readings include work on critical data centre studies, but also research that tries to quantify the carbon emissions of machine learning and especially large language models. This is another area where I think it’s important to move beyond general statements about AI being environmentally harmful and examine the actual infrastructures, practices, and scales involved.

There’s obviously much more literature on this topic than we can cover in one session. I’ve also included a paper of my own in which we quantify the environmental impacts of using LLMs for computational analysis in digital humanities, which might be interesting to anyone working with these methods in their own research.

10. The Ethics of AI Ethics and Critical Concerns

The tenth and final session is about the ethics of AI ethics itself, as well as broader critical concerns about AI.

One of our readings is a philosophical text that examines the problems with AI ethics as a field. This is the only explicitly philosophical ethics text I’ve included among the core readings.

One important point here is that there can be no single, universal ethical approach to AI. Ethics depends on values, and different people and communities have different values. Supporting one value may mean neglecting or compromising another. These are ultimately political choices. I would argue that if ethics is inevitably political, we should at least make those choices deliberately.

The second reading is Against the Uncritical Adoption of AI Technologies in Academia, a manifesto that I think provides a very good summary of concerns about the uncritical adoption of AI in academic contexts.

In the further readings, I’ve also included some of my own work on critical concerns in computational humanities, as well as perspectives on AI ethics from computer science and philosophy. These are particularly useful if you want to better understand the different approaches that have shaped developments such as explainable AI, and why some critical scholars argue that these approaches don’t necessarily address the underlying problems.

And There’s More!

Beyond the ten sessions, I’ve included links to other reading lists and resources that I found particularly helpful. These include Lisa Nakamura’s critical AI syllabus, the Knowing Machines Critical Dataset Studies reading list, the Critical Algorithm Studies reading list, and Professor Casey’s AI Ethics playlist (all links are in the github reading list). I’ve also collected some interesting (thought) experiments and other resources, including ELIZA, Clever Hans, the Chinese Room, the Turing Test, and Emily Bender’s octopus thought experiment. There’s also a web resource asking Is AI Profitable? and a fun little article about why ChatGPT is banned from saying “goblins,” which offers some interesting insights into model training and its consequences.

And finally, I’ve included a selection of books that we won’t be reading in class but that are well worth exploring. Among them are Adam Becker’s More Everything Forever, which examines the techno-salvationist ideologies of Silicon Valley, The AI Con, Unmasking AI, The Age of Surveillance Capitalism, Artificial Unintelligence, Automating Inequality, Algorithms of Oppression, and Weapons of Math Destruction.

There are plenty more, and I’ll probably keep adding things.

Only a first glance…

I should emphasise again that this isn’t meant to be a comprehensive overview of AI ethics. The field is far too broad and fragmented for that.

Philosophical ethics, computer science ethics, critical data studies, and other approaches often ask quite different questions. Computer science approaches may focus on identifying and fixing specific technical problems, often after systems have already been developed. Philosophical approaches might ask how we identify and evaluate moral actions or understand why particular decisions are made. Critical data studies, meanwhile, often directs our attention to the datasets and underlying power structures that produce these problems in the first place. These perspectives can overlap, but they don’t always agree on what the problem is, let alone how to solve it.

My reading list reflects the approach that I find most useful for humanities researchers, particularly those working with historical sources and computational methods (like me).

And because I wanted the list to be genuinely usable in my concrete teaching setting (obviously), I deliberately limited the core readings to two per session. That wasn’t always easy! But I think it’s more helpful to have a manageable selection of important texts than an enormous bibliography where you still don’t know where to begin. The optional readings are there for anyone who wants to dig deeper.

Ultimately, I hope this list helps others find their way into critical AI and data ethics without having to spend quite as much time figuring out where to start as I did.

And I’d love to hear your suggestions! If there’s a text, book, reading list, video, or other resource that you think absolutely belongs here, please let me know. I’d be very happy to add more recommendations.

You can find the syllabus and reading list here: https://github.com/sarahalang/critical-ai-and-data-ethics.

You’re welcome 😉