To audit is literally to listen — the word comes from the Latin verb audire, to hear. When auditors review tax returns or business records, they listen for dissonance. Do the numbers add up? Do the records match reality? Do claims on paper hold up on closer inspection?

As AI reshapes human affairs, a similar need has emerged: to measure the outputs and understand the impacts of socially influential AI systems including large language models, algorithmic recommendations and other automated systems.

AI audits offer a way to evaluate how these systems behave in the real world, determining whether they function as designed, whether that design is flawed and whether the systems produce harms their creators did not anticipate.

Danaé Metaxa, Raj and Neera Singh Term Assistant Professor in Computer and Information Science, and co-authors recently published “Auditing AI,” a new, open-access book that explains how AI audits can help policymakers and everyday users better understand AI systems.

In a recent conversation, Metaxa discussed what motivated them to co-write the book, what AI audits can and cannot reveal, and how their lab at Penn Engineering is working to make AI systems more accountable.

What is an AI audit? And how does it relate to more well-known types of audits, like those of tax returns or business records?

AI audits are a process for understanding how an AI system is working behind the scenes, without direct access to the internals of that system. It’s a rigorous scientific method of systematically probing those systems and measuring their outputs to make inferences about how they work and what social impact they are having.

AI audits are closely related to audit studies done by social scientists beginning in the 1960s during the civil rights movement, to evaluate whether employers or lenders were adhering to new civil rights laws. The goal is to hold an entity — an employer, a social media company, an AI tool provider — accountable for their opaque process’ outputs.

Tax audits and financial audits share the name because they, too, are invested in accountability, but differ in a key way: When a person or company is being audited financially, they are obligated to provide data to the auditor, who has a much easier job of evaluating that entity after getting all that insider information.

In contrast, AI audits are typically done from the outside in, without explicit consent and cooperation from the entity being audited. My co-authors and I hope that, in the future, government regulators might require companies to cooperate with AI auditors and provide that kind of data, too. At the moment however, the broader landscape of AI technology is extremely opaque, with firms revealing little about how their systems operate.

What motivated you and your co-authors to write this book, and why did it feel important to publish it now?

Online platforms like social media sites, search engines and LLM chatbots are used more and more each year. At the same time, companies have become less willing to work with researchers and provide open data. Bad publicity and less regulation have resulted in a bigger-than-ever need to evaluate these systems from the outside in.

A few years ago, a lawyer from a prominent human rights organization contacted me regarding a legal settlement his firm was negotiating with a large social media platform whose AI-powered content filters were allegedly discriminating against some user groups. He explained to me that the platform was telling him that it wasn’t possible to improve the bias of their tools or to track such improvement over time.

As an expert in AI auditing, I could confirm that improving these tools was possible, and describe to him the kind of evaluations that he should ask the company to provide over time to demonstrate improvement.

This book is for people like him: those who encounter opaque systems in the course of their work or life, and need to know that there is a rigorous methodology for evaluating them, even without technical details about the algorithm or the data set.

I want to empower these individuals to ask for audit reports when they evaluate, say, a proprietary AI tool for use in their own workplace, or when they have some experiences in their daily use of these tools that seem off and worth investigating further.

What do you wish more people understood about the ways AI systems can go wrong?

In 1960, American Airlines (AA) developed SABRE, a software system for travel agents to book airline reservations. AA’s competitors started noticing some concerning behavior: flights consistently missing from SABRE’s search results, passengers unbooked and rebooked on AA flights instead.

These airlines conducted one of the earliest known examples of an AI audit: They compiled data about such patterns in the SABRE system’s search results. The U.S. Civil Aeronautics Board launched an investigation in 1983. In response, the then-president of AA essentially responded, “Of course our system is biased in our favor; that’s the reason we developed it!”

Eventually, the Civil Aeronautics Board banned SABRE and systems like it from what they termed “display bias.” But anyone who’s used an online search system and seen a little dropdown in the corner indicating results are ordered by “Top Recommended” instead of “Increasing Price” or another objective measure is interacting with a system like SABRE, where the developers of the system are biasing what you see.

The key take-away is that algorithmically powered content you interact with — whether it’s an online retailer’s search results, your social media feed or the reply you get from a chatbot — can be biased in ways that are harmful to you. It’s not always intentional, like it was with SABRE. But AI audits are the way we rigorously and systematically evaluate those systems to see how they behave and how they impact users like you.

Can you describe some of the audits your lab at Penn Engineering has conducted? What systems have you studied, and what have you found?

My group has audited and is currently auditing a wide range of systems, including online targeted advertising, online retailers’ dynamic pricing, search engines as they report news and political content, social media platforms’ personalized feeds and LLM chatbots’ discussion of social issues.

One of our most recent studies, which we presented at the 2026 Association for Computing Machinery Conference on Fairness, Transparency and Accountability (ACM FAccT), conducted a large-scale evaluation of chatbots like ChatGPT and Claude for the accuracy of the information they provide on a critical health issue: abortion.

We developed a ground truth data set capturing the complex legal landscape across the U.S. and across different types of procedures and specific restrictions by things like gestational week and exceptions for things like sexual assault. We then collected thousands of LLM responses to many questions real people might ask about this topic, and evaluated the responses the LLMs gave in comparison to legal ground truth.

We found a concerning trend: a type of “double burden,” where the average response accuracy across all 50 states was around 78%, but in states with greater restrictions on abortion, accuracy was much lower (as low as 44% in North Dakota) compared to states with fewer restrictions (accuracy as high as 99% in Vermont).

The risk here is extremely high. In our work, we also used national pregnancy data by state and gestational week to do a real-world impact analysis, identifying how many pregnant people could be misled by incorrect chatbot responses. Being extremely conservative in our estimates, over 1.3 million people could be getting incorrect information from these chatbots at any given time.

What impact do you hope the book will have?

We want people who are not AI auditing experts but whose work and life brings them in contact with these types of opaque systems to know that a rigorous process and methodology for probing those systems exists.

We also want individuals to feel empowered to ask for such evaluations (e.g., as a precondition for signing a contract to use an AI service) or to hire professionals to conduct audits.

Beyond the individual, I hope wider knowledge about AI audits can lead to support from the general public and from lawmakers to require public audits when AI is used in high-stakes settings.

In 2023, New York City introduced the first law requiring audits: Local Law 144 requires that any company using AI tools in hiring publish audit reports. The law has many limitations, as our group found in a recent paper, but it’s a start, and we hope to see more such legislation.

“Auditing AI” can be read freely online.

To learn more about the Metaxa Lab, which is part of the Human-Computer Interaction group at Penn Engineering, please visit the lab online.