Nicki James Shepherd on why AI in education keeps getting disability wrong

Nicki James Shepherd on why AI in education keeps getting disability wrong

Some Universities are rolling out AI faster than anyone can properly think about it. Adaptive learning platforms, automated feedback, chatbot tutors, dashboards that flag “at-risk” students. Most of it arrived in a rush after late 2022, and most of it was built around an assumption nobody wrote down: that the learner on the other side of the screen is nondisabled and neurotypical.

That assumption is what my recent paper is about. It’s called “Disability, Data, and Design: Toward a Critical Framework for Evaluating AI in Inclusive Education,” published in IRE Journals in June 2025, and this post is the plain-English version.

Two fields that should be talking, and aren’t

Here’s the odd thing I kept running into while researching this. The people who build and study AI in education (the field is called AIED) barely mention disability, and when they do, it’s usually as an accessibility checkbox. Can a screen reader parse the interface? Are the videos captioned? Fine questions, but they never touch the deeper one: what does the system assume a “normal” learner looks like in the first place?

Meanwhile, critical disability scholars have spent years pulling apart how algorithms harm disabled people in welfare systems, hiring, and criminal justice. Sharp, important work. But almost none of it has been aimed at education, where adaptive platforms are already making decisions about millions of students every day.

So we have two fields, both worried about the same kind of injustice, sitting in separate rooms. My paper is an attempt to build a door between them.

The framework, in ordinary words

I call it the Critical Disability Framework for AI in Education, or CDF-AIED. It’s not a compliance checklist. It’s a set of four questions you can ask about any AI system in a classroom or lecture hall.

Who is the “normal” learner? Every adaptive system has a model of how learning is supposed to go. That model comes from training data, and the data comes from real students, most of whom were neurotypical. So when a dyslexic or autistic student’s pattern doesn’t match, the system reads it as a deficit. Not because the student is failing, but because the yardstick was never built for them.

Who actually benefits from the adaptation? An AI tutor can raise average outcomes while quietly widening the gap between the students it understands and the students it doesn’t. Averages hide this. You only see it if you break the results down by disability, and almost nobody does.

What are the politics of disclosure and surveillance? To get personalised support, a student either declares a disability (with all the stigma and paperwork that carries) or the system silently infers one from their behaviour. That second option should worry you more than it probably does. A platform that builds a profile suggesting “attention difficulties” has effectively diagnosed a student without consent, and that shadow label doesn’t come with any of the legal protections a real disclosure would.

Who holds the design power? If a system is designed, tested, and evaluated entirely by neurotypical people, neurotypical assumptions get baked in at every layer. Disabled people have deep practical expertise in navigating systems that weren’t built for them. Ignoring that expertise isn’t just unfair, it produces worse tools.

What this looks like in practice

Take the “at-risk” dashboards that are now common in UK universities. They flag students based on log-in frequency, forum posts, and submission timing. A disabled student managing chronic fatigue or a medication schedule may study in patterns the system has never seen. The dashboard calls that disengagement. Then an intervention designed for unmotivated students lands on someone whose actual problem is a structural barrier, and adds pressure instead of removing it.

Or take the new wave of AI tutors built on large language models. The people who rated the model’s answers during training were mostly neurotypical, which means the model’s whole idea of a “helpful explanation” carries their preferences. A student with ADHD or autism may find those explanations consistently miss how they think, and there’s no setting to fix that, because the bias lives in the training, not the interface.

Where I land on this

I wrote this paper as a disabled practitioner-researcher, so I’ll admit I’m not neutral. I don’t think AI in education is doomed, and I’m not arguing for a ban. Some of these tools genuinely help people, including me. What I’m arguing is that we’re pouring assumptions about normal minds and normal bodies into concrete, at scale, and concrete is very hard to break up later.

The framework is a starting point, not a finished answer. It needs testing against real systems, and it needs disabled students involved in that testing as co-designers rather than subjects. Their knowledge of how they learn is, as I put it in the paper, the most important data set AIED has never collected.

If you build, buy, or research educational AI, the four questions above are yours to use. Ask them before the next procurement decision, not after.

The full paper is open access at IRE Journals, Volume 8, Issue 12 (June 2025), DOI: 10.64388/IREV8I12-1719101.

Latest Blog Post

Nicki James Shepherd on why AI in education keeps getting disability wrong


Scroll to Top