Black students more likely to be accused of using AI at school
Problem stems from even more biases in AI
The research, by Common Sense Media, found that Black teenagers are nearly twice as likely as their peers to have their schoolwork mistakenly flagged as AI-generated.
The study [pdf], based on surveys of 1,045 American students (13- to 18-year-olds) and their parents, highlights the potential for AI detection software to exacerbate existing racial disparities in education.
It found that nearly 20% of Black teenagers had their work incorrectly flagged as AI-generated, compared to just 7% of white teens and 10% of Latino teens.
The report suggests these discrepancies may be due, in part, to flaws in AI detection software. Many of the tools used by teachers to identify AI-generated content have been shown to exhibit biases, often reflecting societal prejudices.
Black students are already overrepresented in disciplinary actions, and accusations stemming from biased AI tools can further harm their academic performance.
Experts attribute these disparities to several factors, including the unreliability of AI detection software, the underdiagnosis of learning disabilities in Black students and the "tech privilege" enjoyed by white students. That is, white students are more likely to have access to both AI tools and paraphrasing software that can mask its use.
AI detection tools are notoriously unreliable, struggling to distinguish between AI-generated text and the output of other writing aids, leading to innocent students being penalised.
Black students are also more likely to be falsely accused of cheating due to the underdiagnosis of learning disabilities like dyslexia.
"AI is putting out incredibly biased content," said Amanda Lenhart, the head of research at Common Sense Media.
"Humans come in with biases and preconceived notions about students in their classroom. AI is just another place in which unfairness is being laid upon students of colour."
Teens' AI adoption soars
Among other findings of the Common Sense Media study is that the use of generative AI tools among US teenagers has soared in recent months, far outpacing parental awareness.
The study found that nearly 70% of teens have experimented with tools like ChatGPT and DALL-E. More than 40% said they did so just to beat boredom. Using translation and content creation tools simply for fun is also popular.
However, there is a huge knowledge gap, as nearly half of teens don't discuss their use of AI with their parents.
"The findings tell us that young people are quickly understanding the potential of generative AI platforms, perhaps without fully grasping the pitfalls," said Lenhart.
"This underscores the need for adults to talk with teens about AI."
The lack of communication extends to schools as well, with 83% of parents reporting that their child's school hasn't addressed AI use.
While many schools worldwide have yet to establish clear guidelines or policies regarding AI use, some teachers are actively incorporating these tools into their classrooms.
The report suggests the effectiveness of these efforts depends largely on how schools and teachers approach the subject and whether they provide students with guidance on using AI responsibly.
Schools need to establish clear policies on AI use, while parents should take the initiative to discuss responsible AI exploration with their children.