Artificial Intelligence Can Unknowingly Support Eating Disorders – Experts Create a Risk Map

Generative AI systems – such as conversational AI or image-producing models – can pose a serious risk to individuals with an eating disorder or those predisposed to one. A new study published on the ArXiv service shows that current filters and safety mechanisms often overlook subtle but clinically significant cues.

The study involved semi-structured, flexibly guided interviews with 15 experts in eating disorders: clinicians, researchers, and representatives of patient organizations. The data was analyzed using an abductive qualitative analysis, which involves moving back and forth between observations and theory to construct the best explanation for observed phenomena.

With the help of experts, researchers compiled seven main categories of risks that generative AI can pose. Firstly, AI can provide general health advice that sounds harmless but may reinforce unhealthy dieting ideals. It can also directly encourage disordered eating or other symptoms.

Other identified risks include supporting the concealment of symptoms, for example by advising on how to hide an eating disorder from loved ones or healthcare personnel, as well as producing so-called “thinspiration” content that idolizes extreme thinness. AI can also reinforce negative self-images, direct attention compulsively to the body and food, and perpetuate narrow and stereotypical perceptions of what people with eating disorders are like.

The study emphasizes that risks related to eating disorders do not always appear as blatant violations but can be hidden in seemingly mundane conversations. According to experts, recognizing these subtle but clinically important signals is essential if AI systems are to be modified to be safer for vulnerable users.

Source: From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorders, ArXiv (AI).