Health marketers have been monitoring social media for well over a decade to understand what the web is saying about a product. 

More recently, researchers have gone beyond analyzing mere consumer sentiment, running online text through AI to predict the presence of mental disorders, like depression or anxiety.

However, as a recent study suggests, there are limits to drawing conclusions based on the way people express themselves online. 

The study, published in the April 2 issue of Proceedings of the National Academy of Sciences, found that an AI-based computer model designed to detect the presence of depression by analyzing social posts didn’t appear to work when applied to text written by Black people.

Findings underscore the caveats that come with trying to generalize across cultures based on language, explained one of the study’s senior authors, Sharath Chandra Guntuku. He is a researcher in the Center for Insights to Outcomes at Penn Medicine and an assistant professor (research) of Computer and Information Science in Penn Engineering. 

“We need to have the understanding that, when thinking about mental health and devising interventions for treatment, we should account for the differences among racial groups and how they may talk about depression,” stated Guntuku. “We cannot put everyone in the same bucket.” 

Study’s findings, explained

Past research had suggested the possibility of mining social data to gauge mental disorders. Use of first-person pronouns in posts, such as the first-person “I” along with certain terms and expressions, were found to be predictive of depression among social media users. 

What hadn’t been known was whether and to what extent such techniques are influenced by race. To test this, Guntuku and fellow researchers analyzed 868 Facebook posts from people ages 18 to 72 with and without depression. 

Three-quarters of the posts were by women and the sample included equal numbers of Black and white individuals.

However, when the types of words identified in the past as predictive for depression were plugged into the AI-guided model, it was more than three times less predictive for depression when applied to Black people who use Facebook, researchers found. 

The algorithm’s performance remained dismal even when trained on language used by Black people in their posts.

“We were surprised,” noted Guntuku.

Why did the analysis fail?

As to various reasons behind the findings, it’s possible that Black individuals’ depression patterns require more data to learn versus those of white individuals, one researcher suggested. Or, perhaps Black individuals do not exhibit markers of depression on social media platforms due to perceived stigma.

The differences may also relate to false assumptions in the language-based method designed to pick up depression. For instance, Black people tended to use “I” more overall in their posts, even those without depression. 

Additionally, certain expressions of self-loathing that were markers of depression with white people with depression, or sentiments about being misunderstood or indications of despair or feeling like an outsider, were not specifically tied to depression in Black people.

The report, funded by the National Institutes of Health, should undermine any assumptions that words’ predictive qualities apply across cultures. 

That said, the findings don’t necessarily mean that AI-guided models can’t be used to find depression by poring over the language in peoples’ social posts. Still, the technique won’t work unless the AI is fine-tuned for various populations.

Better predictive models can be established by upping representation of Black people and other races and ethnicities in studies to better account for racial differences in the way depression is expressed across different groups of people, the authors added. 

On the other hand, they also raised the possibility that depression might not show up in language for Black individuals at all, which could be a red flag for AI-based depression prediction.

It’s entirely possible that the “psychological processes thought to predict or maintain depression may be less relevant, or even irrelevant” in these populations, they cautioned.