Twitter users can now fact-check AI images, but it may worsen misinfo & brand safety

The boom of AI – in combination with new paid verification schemes at Twitter and Meta, which can make parody accounts look like the real deal – has compounded the spread of misinformation online.

To help combat the problem, Twitter announced today that it’s testing its Community Notes feature – a tool through which users can add relevant context about a tweet, often debunking myths and explaining away misinformation – for posts with images. In essence, it plans to encourage the masses to help fact-check images.

The company said that only Community Notes contributors who have an impact score – a metric that Twitter assigns to users depending on the helpfulness of their notes – of 10 will be able to add notes to tweets containing images. Twitter also said that if users post the same image with their own captions, the same Community Notes will still appear on the image.

After months of lax content moderation under Elon Musk’s leadership – since acquiring Twitter last fall, the billionaire has laid off most of the staff responsible for moderating posts – some experts see this as a promising development. “The company’s decision to begin testing Community Notes for images is a small but meaningful step in the right direction. It’s a signal that Musk knows his previous stance on moderation is wrong,” says Andrew Graham, founder and head of strategy at Bread & Law, a New York-based PR firm.

“I’ve never really considered Twitter a visual medium, but there have been a few well-publicized instances of fake images going viral and I’m glad the company is trying to do something, as opposed to nothing, on the image moderation front to try to apply some guardrails there,” he says. In combination with the appointment of a new CEO – ad industry veteran Linda Yaccarino – the move makes Graham feel “slightly more optimistic of the platform’s long-term viability.”

Bound to backfire?

Content moderation has been a core concern for users as well as advertisers – the latter of whom generate most of Twitter’s revenue – under Musk’s reign. Spikes in bot activity and hate speech on the platform put major spenders on alert over potential brand safety issues. Now, some 50% of the company’s top 1000 advertisers – including Unilever, Coca-Cola, United Airlines, General Mills, Volkswagen and others – have stopped spending on the platform, per recent Pathmatics data.

A natural instinct might suggest that growing investment in crowdsourced content moderation like the Community Notes tool would quell advertisers’ brand safety concerns. But it’s not that simple, experts say.

Despite acknowledging that the addition of Community Notes to posts containing images is a step in the right direction, Graham isn’t confident that it’s a fully effective fix – or that it will soothe advertisers’ worries. “Community Notes as a feature is a half-measure at best to quell disinformation,” he says. “I don’t think this move alone makes Twitter brand-safe.”

In fact, he points out, Community Notes could actually do just the opposite of what it intends to do – and accelerate the spread of misinformation in some cases. “Crowdsourced content moderation can end up giving more power and control to those who spread disinformation if they learn how to game the system, which is one reason why not all users, and not all use, on a platform is equal,” Graham says. “I can imagine, for instance, a troll army of now-verified users colluding with one another to add notes to an AI-generated image purporting it to be legitimate. This could accelerate disinformation rather than suppress it – and this threat applies equally to text-based content and visual content.”

And Graham is not the only one sounding the alarm. In fact, some media experts don’t see an upside at all. “While there have been arguments that Twitter's previous content moderation was one-sided, there was at least an attempt to moderate content and keep harmful or dangerously misinformative – especially during the coronavirus breakout – content either off the platform or at least with an indication that the content might not be factual,” says Neal Schaffer, a social media marketing consultant and chief executive of social media agency PDCA Social. Now, he says, “putting this responsibility in the hands of its users will only result in a more partial result than we had before. The loudest voice will win.”

In a world where generative AI is advancing at light speed and anyone can pay for a blue checkmark, this decision, he says, will “take fake news to a new level both textually and visually.”

What’s more, Musk’s track record of both dismissing misinformation on Twitter as harmless trolling and boosting posts that mirror his political philosophies suggest to many experts that a focus on community-based moderation won’t be free of Musk’s biases.

As Andrew LaFond, vice-president and executive director of media and connections at ad agency R/GA puts it: “Community Notes will depend on who Twitter approves as contributors and what exactly they mean [when the company has said in its help center that it wants] enough contributors with different points of view weighing in. It's easy to see how Twitter might manipulate who can contribute in order to conform to a preferred point of view. Depending on how Twitter manages the process, Community Notes could fall prey to the [same] trolling tactics as open, public comments.”

As far as advertising is concerned, LaFond says, “it seems clear that Musk wants Twitter to have a much stronger rightward tilt.” And a new focus on community-based content moderation “won’t change that” in his estimation. “Some advertisers will be ok with it, but many will continue to be cautious.”

And ultimately, the expansion of Community Notes on Twitter may only introduce more questions about moderation. “It still boils down to moderation: How is Musk going to moderate the Community Notes feature on images – particularly fake ones designed to disinform? He was pretty direct about continuing to make product decisions despite stepping down as CEO, and this is very much a product question. If I were a big ad buyer, I wouldn’t just take his word for it,” says Graham.

‘Twitter has to realize it’s competing for ad dollars’

Experts’ assessments indicate this could be another nail in the coffin for the platform’s advertising business.

Since seeing top brands axe spend, Musk has tried out a variety of tactics – he’s attempted to woo them back (taking on attitudes ranging from friendly to aggressive – even threatening at one point to “name and shame” brands that had cut spending). In other moods, he’s implied that he doesn’t really need advertisers at all, blazing ahead with his plans to monetize Twitter via paid subscriptions (and even subjecting advertisers to new payment requirements).

It’s been a chaotic series of events that many experts believe has put Twitter on its back foot. And the back-and-forth can’t last forever, says Schaffer. “Twitter has to realize that they are in competition for advertising dollars with TikTok, YouTube, Instagram, Facebook and even LinkedIn. [From] that perspective, I find this latest announcement about Community Notes only veering even further from the direction that Twitter should be going should they wish to become more financially viable sooner.”

For more, sign up for The Drum’s daily US newsletter here.