AI Tools Struggle to Verify Authenticity of Generated Images

When Filipinos sought the truth about a viral image of former lawmaker Elizaldy Co, who is implicated in a corruption scandal, they turned to AI-powered chatbots for verification. Surprisingly, these tools failed to recognize the image as a fake, despite it being generated by the same AI technology they were using. This incident underscores growing concerns about the reliability of AI systems in debunking misinformation, especially as many tech companies are reducing human oversight in fact-checking.

The fabricated image of Co, who has been elusive since the corruption investigation began, reportedly showed him in Portugal. Online users, eager to confirm its authenticity, asked Google’s new AI model about the image. The AI incorrectly identified the image as genuine. Following an investigation by AFP, it was revealed that the image was created using Google’s AI technology.

Alon Yamin, CEO of the AI content detection platform Copyleaks, explained, “These models are trained primarily on language patterns and lack the specialized visual understanding needed to accurately identify AI-generated or manipulated imagery.” This limitation often leads chatbots to provide inconsistent assessments, raising doubts about their effectiveness in tasks such as verifying image authenticity.

The challenges do not stop with Co’s case. During recent protests in Pakistan-administered Kashmir, a fabricated image depicting protestors with flags and torches circulated widely. An analysis by AFP found that this image was also created using Google’s Gemini AI model, but both Gemini and Microsoft’s Copilot incorrectly classified it as a real photograph of the protest. According to Rossine Fallorina from the nonprofit Sigla Research Center, “AI models are programmed only to mimic well. They cannot ascertain whether the resemblance is actually distinguishable from reality.”

In March 2024, researchers at Columbia University conducted a study assessing the accuracy of seven AI chatbots, including ChatGPT and Grok, in verifying images from photojournalists. The study revealed that all seven models failed to properly identify the provenance of the images, indicating a significant gap in their capabilities.

One of the creators of the misleading image of Co, a middle-aged web developer in the Philippines, expressed his dismay at the rapid spread of his creation. He had generated the image using Nano Banana, Gemini’s AI image generator, intending to create something humorous. “I edited my post — and added ‘AI generated’ to stop the spread — because I was shocked at how many shares it got,” he told AFP, requesting anonymity to avoid backlash.

As AI-generated images proliferate on social media platforms, they often resemble real photographs closely, complicating the task of distinguishing between the two. This trend poses challenges, particularly as many users are turning to AI tools for information verification instead of traditional search engines.

Compounding the issue, Meta announced earlier this year that it would discontinue its third-party fact-checking program in the United States, shifting the responsibility of debunking misinformation to users through a system called “Community Notes.” This decision has raised concerns, especially in highly polarized societies where accusations of bias against professional fact-checkers are common.

While AI tools can assist human fact-checkers by quickly geolocating images and identifying visual clues, experts caution that they cannot replace trained professionals. Fallorina emphasized, “We can’t rely on AI tools to combat AI in the long run.”

The ongoing struggle of AI systems to accurately verify their own creations highlights the urgent need for robust solutions in the fight against misinformation. As technology evolves, the reliance on AI without adequate human oversight could lead to more significant challenges in discerning truth from fiction in the digital landscape.