Two of Google’s major tools to fight misinformation on YouTube only work sometimes, and only in certain European languages, a new study has found.
When users watch a YouTube video on a topic that is often embroiled in misinformation, such as COVID-19 vaccines or climate change, a label should pop up to direct them to third-party sources with more information on the issue, for example Wikipedia.
The information panel is also meant to appear in users’ search results in their operating language.
However AI Forensics, a Europe-based nonprofit that investigates major tech companies, found that this is often not the case.
The researchers built a web crawler to test the panels, and found that across the 83 languages on the video platform, the information panels are applied inconsistently to 12 topics on Wikipedia’s list of conspiracy theories as well as four types of publisher labels.
All of the panels are available in English. In German, videos on all but one topic – the Armenian genocide – have these labels. Meanwhile, very few panels are available in Icelandic or Lithuanian.
AI Forensics said the tests show YouTube pays “disproportionate attention towards Western languages” and “neglects” regional languages like Basque, Catalan, and Galician.
YouTube’s content disclaimers go beyond misinformation-prone topics. It also includes labels for news sources and fact-checkers that explain whether they are partially or fully funded by government sources, as a way to “help you better understand the sources of news content,” the company says.
But public funding was flagged in some countries and not others, the AI Forensics report found.
Euronews, for example, receives some funding from the European Union. YouTube flagged this in some countries, but not in Denmark, Greece, Norway, Iceland, Finland, or Portugal.
“We are scared … there might be some group of people that have differential access to safety” measures to fight misinformation on the platform, Salvatore Romano, head of research at AI Forensics, told Euronews Next.
“Instead of increasing [the] trust of users on the platform, it is undermining [it],” Romano added.
Unclear whether YouTube will address discrepancies
YouTube’s website notes that “information panels may not be available in all countries/regions and languages,” and that the company is working to expand their reach.
The AI Forensics team said they have met with YouTube to present their research. During those meetings, YouTube allegedly indicated that the discrepancies between languages were not supposed to happen, and that it didn’t have a way to systematically check how the panels are deployed across Europe.
AI Forensics said YouTube agreed to address these discrepancies but did not go into detail.
YouTube did not respond to a Euronews Next request for clarity on the alleged meetings or the measures it plans to take.
YouTube, under parent company Google, is a signatory to the European Commission’s Code of Practice on Disinformation, a voluntary agreement that sets standards for tech companies to fight disinformation.
In March, YouTube said that as part of its commitment, it will “assess and update the topics prone to misinformation that receive additional context from information panels”.
But Romano believes the haphazard content labeling could be even more extensive than the AI Forensics researchers found, given the report says it shows only a “partial view” with a small sample of panels.
Commission could put ‘pressure on YouTube’ over inconsistencies: AI Forensics
Romano said he’s hoping YouTube will correct the discrepancies. If it doesn’t, he said he will call on the European Commission, the EU’s executive body, to investigate a potential breach of the Digital Services Act (DSA).
The Commission could, for example, request more information from YouTube about how it administers its panels, which is the first step of any DSA investigation, he said.
“The EU Commission could put pressure on YouTube to show them that this type of inconsistency is unacceptable across platforms and across products,” he said, noting that it creates “second-rate markets” throughout Europe where standards meant to protect against misinformation are not being met.
Euronews Next reached out to the Commission to confirm whether it would be asking for more information from YouTube but did not receive an immediate reply.
Romano, at least, will be watching. He said his team built a tool to track how regularly YouTube’s information labels come up in various languages.
Through the publicly available tracker, people can see which topics have content labels in their language. It will also allow researchers to identify any changes to the information panels, Romano said.