Join +750K new investors every month who copy stock picks from billionaire's portfoliosSign Up Free

Meta identifies networks pushing deceptive content likely generated by AI

Published 05/29/2024, 12:19 PM
Updated 05/29/2024, 04:25 PM
© Reuters. FILE PHOTO: Meta AI logo is seen in this illustration taken May 20, 2024. REUTERS/Dado Ruvic/Illustration/File Photo

NEW YORK (Reuters) - Meta said on Wednesday it had found "likely AI-generated" content used deceptively on its Facebook (NASDAQ:META) and Instagram platforms, including comments praising Israel's handling of the war in Gaza published below posts from global news organizations and U.S. lawmakers.

The social media company, in a quarterly security report, said the accounts posed as Jewish students, African Americans and other concerned citizens, targeting audiences in the United States and Canada. It attributed the campaign to Tel Aviv-based political marketing firm STOIC.

STOIC did not immediately respond to a request for comment on the allegations.


While Meta has found basic profile photos generated by artificial intelligence in influence operations since 2019, the report is the first to disclose the use of text-based generative AI technology since it emerged in late 2022.

Researchers have fretted that generative AI, which can quickly and cheaply produce human-like text, imagery and audio, could lead to more effective disinformation campaigns and sway elections.

In a press call, Meta security executives said they removed the Israeli campaign early and did not think novel AI technologies had impeded their ability to disrupt influence networks, which are coordinated attempts to push messages.

Executives said they had not seen such networks deploying AI-generated imagery of politicians realistic enough to be confused for authentic photos.


"There are several examples across these networks of how they use likely generative AI tooling to create content. Perhaps it gives them the ability to do that quicker or to do that with more volume. But it hasn't really impacted our ability to detect them," said Meta head of threat investigations Mike Dvilyanski.


The report highlighted six covert influence operations that Meta disrupted in the first quarter.

In addition to the STOIC network, Meta shut down an Iran-based network focused on the Israel-Hamas conflict, although it did not identify any use of generative AI in that campaign.


Meta and other tech giants have grappled with how to address potential misuse of new AI technologies, especially in elections.

Researchers have found examples of image generators from companies including OpenAI and Microsoft (NASDAQ:MSFT) producing photos with voting-related disinformation, despite those companies having policies against such content.

The companies have emphasized digital labeling systems to mark AI-generated content at the time of its creation, although the tools do not work on text and researchers have doubts about their effectiveness.

© Reuters. FILE PHOTO: Meta AI logo is seen in this illustration taken May 20, 2024. REUTERS/Dado Ruvic/Illustration/File Photo


Meta faces key tests of its defenses with elections in the European Union in early June and in the United States in November. (This story has been corrected to reflect that this is the first disclosure of text-based generative AI use, not of generative AI use altogether, in paragraph 4)

Latest comments

Risk Disclosure: Trading in financial instruments and/or cryptocurrencies involves high risks including the risk of losing some, or all, of your investment amount, and may not be suitable for all investors. Prices of cryptocurrencies are extremely volatile and may be affected by external factors such as financial, regulatory or political events. Trading on margin increases the financial risks.
Before deciding to trade in financial instrument or cryptocurrencies you should be fully informed of the risks and costs associated with trading the financial markets, carefully consider your investment objectives, level of experience, and risk appetite, and seek professional advice where needed.
Fusion Media would like to remind you that the data contained in this website is not necessarily real-time nor accurate. The data and prices on the website are not necessarily provided by any market or exchange, but may be provided by market makers, and so prices may not be accurate and may differ from the actual price at any given market, meaning prices are indicative and not appropriate for trading purposes. Fusion Media and any provider of the data contained in this website will not accept liability for any loss or damage as a result of your trading, or your reliance on the information contained within this website.
It is prohibited to use, store, reproduce, display, modify, transmit or distribute the data contained in this website without the explicit prior written permission of Fusion Media and/or the data provider. All intellectual property rights are reserved by the providers and/or the exchange providing the data contained in this website.
Fusion Media may be compensated by the advertisers that appear on the website, based on your interaction with the advertisements or advertisers.
© 2007-2024 - Fusion Media Limited. All Rights Reserved.