Researchers have uncovered new evidence of networks of thousands of suspect Twitter bots working to influence the Brexit debate in the run-up to the EU referendum.
The findings, from researchers at City, University of London, include a network of more than 13,000 suspected bots that tweeted predominantly pro-Brexit messages before being deleted or removed from Twitter in the weeks following the vote.
The research – which is published in the peer-reviewed Social Science Computer Review journal and was shared exclusively with BuzzFeed News – suggests the suspected bot accounts were eight times more likely to tweet pro-leave than pro-remain content.
"This is research that corroborates what Facebook and others say: that there are bots that serve to falsely amplify certain messages," co-author Dan Mercea told BuzzFeed News.
"There is a potential distortion of public communications and we want to get to the bottom of that. This amplification is of concern as it gives us a false sense of momentum behind certain ideas… If there is false amplification, how do we know if someone is genuine?"
The new evidence of botnet activity in the EU referendum raises serious questions for Twitter, including whether the tech giant has any evidence as to who was behind the bots, and whether or not the site was aware of significant Brexit bot activity at the time.
Damian Collins, the Conservative chair of the House of Commons culture, media, and sport committee, told BuzzFeed News he had written to Twitter to ask for more information as to whether there had been "interference" in the UK's democratic process.
"This is clearly a very serious matter, and one that must be urgently clarified," he said. "I have written to Twitter asking them to look into the bot accounts highlighted by the researchers at City University. I have specifically asked the company about the reach of the tweets, the ownership of the accounts, and who deleted the tweets.
"Any interference in the democratic process of the United Kingdom by people acting illegitimately is a serious matter, and I hope that Twitter will be able to shed some light on what is a troubling piece of research.”
Asked by BuzzFeed News whether Twitter had been aware of botnet activity around the time of Brexit, and whether it had informed the authorities if so, a spokesman for the social network said he had "nothing further to share".
The researchers also analysed the type of content the suspected bots were producing, finding this pool of accounts were eight times more likely to tweet slogans associated with Vote Leave, and tweeted more than average accounts in the run-up to the referendum – then less afterwards, before their removal from the network entirely.
Rather than creating outright fake news, the researchers said, the accounts worked to "amplify" existing memes, hyperpartisan news, and supportive media stories.
The bots, they noted, were often not as effective as real users at spreading messages – often dampening their impact by talking to and retweeting other bots, rather than reaching real users.
The research leaves significant unanswered questions: It is impossible from analysing the public feed to identify who is behind any suspected bot account, though Twitter itself may have information that could shed light on that.
The new evidence of bot activity comes at a time of mounting concerns in the UK and US around campaigns of election interference. Last week, Collins announced he would call representatives of social media sites into parliament to answer questions on whether Russia or other actors influenced the Brexit vote or general election through fake news or similar means.
Collins has separately written to Twitter's UK managing director, Dara Nasr, with specific questions relating to the new City University research, asking for information on the accounts identified by the researchers, their control, and their reach.
Separately on Thursday, Labour MP and former culture secretary Ben Bradshaw called for a parliamentary inquiry into whether Russia had influenced either 2016's Brexit vote or 2017's general election, either through online efforts or "dark money".
BuzzFeed News contacted a former senior staff member of Vote Leave, who said the official campaign had no knowledge of bot activity during the referendum and had made no effort to use bots as part of its strategy.
The research findings also highlight similarities in the behaviour of suspected bots influencing Brexit with some of those noticed by US researchers and authorities in the run-up to Donald Trump's election in 2016. These include amplifying messages from similar accounts connected to the alt-right, and even the pattern of deletion and removals.
The question of who removed the 13,000 accounts in the weeks immediately following the EU referendum is a key one, and mirrors US inquiries. In the States, investigators have expressed concerns that activity from suspicious accounts in the run-up to Trump's election was deleted after election day – raising the possibility of sophisticated botnet operators deactivating accounts to help cover their tracks.
A second possibility is that Twitter discovered the potential botnet in the wake of the EU referendum vote, and deleted the network. "If that’s the case they have known for at least one year about all of this," said Bastos. "What’s more, they would be in the position to tell precisely how large it was and how far afield their messages have cascaded."
BuzzFeed News asked Twitter – which was not contacted by the researchers before they published their findings – whether it had any knowledge of Brexit botnet activity in summer 2016 , and if so whether it had notified any authorities of this.
Twitter declined to comment on that specific question, but instead offered a statement.
"As patterns of malicious activity evolve, we’re adapting to meet them head-on. On average, our automated systems catch more than 3.2 million suspicious accounts globally per week — more than double the amount we detected this time last year," a spokesman said.
"Since June 2017, we’ve suspended more than 117,000 malicious applications for abusing our API, collectively responsible for more than 1.5 billion low-quality Tweets this year.
"Studies of the impact of bots and automation on Twitter necessarily and systematically under-represent our enforcement actions because these defensive actions are not visible via our API, and because they take place shortly after content is created and delivered via our streaming API."
Identifying Twitter bots is a complex matter that can divide researchers, as the people operating the networks often change tactics to mimic human users and avoid detection.
Where some previous studies have used relatively simple methodologies, like identifying any account that tweets more than a certain number of times a day to pick out probable bots, the new research is more sophisticated. The researchers collected more than 10 million tweets from 10 June 2016 to 10 July 2016 that related to major EU referendum hashtags and accounts, and analysed these for unusual activity.
Dan Mercea said they had used a range of measures to identify probable bots, including the age of the account, the ratio of how often accounts retweeted and mentioned other accounts versus how often they were mentioned in return, and the pattern of tweeting from different accounts.
The researchers then focused in on a core 13,493 accounts that had been deleted from Twitter in the wake of the EU referendum.
This methodology, while sophisticated, is still not a perfect identifier of bots – it did, for example, identify the account @nero, which belonged to the alt-right controversialist Milo Yiannopoulos, as a suspected bot.
Marco Bastos, the paper's lead author, said the @nero account triggered his algorithm because it was "massively retweeted by a range of bots and this account has singlehandedly – well, in addition to the bots themselves – pushed some highly viral messages".
He acknowledged any system would have "false positives", but said his paper was much more cautious about flagging bots than previous research efforts – suggesting a higher likelihood there was substantial bot activity during the referendum.
"The bottom line is that there is no perfect method for identifying bots," Bastos said. "Incidentally I’m afraid our approach ... erred to the side of conservatism. In other words, the botnet is likely to be bigger than we identified."
James Ball is a special correspondent for BuzzFeed News and is based in London. PGP: here
Contact James Ball at James.Ball@buzzfeed.com.
Got a confidential tip? Submit it here.