Since the UK decided to leave the EU on 23 June, politicians have leapt to define exactly what the country voted for, whether that’s a hard, soft, grey, or even a red, white, and blue Brexit.
Several leading Leave supporters have in the months after the referendum played down the issue of immigration, instead claiming Brexit was much more about sovereignty or other issues – with Conservative MEP Daniel Hannan saying, "If you're told that 'Brexit was all about immigration,' you can be almost certain that you're talking to a Remain voter." Welsh Conservative leader Andrew Davies accused "champagne socialists" of using immigration as an "easy excuse" to explain the Brexit vote.
But a groundbreaking analysis by researchers at the University of Sheffield, of almost 3 million tweets over a six-month period, shows that among tens of thousands of Leave's most vociferous online supporters, immigration vastly outweighed issues such as sovereignty and the NHS.
The research, shared exclusively with BuzzFeed News, also shows law and justice – making British parliamentarians and judges responsible for UK law – was one of the dominant themes for Leave campaigners online. Leave supporters and their newspapers have in recent weeks run a frenetic campaign protesting against a High Court ruling that parliament must have a say before the UK can trigger Article 50 and begin formally leaving the EU.
The researchers identified 41,443 Leave supporters and 41,445 Remain backers, based on their pattern of using campaign hashtags in the run-up to referendum day. They then analysed the issues those users tweeted about in relation to Brexit on every day from June to November (detailed methodology is available at the foot of this post).
The analysis of the Twitter conversation about Brexit showed:
- In the run-up to referendum day, Leave voters sent twice as many tweets about borders and immigration as they did about sovereignty, employment, the justice system, or the NHS.
- Leave supporters sent thousands more tweets about the healthcare system than Remain supporters did before referendum day – the data suggests this was prompted by claims Brexit would bring £350 million a week extra for the NHS. But in the months following the vote this trend flipped, with Remain backers talking far more about the NHS.
- Despite being overshadowed by immigration as a concern, law and justice – was still a huge issue for Leave supporters, who sent four times as many tweets on this topic as did Remainers.
- The researchers collected just 753 tweets in total from Leave and Remain supporters mentioning Article 50 – the legal process to leave the EU – in the run-up to referendum day. Yet on 3 November alone – the day the High Court ruled parliament should have a say on its triggering – there were more than 50,000.
- Brexit on Twitter was "an echo chamber, not a debate" according to one of the researchers – only 7% of tweets were replies to other users.
- Leave supporters engaged more in debate than Remain fans, using more replies and retweets, which one researcher said showed "more effort to spread ... their views".
BuzzFeed News has taken five of the most actively discussed topics during the month of June and mapped them to show how the day-by-day conversation ran. The number of tweets from Leave supporters on each topic is shown above the line, while Remain users are mirrored below it. You can click any topic to include or exclude it, for easy comparison.
Twitter is often referred to as a liberal echo chamber, but in some ways the month running up to Brexit countered this image: Though the research sample contained almost exactly the same number of Leave and Remain supporters, it was the Leave supporters who were tweeting most actively in the run-up to Brexit – with immigration and the economy competing as their most active topics.
After losing the referendum, the Remain supporters tweeted more actively across all topics, closing the gaps with Leave backers.
The Twitter debate mirrored the real-world interest in voting, with a small spike of activity around 7 June – one of the first deadlines to register to vote – followed by a much larger surge in conversation in the final days before the referendum.
The gap between the number of tweets from Leave and Remain supporters about borders and immigration becomes most apparent in the final week of the campaign, with a huge surge in immigration tweets from Leave backers coinciding with Nigel Farage's launch of his infamous "BREAKING POINT" poster showing a queue of refugees. The poster gave the impression they were heading to Britain, despite the picture being from the Croatian border with Slovenia.
Twitter alone cannot explain why the referendum result happened, or what fuelled it. But combined with other evidence such as exit polls, and research from NatCen showing economically deprived, anti-immigration voters were crucial for Leave's victory, it builds a picture of what tens of thousands of enthusiastic supporters, rather than just political figureheads, discussed in the months before and after the vote.
"The key challenge and benefit of this work is in getting the insights in real time," said Kalina Bontcheva, a senior researcher on the project. This means "always having up-to-the-minute knowledge of the topics, participants, and opinions and being able to study changes in these over time," she said.
"Opinion polls, like those performed by YouGov and similar organisations, rely on much smaller samples and are generally a frozen snapshot at a particular date. Analysing tweets in real time helps us also study immediate reactions to a particular event, very soon after it's happened."
Bontcheva added the research had shown how Leave and Remain supporters used Twitter differently.
"With only around 7% of all tweets being replies and over 58% [being] retweets, this resembles more an echo chamber, rather than a debate," she said. "It was interesting to see also that Leave supporters engaged significantly more in the debate through replies and retweets, i.e. made more effort to spread further their views."
Immigration dwarfed every issue except the economy on Twitter, especially for Leave voters. The sample collected by researchers collected more than 66,000 tweets in this category from Leave supporters – 26,000 more than Remain.
Critically, Leave supporters' tweets on immigration peaked in the final days of the referendum campaign, with 20 and 21 June seeing a sharp increase in activity. Opinion poll after opinion poll showed immigration was the issue that resonated most with Leave supporters – as the Remain team were well aware.
No other issue came close for Leave: They sent twice as many tweets on immigration as on "democracy", the researchers' category for sovereignty, and almost three times as many on immigration as on employment, justice, or the health system.
Remain supporters tweeted about immigration more than their Leave counterparts on only 15 of the 146 days covered by the research – none of these were before referendum day. In the weeks before the vote, Leave supporters sent roughly the same number of tweets about immigration as the economy – while Remain supporters sent far more on the economy.
The days immediately after the referendum saw a high volume of immigration tweets from both sides, which appear to have been fuelled by supporters in both camps arguing about whether or not the Leave vote was about immigration. The issue spiked once again in party conference season, as Theresa May assured voters Brexit would mark an end to free movement of people.
"Let's give our NHS the £350 million the EU takes this week."
This Vote Leave proposal – and variations of it printed on the side of a bus – proved one of the most contentious of the campaign, provoking protracted rows over how much money the UK pays to the EU, and how much of this might be available to spend on the NHS if Brexit did occur.
The resultant conversation on social media is a tale of two time periods. Before referendum day, Leave voters dominated the conversation on healthcare, often centered around that claim.
In the run-up to election day, Leave supporters in the research sample sent 2,000 more tweets relating to the NHS than the Remain sample. But this changed abruptly within days of the result, as leading Leave campaigners backtracked on or softened the campaign claim – leaving Remain supporters to ask for the money or castigate the government for refusing to deliver on it.
This shift shows up sharply in the figures: In the months after the vote, the Remain sample tweeted about healthcare more than 22,000 times. Leave voters, by contrast, raised the topic in just 12,073 tweets.
The UK's courts have come under fierce criticism from Leave supporters since the High court ruled that MPs must have a say before the UK can trigger the process to leave the EU – known as Article 50. There's now an appeal against the decision.
In fact, making British judges responsible for UK laws was one of the big campaign themes in Leave's online conversation.
As the charts show, Leave supporters absolutely dominated the pre-referendum discussion on EU and UK laws, sending four times as many tweets on the topic as did the Remain backers. Unlike in areas such as healthcare, this pattern continued (albeit with a less dramatic divide) after the referendum, with Leavers still talking about justice around twice as much as Remainers.
Article 50 is the only formal legal route to leave the European Union, and is a complex process. One of the starkest findings of the Twitter research is that no one was talking about it until after the decision to leave was made.
In the three frenetic weeks before the referendum, the sample of Leave and Remain voters – more than 80,000 active Twitter users – sent just 753 tweets referencing Article 50. The entire sample of 3 million tweets, which includes users not identified as supporters of either camp, identified fewer than 1,500 tweets total on the subject in this time.
Article 50 was the tool to make Brexit happen, yet almost no one spoke about it on Twitter before the vote.
The same could not be said for the months after Brexit, when Article 50 slowly rose to prominence and then came to dominate over other topics, following the High Court ruling that a parliamentary vote would be required to trigger the process.
Through August onwards, Article 50 mustered at least 750 tweets a day from both Remain and Leave supporters. By 3 November, this peaked at a massive 50,000 tweets a day – suggesting one thing everyone wanted to discuss after deciding to leave was the one thing they never mentioned in the run-up to the choice.
This is the detailed methodology of how the research team behind the data identified tweeters, topics, and more during the process. All interpretation of the data in the above article is that of BuzzFeed News.
The team: Dominic Rout, Ian Roberts, Diana Maynard, Mark Greenwood, Kalina Bontcheva
Leaver/Remainer identification method
We first analysed manually a sample of random tweets with hashtags supportive of Leave/Remain (e.g. #VoteToLeave, #SaferIn). We found that stance cannot be classified reliably simply on the basis of a tweet mentioning a Leave/Remain hashtag, since in many cases Leave hashtags are used as a reference to the Leave campaign, while the tweet itself is supportive of Remain or neutral. The converse is also true, i.e. Remain hashtags are used to refer to the Remain stance/campaign.
Unlike other analysts, we therefore adopted a more reliable, even if somewhat more restrictive, approach. First, for tweets containing Leave/Remain hashtags we considered only the last hashtag as the most indicative of its intended stance (pro-Leave or pro-Remain). Next, we filtered out all users with fewer than three stance-expressing tweets, in order to arrive at a more reliable sample of Twitter Leave/Remain supporters.
We then used tweets from these users to analyse the topics they discussed within.
Topic identification method
Topic analysis of tweets is based on reusing the methods from the Political Futures Tracker project – funded and led by Nesta (team led by George Windsor) with the GATE team at the University of Sheffield (led by Kalina Bontcheva). The PFT project focused on real-time analysis of tweets in the run-up to the May 2015 General Election.
In brief, for topics we adopted the UK government’s high-level “policy areas” on https://www.gov.uk/government/topics, such as public health, immigration, and law. For each of these, we then associated sets of keywords (e.g. for public health: NHS, nursing, doctors), which were discovered automatically from the party manifestos and UK election tweets, then post-edited and extended manually.
Given the high Brexit data volumes and the need for reliable, real-time analysis, we chose to opt for this high-accuracy, efficient method.
This was also complemented by a more traditional search-based approach, to capture statistics on popular Brexit-specific topics (e.g. Article 50) which do not exist in the PFT’s topic taxonomy.
Large-scale, real-time analysis with GATE
The data analysis was carried out both in real time on the tweet stream (enabled us to live-blog during the referendum and analyse 2 million tweets on referendum day alone) and in parallelised data batches post-hoc, as we chose to run new types of analysis on our 64 million-tweet dataset (spanning May to November 2016 and still growing).
We used GATE Cloud for this, which supports continuous Twitter data collection; real-time and batch analysis for words, names, topics, and sentiment over time; result aggregation, visualisation, and interactive search over the analysed tweets. We have also made the Brexit Analysis pipeline publicly available as a service on GATE Cloud.
For a chosen time period and optionally tweet type (original, reply, retweet), GATE Cloud enabled us to analyse the:
top X most frequently mentioned words, nouns, verbs, or noun phrases
top X most frequent posters/frequently mentioned tweeters
top X most frequent locations, organisations, or persons within those tweets
top X topics according to our topic classifier
geographic spread (via NUTS 2 regions) of tweets, topics, user mentions, etc.
frequent URLs, language of the tweets, and sentiment
James Ball is a special correspondent for BuzzFeed News and is based in London. PGP: here
Contact James Ball at James.Ball@buzzfeed.com.
Chris Applegate is an editorial developer for BuzzFeed and is based in London.
Contact Chris Applegate at firstname.lastname@example.org.
Got a confidential tip? Submit it here.