Fact-checking org spreads disinformation with overblown claims of botnets
On January 24th, Logically.AI published the following article:
A Logically investigation has uncovered evidence of a Twitter botnet promoting the hashtag #KeepPrisonsSingleSex.
This was picked up by the New Statesman the following day:
On February 2nd, the Keep Prisons Single Sex twitter account noted that Private Eye was also taking this story at face value:
When this happened, the #KeepPrisonsSingleSex hashtag started trending, along with Private Eye and #NotABot, as affronted campaigners mocked the idea that they were “bots”.
Examining Logically’s claims
Beyond the headlines, logically makes the following claims in their report:
Fox was right to observe that the hashtag had “created a huge amount of interest outside of [Parliament].” However, the Twitter conversation she cited shows clear signs of coordinated inauthentic behavior — the 37k mentions were not 37k people, but instead, at least in part, a swarm of bots. One of the tweets that Baroness Fox read out to the House of Lords was artificially boosted by accounts Logically identified as being connected to the botnet.
Logically found “clear signs”, and that a “swarm of bots” resulted in crucial tweets being “artificially boosted”.
In the New Statesman article, this becomes:
It reveals that support for the #KeepPrisonsSingleSex campaign was dramatically inflated and much smaller than numbers initially conveyed, with the hashtag having been primarily driven by a large network of bots. After tracking thousands of tweets over several days, Logically found that the hashtag was part of a “social media manipulation campaign”, whereby a group of anonymous accounts amplified the hashtag, constantly tweeting it themselves and retweeting each other, before disappearing after a few days. The tweets read out by Fox in the House of Lords might have been genuine, but many had been amplified by the very bot accounts Logically identified.
The support was “dramatically inflated” and “primarily driven” by a “large network of bots”. The article goes on to raise the spectre of “religious conservatives on the hard-right wing” in the US.
In Private Eye, this story became “coordinated activity by a network of bots” and warned policy makers to “pay attention to how they are being manipulated”.
The New Statesman piece even warned, not just of the botnet, but “misleading information associated with it”.
All sounds quite sinister and manipulative so far. A large network of bots manipulating our democracy, and hints of a hard-right agenda in play.
Except, at the end of the Logically report, we find this extraordinary caveat:
However, there is always the chance that some humans, as weird as we are, will have behavior patterns that flag as automated or bot activity. People may be dedicated to retweeting, do so at odd times of the day, or an inordinate amount. Other people may spam account handles or hashtags. In the case of this investigation, the finding remains the same even if the some of the accounts that were influential on Monday were human; their behaviour was coordinated and deliberately inauthentic, and done so to achieve a specific platform response.
This is an absolutely bizarre conclusion. Their headline states that, not just individual bots, but a botnet working in concert was directly responsible for pushing a hashtag to Westminster. But they then admit that they don’t actually know if any of the accounts are bots at all.
How you can say that the finding remains the same whether or not the accounts were human is quite staggering. There is a vast difference between enthusiastic individuals trying to get a hashtag trending and a coordinated botnet knowingly disseminating misinformation.
So I did what Logically did. When the hashtag started trending again, mocking Private Eye, I decided to examine the tweets on this hashtag, and see if I could see the same sort of behaviours and verify the robustness and extent of their claims.
Identifying and Quantifying Bot Influence
All of Logically’s claims are general - there are no numbers given to the weighting of supposed bot influence, just pretty pictures. They provide no indication of what percentage of retweets of a given tweet were due to bots, what percentage of accounts tweeting on the hashtag were fake, or even a rough indication of how many bots there were or how big the network would have been without them. When it comes to identifying bot accounts, they say the following:
“we found a small number of anonymous accounts spamming retweets back and forth to each other”
“the accounts that exercised the highest degree of influence on the network were “name+number” accounts, which often indicates a bot”
“Some of the accounts in the cluster retweeted the hashtag more than 200 times, with the highest-activity account more than 600 tweets”
“Analysis from TruthNest, a service that detects bot accounts, gave probabilities of 80 percent or higher that some of the highest-tweeting accounts in this cluster were bots”
Anonymity doesn’t demonstrate that an account is a bot, and plenty of accounts that appear to be real humans have in the past turned out to be bots (with AI-generated profile pictures). Similarly, real accounts are often hijacked and recruited into bot networks, so whether an account "appears to have a real name is hardly a strong claim.
“Name and number” accounts are sometimes bots, but again all this really indicates is that whoever signed up for an account accepted the Twitter-suggested username. There are many of reasons for this - retaining anonymity, not caring about usernames, or just being not-terribly tech savvy. [Edit: As pointed out to me on twitter, since 2017 Twitter doesn’t even give you the option to change - or even see - your username during signup, and buries it under the options]. Many high-profile real people in this space have “name and number” accounts.
That leaves points 3 & 4 - supposed high volume of tweets, and an external bot-identifying service: Truthnest. Could these together reliably identify bots?
So, I collated the latest tweets from the latest #KeepPrisonsSingleSex hashtag over the 24 hour period when Private Eye’s coverage prompted it to start trending again - around 10,000 tweets - and messaged the top 3 tweeters asking them if they are bots. The responses I got were:
“Hi. I'm not a bot. I'm a very pissed off woman in my early 40s.” - 141 tweets, 0% Truthnest bot score
“I’m definitely not a bot. […] I’m very passionate about women’s rights. And especially worried about prisons being the thin end of the wedge with regards to gov and others dismantling them.” - 170 tweets, 80% Truthnest bot score
“Absolutely fine - definitely not a bot 😂🤣🤣“- 246 tweets, 30% Truthnest bot score.
Another account - who also featured prominently in the graphs in the original Logically report - was 7th highest tweeter this time around, and has a Truthnest score of 60%. This account (PHughes74470229) I knew had denied being a bot:
Still, I messaged them anyway out of due diligence and received the reply: “No, a real person, and very angry at that. Why would anyone think women would not protest this?”
I even asked one of my Twitter mutuals who happened to be 6th (with 129 tweets) and they responded “Hi, no I'm not a bot” (“You nearly got a response of beep, bop,beep, does not compute.“).
These are all accounts who generated traffic similar to that analysed by Logically in getting the hashtag trending again - some are even the same accounts - and all at least protest they are indeed humans when asked directly. It turns out that humans are well capable of producing this level of traffic all on their own, entirely on their own initiative.
Volume of tweets, Truthnest score, dedication to retweeting and having a username full of numbers are not robust indicators of being a bot.
Indeed, digging further, there isn’t any obvious automated quality to this traffic - the tweets from high-volume tweeters don’t appear monotonically throughout the day or in unrealistic quantities. Taking one account as an example, around 100 tweets are produced between just before 9am and just after 10am. These occur at irregular intervals, with random gaps of thirty seconds to a minute between them. There’s then a break of a couple of hours before tweeting resumes again. While overly-consistent traffic would be suspicious, it is impossible to differentiate between someone going through their feed retweeting every time they see #KeepPrisonsSingleSex, and a bot tweeting at semi-random intervals.
Looking at this recent trending incident, here are some headline numbers:
About 75% of traffic was generated by 2,942 accounts, most of whom tweeted 10 times or less.
The remaining 25% of tweet/retweet traffic was generated by 34 accounts. Of these at least half - including the 3 highest volume tweeters - either insist they are real people, or are accounts I know to be real.
Of the highest volume tweeters, only 4 are name+number accounts, contributing around 0.2% to the total trend, and the highest volume one of those (responsible for about 0.1%, and previously highlighted in Logically’s report) is definitively not a bot.
So around 3000 people, seemingly sufficiently annoyed at being called “bots” were able to get Private Eye, #KeepPrisonsSingleSex and #NotABot trending together, and the majority of that trend was due to organic low-level interaction. A handful of dedicated retweeters were able to inflate the trend, but it is not at all a majority effect, and there is no adequate evidence to say that bots are responsible, least of all a botnet. Logically’s stated method of bot identification would misclassify some of the top tweeters, and in any event the contribution to the trend of high-tweeting name + number accounts was miniscule.
Now, I don’t doubt for one moment that there are bots sowing misinformation on social media, and identifying them can be hard, but what Logically have done is produce a partisan piece of disinformation with the sole intention of discrediting the Keep Prisons Single Sex campaign. They don’t robustly identify any bots, nor do they quantify the impact of those bots on the trend itself.
There is a massive difference between saying that dedicated campaigners have gamed Twitter’s “trending” by tweeting and retweeting as much as possible, and saying that this is the work of a coordinated botnet. The one is pretty much par for the course on social media, while the other is clandestine mass manipulation.
We’re left with handwaving insinuations about influence, and tweets being read out in the House of Lords that were “amplified”, but no indication by who or to what degree. Did a tweet get 1% of its retweets from bots? 5%? 100%? Of the 37,000 mentions, how many would they class as fake? Not only do they not say, they admit in the conclusion that they cannot say, but - incredibly - insist it makes no difference either way.
This story was quickly picked up by The New Statesman who left it to the reader to draw inferences between clandestine bot activity and the US religious right, while claiming that the primary driver of the trend was bots.
A supposed fact checking organisation is now a primary source of unchecked disinformation.
There’s a story here, and I for one hope that Private Eye revisit it and do a little more digging.
[edit:] For further reading, have a look at James O’Malley’s piece, which also digs into the recent trending tweets and casts a skeptical eye over the claims in the Logically.AI report.