I fully give credit to Twitter for upping its efforts in detecting and removing spam and abusive content – as I’ve noted previously, they have significantly improved in this area, with a range of new tools and initiatives in place to better protect users and enhance the platform.
But there is still a key area of note that Twitter needs to address - that being issue assessment and response to user reports, with far too many problems being rejected as non-violations, only to be revised in retrospect.
Here are some recent examples:
- News commentator Rochelle Richie reported abusive tweets leveled at her by Cesar Sayoc, the man who later went on to mail pipe bombs to various high profile political identities. Twitter, at the time of her report, told Richie that the tweets in question were not in violation of its rules, then later, after the story became big news, revised its position and said that, in fact, they were against its terms.
- Vox co-founder Matthew Yglesias recently had his personal details shared via tweet. Various users reported it, and Twitter told them all it was not a violation. Slate inquired about the decision, and Twitter then revised its position, saying it was a violation, and subsequently removed the tweet (though the user was not penalized).
- For months, Twitter refused to remove Alex Jones from its platform despite various clear violations of its policies, and the fact that every other platform had banned Jones for similar reasons. Twitter eventually did ban Jones, and has defended its handling of the case, but many have questioned the process where various complaints were essentially dismissed, only for Jones to be banned for the same – or similar – at a later stage.
The concern here is that these are just the high profiles cases, we’re not hearing about the many, many other Twitter reports that are rejected daily, despite them potentially being actual violations of the platform’s rules.
I’ll give you a personal example – earlier this year, our tweet monitoring uncovered this account (on the right) with the handle @SocialMediatday, which, at the time was an exact duplicate of our account, @socialmedia2day.
You can see that our, real, account has a lot more followers, but this new one was directly copying us, down to the images, the tweets – potentially confusing our audience, and likely for malicious purpose.
We reported this to Twitter, referring to its impersonation policy:
Guess what their response was:
“We've investigated the reported account and have determined that it is not in violation of Twitter’s impersonation policy.”
Seems like a pretty clear-cut case to me – we assume that Twitter’s inaction related to this element:
“In order to be impersonation, the account must also portray another person in a misleading or deceptive manner.”
The account, theoretically, hadn’t yet been used in a deceptive manner – the tweets were direct copies of our tweets, with links back to our content. It wasn’t like they’d used this to trick our audience - at least, not yet. But that’s likely the aim, which probably means that we need to monitor this account for a potential violation, which we can then report to Twitter at a later time, and maybe then they’ll act. Even though this is a clear example of direct impersonation.
Of course, I did ask Twitter about this, I sought more information on why this was not a violation. I got an auto-generated ‘case closed’ response.
So what’s the problem here – why are Twitter’s rules being misinterpreted by its own people, and what does that mean for the overall health of the platform?
The biggest problem for Twitter is likely cost. In order to assess and manage the massive amount of issue reports being submitted every day, social platforms outsource their moderation efforts to lower cost centers in third-world nations – the Philippines, in particular, has become a key hub for digital content moderators. That means you can get more work done, but there are interpretation and language gaps, along with speed expectations, which lead to errors.
Because of this, the platforms need to make their rules as black and white as possible – this is a violation, this is not. But that, in itself, puts a lot onto the individual interpreter of the report. Someone more accustomed with the nuance of such violations might pick up a key error, but those trained in the basics, often interpreting a language that’s not their native tongue, might miss it, because it’s not, on the surface, an overt issue.
Twitter itself doesn’t share a lot of insight about its content moderation process, but they did share this insight with Anmesty International earlier this year:
“Amnesty International also asked Twitter to share details about the content moderation process such as figures of the number of moderators employed per region and details about how content moderators are trained on gender and other identity-based discrimination, as well as international human rights standards more broadly. Twitter declined to share this information and stated: “We are well aware that ensuring our teams are taking into account gender-based nuance in behavior is essential. Every agent handling abuse reports receives in-depth trainings around the nuanced ways historically marginalized groups are harassed, intersectionality, and the impact that abuse online has on people's everyday lives, across a range of issues.” However, Twitter did not share details about what this training includes, nor did they share the number of moderators employed per region and language.”
As hinted here, scale may be a problem – earlier this year, Facebook reported that it now has 7,500 content moderators around the world, after adding some 2,500 more since 2017. This means they Facebook has added more moderators than Twitter’s total employee headcount within the last year alone. Facebook, of course, has a lot more users than Twitter, but still, it highlights the rising need for human moderation, an area where Twitter may be falling short, which may be leading to interpretation errors.
No doubt Twitter’s contractors are working with the company on this, putting in new benchmarks and expectations, but it is a concern that we’re hearing so many reports of such errors - which, as noted, hints at many more.
Twitter may need to re-train its staff, they may need to hire more, but clearly, this is an area where things are not working as they should, and where technological advances in themselves (e.g. AI detection) will not save the day.